2024-09-21T01:43:38.191 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T01:43:38.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T01:43:38.313 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913401' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIf1JNtjShPAFzTZHaAfYz8vZOjdN0ST2SBhYDBMqglLkAOqLtSTSZkhE/NCPMDiR0XEjw00ADwW2Ew234N6mis= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGBNzZlIcbH8Fd1CXVfshWy3KNy6/9KXUyuHh75Aqo7Yumc33qnC47n1h1+bdJOeyCIv6Y5K+yh7H0BXEY/tI0= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLm1XQPVYzkSJfSAkZusQWiN4OWKD1ar+izdm+Bb3Zwp7knho5+Jx2kwglUEgMXXqWLmFXYpQnIHp3lt9xDyfkw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T01:43:38.313 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T01:43:38.314 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T01:43:38.314 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T01:43:38.315 INFO:teuthology.task.internal:Checking packages... 2024-09-21T01:43:38.335 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T01:43:38.336 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T01:43:38.336 INFO:teuthology.packaging:ref: None 2024-09-21T01:43:38.336 INFO:teuthology.packaging:tag: None 2024-09-21T01:43:38.336 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:43:38.336 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:43:38.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:43:38.472 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T01:43:38.473 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T01:43:38.479 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T01:43:38.480 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T01:43:38.487 INFO:teuthology.task.internal:Saving configuration 2024-09-21T01:43:38.506 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T01:43:38.512 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T01:43:38.535 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:38:40.445911', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIf1JNtjShPAFzTZHaAfYz8vZOjdN0ST2SBhYDBMqglLkAOqLtSTSZkhE/NCPMDiR0XEjw00ADwW2Ew234N6mis='} 2024-09-21T01:43:38.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:38:40.449059', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGBNzZlIcbH8Fd1CXVfshWy3KNy6/9KXUyuHh75Aqo7Yumc33qnC47n1h1+bdJOeyCIv6Y5K+yh7H0BXEY/tI0='} 2024-09-21T01:43:38.580 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:38:40.447662', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLm1XQPVYzkSJfSAkZusQWiN4OWKD1ar+izdm+Bb3Zwp7knho5+Jx2kwglUEgMXXqWLmFXYpQnIHp3lt9xDyfkw='} 2024-09-21T01:43:38.581 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T01:43:38.587 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T01:43:38.587 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T01:43:38.587 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T01:43:38.587 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T01:43:38.665 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb503bdf520>, signals=[15]) 2024-09-21T01:43:38.665 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T01:43:38.671 INFO:teuthology.task.internal:Opening connections... 2024-09-21T01:43:38.671 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T01:43:38.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:43:38.746 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T01:43:38.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:43:38.817 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-09-21T01:43:38.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:43:38.889 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T01:43:38.895 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-21T01:43:38.913 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-21T01:43:38.914 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-21T01:43:38.971 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:43:38.972 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:43:38.973 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-21T01:43:38.990 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-21T01:43:39.009 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-21T01:43:39.009 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-21T01:43:39.066 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:43:39.067 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:43:39.068 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:43:39.068 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-21T01:43:39.090 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-09-21T01:43:39.108 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-09-21T01:43:39.108 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-09-21T01:43:39.166 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:43:39.167 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:43:39.168 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:43:39.168 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:43:39.168 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-09-21T01:43:39.190 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T01:43:39.205 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T01:43:39.210 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T01:43:39.211 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:43:39.213 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:43:39.215 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:43:39.232 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T01:43:39.241 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T01:43:39.241 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:43:39.269 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:43:39.272 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:43:39.358 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T01:43:39.433 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T01:43:39.446 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T01:43:39.447 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T01:43:39.447 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T01:43:39.447 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:43:39.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:43:39.463 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-21T01:43:39.521 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-21T01:43:39.521 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-503.el9.x86_64 2024-09-21T01:43:39.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-09-21T01:43:43.775 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-21T01:43:48.333 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 20 MB 00:03 2024-09-21T01:43:52.742 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T01:43:54.168 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-21T01:43:54.989 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-21T01:44:00.411 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-21T01:44:02.341 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T01:44:02.342 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T01:44:02.441 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T01:44:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:44:02.451 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 79 M 2024-09-21T01:44:02.452 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 126 M 2024-09-21T01:44:02.452 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-21T01:44:02.595 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 372 kB/s | 36 kB 00:00 2024-09-21T01:44:02.978 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-21T01:44:03.437 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-21T01:44:03.738 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-21T01:44:03.738 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:44:03.738 INFO:teuthology.orchestra.run.smithi092.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-21T01:44:04.420 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-21T01:44:04.768 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-21T01:44:04.768 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-21T01:44:05.721 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-21T01:44:05.722 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-21T01:44:08.171 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-21T01:44:08.879 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T01:44:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:44:11.696 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:44:11.825 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:44:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:44:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:44:21.575 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:44:59.821 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:44:59.861 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:45:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:45:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T01:45:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:45:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:45:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:45:01.051 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-21T01:45:01.380 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T01:45:01.500 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T01:45:01.501 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T01:45:01.501 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T01:45:01.501 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T01:45:01.502 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T01:45:01.502 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T01:45:01.502 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T01:45:01.502 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T01:45:01.561 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T01:45:01.561 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T01:45:01.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-21T01:45:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T01:45:01.595 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T01:45:01.596 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-21T01:45:01.596 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:45:01.596 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T01:45:01.596 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T01:45:01.596 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-508.el9.x86_64 2024-09-21T01:45:01.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T01:45:02.222 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-21T01:45:03.484 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T01:45:03.523 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-21T01:45:03.525 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-21T01:45:03.541 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.M5ev9IvhiQ 2024-09-21T01:45:03.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.M5ev9IvhiQ 2024-09-21T01:45:03.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.M5ev9IvhiQ 2024-09-21T01:45:03.764 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.M5ev9IvhiQ is 6KB 2024-09-21T01:45:03.810 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.M5ev9IvhiQ 2024-09-21T01:45:03.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T01:45:03.902 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T01:45:03.902 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T01:45:03.902 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T01:45:03.902 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T01:45:03.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T01:45:04.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-21T01:45:34.077 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T01:45:34.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-21T01:45:34.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:45:52.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-21T01:46:01.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-21T01:46:01.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:47:01.626 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T01:47:13.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-21T01:47:13.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:47:14.291 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-21T01:47:14.626 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-21T01:47:14.626 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T01:47:14.626 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-21T01:47:14.641 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-508.el9.x86_64 2024-09-21T01:47:14.642 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T01:47:14.642 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T01:47:14.642 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T01:47:15.642 DEBUG:teuthology.parallel:result is None 2024-09-21T01:47:15.643 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T01:47:15.649 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T01:47:15.649 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:47:15.651 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:47:15.654 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:47:15.673 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T01:47:15.679 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T01:47:15.695 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T01:47:15.695 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:47:15.709 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:47:15.712 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:47:15.763 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T01:47:15.770 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T01:47:15.770 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:47:15.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:47:15.791 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:47:15.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:47:15.814 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:47:15.816 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:47:15.833 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:47:15.880 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:47:15.901 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:47:15.904 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:47:15.919 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:47:15.921 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:47:15.939 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:47:15.940 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T01:47:15.947 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T01:47:15.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:47:15.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:47:15.952 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:47:15.980 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T01:47:15.988 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T01:47:15.989 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:47:16.019 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:47:16.021 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:47:16.038 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:47:16.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:47:16.169 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:47:16.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:47:16.298 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:47:16.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:47:16.367 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:47:16.409 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:47:16.458 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:47:16.541 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:47:16.583 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:47:16.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:47:16.648 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:47:16.695 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:47:16.755 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:47:16.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:47:16.841 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:47:16.841 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:47:16.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-21T01:47:16.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-21T01:47:16.912 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-09-21T01:47:16.943 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:47:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:47:16.982 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:47:17.366 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T01:47:17.374 INFO:teuthology.task.internal:Starting timer... 2024-09-21T01:47:17.374 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T01:47:17.399 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T01:47:17.425 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T01:47:17.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-21T01:47:17.486 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-21T01:47:17.488 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-09-21T01:47:17.558 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-09-21T01:47:17.560 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-09-21T01:47:17.605 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-09-21T01:47:17.607 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T01:47:17.607 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-21T01:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-21T01:47:17.639 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-09-21T01:47:17.663 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-09-21T01:47:17.663 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-09-21T01:47:17.682 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-09-21T01:47:17.682 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-09-21T01:47:17.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:47:17.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:47:17.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:47:17.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:47:17.737 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:47:17.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:47:17.765 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T01:47:17.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:47:17.809 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:47:17.841 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:47:17.872 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T01:47:17.881 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T01:47:17.893 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T01:47:17.894 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T01:51:14.797 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-09-21T01:51:14.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-21T01:51:14.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:51:14.874 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-21T01:51:14.949 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-21T01:51:14.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T01:51:14.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:51:15.021 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-21T01:51:15.098 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-21T01:51:15.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-21T01:51:15.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:51:15.172 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-21T01:51:15.250 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-21T01:51:15.250 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T01:51:15.264 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T01:51:15.264 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:51:15.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:51:15.267 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:51:15.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:51:15.269 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:51:15.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:51:15.294 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:51:15.295 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:51:15.308 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:51:15.309 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:51:15.318 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:51:15.332 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-21T01:51:15.333 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:51:15.333 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-09-21T01:51:15.343 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-21T01:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-09-21T01:51:15.356 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-09-21T01:51:15.357 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:51:15.358 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:51:15.367 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-09-21T01:51:15.370 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:51:15.372 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:51:15.381 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:51:15.395 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:51:15.424 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:51:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:51:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-21T01:51:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.428 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:51:15.431 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:51:15.431 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-21T01:51:15.431 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.431 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.431 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.431 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.452 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:51:15.455 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:51:15.455 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-09-21T01:51:15.455 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.456 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.456 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.456 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:51:15.457 INFO:teuthology.run_tasks:Running task install... 2024-09-21T01:51:15.464 DEBUG:teuthology.task.install:project ceph 2024-09-21T01:51:15.464 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T01:51:15.465 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T01:51:15.465 INFO:teuthology.task.install:Using flavor: default 2024-09-21T01:51:15.470 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T01:51:15.470 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T01:51:15.470 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:51:15.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:51:15.472 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:51:15.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:51:15.473 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:51:15.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:51:15.617 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:51:15.618 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:51:15.635 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:51:15.636 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:51:15.639 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:51:15.639 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:51:15.729 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:51:15.729 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:51:15.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:51:15.749 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:51:15.749 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:51:15.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:51:15.754 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:51:15.754 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:51:15.754 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:51:15.755 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:51:15.755 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:51:15.775 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:51:15.775 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:51:15.781 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:51:15.782 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:51:15.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:51:15.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:51:15.848 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:51:15.895 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:51:15.915 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:51:15.923 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:51:15.959 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-21T01:51:15.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-21T01:51:15.979 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-09-21T01:51:15.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-09-21T01:51:15.987 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-09-21T01:51:15.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-09-21T01:51:16.181 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-21T01:51:16.205 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-09-21T01:51:16.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:51:16.211 INFO:teuthology.orchestra.run.smithi136.stdout:45 files removed 2024-09-21T01:51:16.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:51:16.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:51:16.762 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T01:51:16.775 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-21T01:51:16.776 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-21T01:51:17.046 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-21T01:51:17.067 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T01:51:17.068 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T01:51:17.330 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T01:51:17.336 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-21T01:51:17.343 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T01:51:18.009 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T01:51:18.019 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T01:51:19.064 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.3 MB 00:01 2024-09-21T01:51:20.529 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-21T01:51:20.998 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-21T01:51:22.669 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-21T01:51:26.146 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-21T01:51:27.149 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-21T01:51:27.805 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-21T01:51:28.369 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-21T01:51:28.726 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T01:51:28.757 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-21T01:51:29.678 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T01:51:29.828 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-21T01:51:30.974 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T01:51:34.281 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-21T01:51:34.993 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T01:51:36.427 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:51:36.428 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:51:36.438 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-21T01:51:36.440 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:51:36.441 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:51:36.442 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:51:36.443 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:51:36.445 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:51:36.448 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:51:36.449 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:51:36.511 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-09-21T01:51:36.529 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-09-21T01:51:36.529 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-09-21T01:51:36.529 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-09-21T01:51:36.529 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-09-21T01:51:36.529 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:51:36.530 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:51:36.531 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:51:36.532 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:51:36.533 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:51:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:51:36.535 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:51:36.536 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:51:36.537 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:51:36.538 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:51:36.539 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:51:36.540 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:51:36.541 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-09-21T01:51:36.542 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-09-21T01:51:36.542 INFO:teuthology.orchestra.run.smithi136.stdout:Install 149 Packages 2024-09-21T01:51:36.542 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 8 Packages 2024-09-21T01:51:36.542 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:51:36.542 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 228 M 2024-09-21T01:51:36.542 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-09-21T01:51:37.174 INFO:teuthology.orchestra.run.smithi136.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-21T01:51:37.178 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:51:37.178 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:51:37.190 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:51:37.191 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:51:37.191 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:51:37.192 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:51:37.194 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:51:37.196 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:51:37.197 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:51:37.259 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-21T01:51:37.269 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:51:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:51:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:51:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:51:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:51:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:51:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:51:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:51:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:51:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:51:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:51:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:51:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout:Install 149 Packages 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-09-21T01:51:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:51:37.283 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 228 M 2024-09-21T01:51:37.283 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-21T01:51:37.358 INFO:teuthology.orchestra.run.smithi136.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.7 MB/s | 873 kB 00:00 2024-09-21T01:51:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.7 MB/s | 148 kB 00:00 2024-09-21T01:51:37.525 INFO:teuthology.orchestra.run.smithi136.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-21T01:51:37.625 INFO:teuthology.orchestra.run.smithi136.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 15 MB/s | 1.5 MB 00:00 2024-09-21T01:51:37.701 INFO:teuthology.orchestra.run.smithi136.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.8 MB/s | 2.3 MB 00:00 2024-09-21T01:51:37.951 INFO:teuthology.orchestra.run.smithi136.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 15 MB/s | 5.0 MB 00:00 2024-09-21T01:51:38.068 INFO:teuthology.orchestra.run.smithi092.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-21T01:51:38.151 INFO:teuthology.orchestra.run.smithi136.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T01:51:38.202 INFO:teuthology.orchestra.run.smithi136.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 506 kB/s | 25 kB 00:00 2024-09-21T01:51:38.511 INFO:teuthology.orchestra.run.smithi092.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.9 MB/s | 873 kB 00:00 2024-09-21T01:51:38.580 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:51:38.581 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:51:38.592 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:51:38.593 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:51:38.594 INFO:teuthology.orchestra.run.smithi092.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.7 MB/s | 148 kB 00:00 2024-09-21T01:51:38.594 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:51:38.595 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:51:38.597 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:51:38.599 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:51:38.600 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:51:38.661 INFO:teuthology.orchestra.run.smithi092.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 7.7 MB/s | 5.5 MB 00:00 2024-09-21T01:51:38.662 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:51:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:51:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:51:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-09-21T01:51:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-21T01:51:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:51:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:51:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:51:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:51:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:51:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:51:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:51:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:51:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:51:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:Install 149 Packages 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 8 Packages 2024-09-21T01:51:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:51:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 228 M 2024-09-21T01:51:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-21T01:51:38.845 INFO:teuthology.orchestra.run.smithi092.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-09-21T01:51:39.143 INFO:teuthology.orchestra.run.smithi136.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 12 MB/s | 18 MB 00:01 2024-09-21T01:51:39.218 INFO:teuthology.orchestra.run.smithi136.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.0 MB/s | 226 kB 00:00 2024-09-21T01:51:39.237 INFO:teuthology.orchestra.run.smithi092.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.0 MB 00:00 2024-09-21T01:51:39.272 INFO:teuthology.orchestra.run.smithi099.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-09-21T01:51:39.302 INFO:teuthology.orchestra.run.smithi136.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 325 kB/s | 27 kB 00:00 2024-09-21T01:51:39.419 INFO:teuthology.orchestra.run.smithi136.stdout:(13/157): ceph-radosgw-19.3.0-5120.g18432218.el 7.1 MB/s | 10 MB 00:01 2024-09-21T01:51:39.444 INFO:teuthology.orchestra.run.smithi136.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 224 kB/s | 32 kB 00:00 2024-09-21T01:51:39.504 INFO:teuthology.orchestra.run.smithi092.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-09-21T01:51:39.536 INFO:teuthology.orchestra.run.smithi136.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.7 MB/s | 158 kB 00:00 2024-09-21T01:51:39.578 INFO:teuthology.orchestra.run.smithi136.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 4.7 MB/s | 766 kB 00:00 2024-09-21T01:51:39.653 INFO:teuthology.orchestra.run.smithi136.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.1 MB/s | 127 kB 00:00 2024-09-21T01:51:39.837 INFO:teuthology.orchestra.run.smithi136.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 1.9 MB/s | 507 kB 00:00 2024-09-21T01:51:39.912 INFO:teuthology.orchestra.run.smithi136.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 602 kB/s | 45 kB 00:00 2024-09-21T01:51:39.996 INFO:teuthology.orchestra.run.smithi136.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T01:51:40.022 INFO:teuthology.orchestra.run.smithi099.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.1 MB/s | 873 kB 00:00 2024-09-21T01:51:40.029 INFO:teuthology.orchestra.run.smithi092.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 9.6 MB/s | 20 MB 00:02 2024-09-21T01:51:40.080 INFO:teuthology.orchestra.run.smithi136.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T01:51:40.104 INFO:teuthology.orchestra.run.smithi092.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 337 kB/s | 25 kB 00:00 2024-09-21T01:51:40.156 INFO:teuthology.orchestra.run.smithi099.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.1 MB/s | 148 kB 00:00 2024-09-21T01:51:40.247 INFO:teuthology.orchestra.run.smithi136.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 1.9 MB/s | 321 kB 00:00 2024-09-21T01:51:40.579 INFO:teuthology.orchestra.run.smithi092.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 13 MB/s | 18 MB 00:01 2024-09-21T01:51:40.629 INFO:teuthology.orchestra.run.smithi092.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.5 MB/s | 226 kB 00:00 2024-09-21T01:51:40.680 INFO:teuthology.orchestra.run.smithi092.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 541 kB/s | 27 kB 00:00 2024-09-21T01:51:40.689 INFO:teuthology.orchestra.run.smithi099.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 4.3 MB/s | 2.3 MB 00:00 2024-09-21T01:51:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 638 kB/s | 32 kB 00:00 2024-09-21T01:51:40.747 INFO:teuthology.orchestra.run.smithi136.stdout:(23/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T01:51:40.772 INFO:teuthology.orchestra.run.smithi136.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 584 kB/s | 307 kB 00:00 2024-09-21T01:51:40.814 INFO:teuthology.orchestra.run.smithi092.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T01:51:40.847 INFO:teuthology.orchestra.run.smithi136.stdout:(25/157): librgw2-19.3.0-5120.g18432218.el9.x86 4.4 MB/s | 5.3 MB 00:01 2024-09-21T01:51:40.872 INFO:teuthology.orchestra.run.smithi092.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T01:51:40.881 INFO:teuthology.orchestra.run.smithi136.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 796 kB/s | 86 kB 00:00 2024-09-21T01:51:40.906 INFO:teuthology.orchestra.run.smithi136.stdout:(27/157): python3-rgw-19.3.0-5120.g18432218.el9 624 kB/s | 99 kB 00:00 2024-09-21T01:51:40.923 INFO:teuthology.orchestra.run.smithi099.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-09-21T01:51:40.930 INFO:teuthology.orchestra.run.smithi092.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T01:51:40.940 INFO:teuthology.orchestra.run.smithi136.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.8 MB/s | 170 kB 00:00 2024-09-21T01:51:40.965 INFO:teuthology.orchestra.run.smithi136.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 586 kB/s | 34 kB 00:00 2024-09-21T01:51:41.014 INFO:teuthology.orchestra.run.smithi092.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 6.0 MB/s | 507 kB 00:00 2024-09-21T01:51:41.015 INFO:teuthology.orchestra.run.smithi136.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.0 MB/s | 154 kB 00:00 2024-09-21T01:51:41.191 INFO:teuthology.orchestra.run.smithi136.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 9.0 MB/s | 3.1 MB 00:00 2024-09-21T01:51:41.299 INFO:teuthology.orchestra.run.smithi136.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.6 MB/s | 292 kB 00:00 2024-09-21T01:51:41.400 INFO:teuthology.orchestra.run.smithi136.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 493 kB/s | 49 kB 00:00 2024-09-21T01:51:41.500 INFO:teuthology.orchestra.run.smithi136.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 165 kB/s | 16 kB 00:00 2024-09-21T01:51:41.573 INFO:teuthology.orchestra.run.smithi092.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.5 MB/s | 5.3 MB 00:00 2024-09-21T01:51:41.583 INFO:teuthology.orchestra.run.smithi136.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.3 MB/s | 285 kB 00:00 2024-09-21T01:51:41.665 INFO:teuthology.orchestra.run.smithi092.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 494 kB/s | 45 kB 00:00 2024-09-21T01:51:41.750 INFO:teuthology.orchestra.run.smithi136.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.6 MB/s | 784 kB 00:00 2024-09-21T01:51:41.757 INFO:teuthology.orchestra.run.smithi092.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.6 MB/s | 148 kB 00:00 2024-09-21T01:51:41.790 INFO:teuthology.orchestra.run.smithi099.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 5.7 MB/s | 5.0 MB 00:00 2024-09-21T01:51:41.809 INFO:teuthology.orchestra.run.smithi136.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 316 kB/s | 18 kB 00:00 2024-09-21T01:51:41.832 INFO:teuthology.orchestra.run.smithi092.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T01:51:41.915 INFO:teuthology.orchestra.run.smithi092.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T01:51:41.993 INFO:teuthology.orchestra.run.smithi092.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.9 MB/s | 307 kB 00:00 2024-09-21T01:51:42.018 INFO:teuthology.orchestra.run.smithi136.stdout:(38/157): ceph-mgr-dashboard-19.3.0-5120.g18432 13 MB/s | 14 MB 00:01 2024-09-21T01:51:42.060 INFO:teuthology.orchestra.run.smithi092.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T01:51:42.123 INFO:teuthology.orchestra.run.smithi099.stdout:(7/157): ceph-base-19.3.0-5120.g18432218.el9.x8 1.8 MB/s | 5.5 MB 00:03 2024-09-21T01:51:42.385 INFO:teuthology.orchestra.run.smithi136.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 139 kB/s | 80 kB 00:00 2024-09-21T01:51:42.419 INFO:teuthology.orchestra.run.smithi092.stdout:(25/157): ceph-test-19.3.0-5120.g18432218.el9.x 17 MB/s | 40 MB 00:02 2024-09-21T01:51:42.444 INFO:teuthology.orchestra.run.smithi092.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 225 kB/s | 86 kB 00:00 2024-09-21T01:51:42.519 INFO:teuthology.orchestra.run.smithi092.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 170 kB 00:00 2024-09-21T01:51:42.560 INFO:teuthology.orchestra.run.smithi136.stdout:(40/157): ceph-mgr-diskprediction-local-19.3.0- 4.8 MB/s | 7.4 MB 00:01 2024-09-21T01:51:42.578 INFO:teuthology.orchestra.run.smithi092.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 587 kB/s | 34 kB 00:00 2024-09-21T01:51:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 764 kB/s | 191 kB 00:00 2024-09-21T01:51:42.636 INFO:teuthology.orchestra.run.smithi092.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 14 MB/s | 3.1 MB 00:00 2024-09-21T01:51:42.661 INFO:teuthology.orchestra.run.smithi092.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T01:51:42.699 INFO:teuthology.orchestra.run.smithi099.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 20 MB/s | 18 MB 00:00 2024-09-21T01:51:42.765 INFO:teuthology.orchestra.run.smithi099.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 379 kB/s | 25 kB 00:00 2024-09-21T01:51:42.860 INFO:teuthology.orchestra.run.smithi136.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 951 kB/s | 801 kB 00:00 2024-09-21T01:51:42.969 INFO:teuthology.orchestra.run.smithi136.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-21T01:51:43.028 INFO:teuthology.orchestra.run.smithi136.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-21T01:51:43.028 INFO:teuthology.orchestra.run.smithi092.stdout:(31/157): ceph-radosgw-19.3.0-5120.g18432218.el 3.0 MB/s | 10 MB 00:03 2024-09-21T01:51:43.078 INFO:teuthology.orchestra.run.smithi136.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-21T01:51:43.103 INFO:teuthology.orchestra.run.smithi136.stdout:(46/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 466 kB/s | 253 kB 00:00 2024-09-21T01:51:43.137 INFO:teuthology.orchestra.run.smithi136.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-21T01:51:43.195 INFO:teuthology.orchestra.run.smithi092.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 1.7 MB/s | 292 kB 00:00 2024-09-21T01:51:43.196 INFO:teuthology.orchestra.run.smithi136.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T01:51:43.262 INFO:teuthology.orchestra.run.smithi136.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-09-21T01:51:43.313 INFO:teuthology.orchestra.run.smithi136.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 914 kB/s | 107 kB 00:00 2024-09-21T01:51:43.354 INFO:teuthology.orchestra.run.smithi092.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 11 MB/s | 7.4 MB 00:00 2024-09-21T01:51:43.371 INFO:teuthology.orchestra.run.smithi136.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-21T01:51:43.379 INFO:teuthology.orchestra.run.smithi092.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 268 kB/s | 49 kB 00:00 2024-09-21T01:51:43.430 INFO:teuthology.orchestra.run.smithi136.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T01:51:43.497 INFO:teuthology.orchestra.run.smithi136.stdout:(53/157): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2024-09-21T01:51:43.513 INFO:teuthology.orchestra.run.smithi092.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 16 MB/s | 14 MB 00:00 2024-09-21T01:51:43.514 INFO:teuthology.orchestra.run.smithi136.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 229 kB/s | 33 kB 00:00 2024-09-21T01:51:43.538 INFO:teuthology.orchestra.run.smithi092.stdout:(36/157): ceph-prometheus-alerts-19.3.0-5120.g1 90 kB/s | 16 kB 00:00 2024-09-21T01:51:43.563 INFO:teuthology.orchestra.run.smithi092.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 1.5 MB/s | 285 kB 00:00 2024-09-21T01:51:43.598 INFO:teuthology.orchestra.run.smithi136.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-09-21T01:51:43.605 INFO:teuthology.orchestra.run.smithi092.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 8.4 MB/s | 784 kB 00:00 2024-09-21T01:51:43.615 INFO:teuthology.orchestra.run.smithi136.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-21T01:51:43.622 INFO:teuthology.orchestra.run.smithi092.stdout:(39/157): cephfs-top-19.3.0-5120.g18432218.el9. 219 kB/s | 18 kB 00:00 2024-09-21T01:51:43.706 INFO:teuthology.orchestra.run.smithi136.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-21T01:51:43.714 INFO:teuthology.orchestra.run.smithi092.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 530 kB/s | 80 kB 00:00 2024-09-21T01:51:43.732 INFO:teuthology.orchestra.run.smithi136.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-21T01:51:43.814 INFO:teuthology.orchestra.run.smithi092.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 995 kB/s | 191 kB 00:00 2024-09-21T01:51:43.820 INFO:teuthology.orchestra.run.smithi136.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 401 kB/s | 45 kB 00:00 2024-09-21T01:51:43.839 INFO:teuthology.orchestra.run.smithi092.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-21T01:51:43.861 INFO:teuthology.orchestra.run.smithi136.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.0 MB/s | 662 kB 00:00 2024-09-21T01:51:43.881 INFO:teuthology.orchestra.run.smithi092.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-21T01:51:43.907 INFO:teuthology.orchestra.run.smithi092.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T01:51:43.940 INFO:teuthology.orchestra.run.smithi092.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T01:51:43.966 INFO:teuthology.orchestra.run.smithi092.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T01:51:43.995 INFO:teuthology.orchestra.run.smithi136.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 183 kB/s | 24 kB 00:00 2024-09-21T01:51:43.999 INFO:teuthology.orchestra.run.smithi092.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-09-21T01:51:44.020 INFO:teuthology.orchestra.run.smithi136.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-21T01:51:44.024 INFO:teuthology.orchestra.run.smithi092.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T01:51:44.079 INFO:teuthology.orchestra.run.smithi136.stdout:(63/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-09-21T01:51:44.104 INFO:teuthology.orchestra.run.smithi136.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-21T01:51:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 987 kB/s | 107 kB 00:00 2024-09-21T01:51:44.138 INFO:teuthology.orchestra.run.smithi136.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T01:51:44.158 INFO:teuthology.orchestra.run.smithi092.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-21T01:51:44.183 INFO:teuthology.orchestra.run.smithi092.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-21T01:51:44.196 INFO:teuthology.orchestra.run.smithi136.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T01:51:44.198 INFO:teuthology.orchestra.run.smithi099.stdout:(10/157): ceph-test-19.3.0-5120.g18432218.el9.x 28 MB/s | 40 MB 00:01 2024-09-21T01:51:44.209 INFO:teuthology.orchestra.run.smithi092.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T01:51:44.222 INFO:teuthology.orchestra.run.smithi136.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-09-21T01:51:44.234 INFO:teuthology.orchestra.run.smithi092.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-21T01:51:44.249 INFO:teuthology.orchestra.run.smithi099.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T01:51:44.268 INFO:teuthology.orchestra.run.smithi092.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-21T01:51:44.293 INFO:teuthology.orchestra.run.smithi092.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 309 kB/s | 18 kB 00:00 2024-09-21T01:51:44.299 INFO:teuthology.orchestra.run.smithi099.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 543 kB/s | 27 kB 00:00 2024-09-21T01:51:44.319 INFO:teuthology.orchestra.run.smithi092.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T01:51:44.341 INFO:teuthology.orchestra.run.smithi099.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 765 kB/s | 32 kB 00:00 2024-09-21T01:51:44.343 INFO:teuthology.orchestra.run.smithi092.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T01:51:44.356 INFO:teuthology.orchestra.run.smithi136.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T01:51:44.385 INFO:teuthology.orchestra.run.smithi092.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T01:51:44.408 INFO:teuthology.orchestra.run.smithi099.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T01:51:44.410 INFO:teuthology.orchestra.run.smithi092.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-21T01:51:44.444 INFO:teuthology.orchestra.run.smithi092.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-21T01:51:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T01:51:44.495 INFO:teuthology.orchestra.run.smithi092.stdout:(61/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-09-21T01:51:44.508 INFO:teuthology.orchestra.run.smithi099.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T01:51:44.520 INFO:teuthology.orchestra.run.smithi092.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-21T01:51:44.545 INFO:teuthology.orchestra.run.smithi092.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-21T01:51:44.559 INFO:teuthology.orchestra.run.smithi099.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 9.9 MB/s | 507 kB 00:00 2024-09-21T01:51:44.570 INFO:teuthology.orchestra.run.smithi092.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T01:51:44.604 INFO:teuthology.orchestra.run.smithi092.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-21T01:51:44.629 INFO:teuthology.orchestra.run.smithi092.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-21T01:51:44.654 INFO:teuthology.orchestra.run.smithi092.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T01:51:44.738 INFO:teuthology.orchestra.run.smithi092.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T01:51:44.751 INFO:teuthology.orchestra.run.smithi099.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 28 MB/s | 5.3 MB 00:00 2024-09-21T01:51:44.772 INFO:teuthology.orchestra.run.smithi092.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T01:51:44.801 INFO:teuthology.orchestra.run.smithi099.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 907 kB/s | 45 kB 00:00 2024-09-21T01:51:44.805 INFO:teuthology.orchestra.run.smithi092.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-21T01:51:44.852 INFO:teuthology.orchestra.run.smithi099.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T01:51:44.864 INFO:teuthology.orchestra.run.smithi092.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 19 MB/s | 4.9 MB 00:00 2024-09-21T01:51:44.889 INFO:teuthology.orchestra.run.smithi092.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T01:51:44.901 INFO:teuthology.orchestra.run.smithi099.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T01:51:44.931 INFO:teuthology.orchestra.run.smithi092.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-21T01:51:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T01:51:44.956 INFO:teuthology.orchestra.run.smithi092.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T01:51:45.007 INFO:teuthology.orchestra.run.smithi092.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T01:51:45.019 INFO:teuthology.orchestra.run.smithi099.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T01:51:45.069 INFO:teuthology.orchestra.run.smithi099.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 2.0 MB/s | 99 kB 00:00 2024-09-21T01:51:45.082 INFO:teuthology.orchestra.run.smithi092.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-21T01:51:45.107 INFO:teuthology.orchestra.run.smithi092.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T01:51:45.127 INFO:teuthology.orchestra.run.smithi099.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 86 kB 00:00 2024-09-21T01:51:45.134 INFO:teuthology.orchestra.run.smithi092.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T01:51:45.157 INFO:teuthology.orchestra.run.smithi092.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-21T01:51:45.182 INFO:teuthology.orchestra.run.smithi092.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T01:51:45.208 INFO:teuthology.orchestra.run.smithi092.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T01:51:45.233 INFO:teuthology.orchestra.run.smithi092.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-21T01:51:45.286 INFO:teuthology.orchestra.run.smithi099.stdout:(26/157): ceph-radosgw-19.3.0-5120.g18432218.el 3.3 MB/s | 10 MB 00:03 2024-09-21T01:51:45.309 INFO:teuthology.orchestra.run.smithi092.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-21T01:51:45.334 INFO:teuthology.orchestra.run.smithi092.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T01:51:45.353 INFO:teuthology.orchestra.run.smithi099.stdout:(27/157): rbd-mirror-19.3.0-5120.g18432218.el9. 14 MB/s | 3.1 MB 00:00 2024-09-21T01:51:45.359 INFO:teuthology.orchestra.run.smithi092.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T01:51:45.385 INFO:teuthology.orchestra.run.smithi092.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T01:51:45.395 INFO:teuthology.orchestra.run.smithi099.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.5 MB/s | 170 kB 00:00 2024-09-21T01:51:45.410 INFO:teuthology.orchestra.run.smithi092.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-21T01:51:45.412 INFO:teuthology.orchestra.run.smithi099.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 588 kB/s | 34 kB 00:00 2024-09-21T01:51:45.435 INFO:teuthology.orchestra.run.smithi092.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-21T01:51:45.454 INFO:teuthology.orchestra.run.smithi099.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T01:51:45.460 INFO:teuthology.orchestra.run.smithi092.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T01:51:45.485 INFO:teuthology.orchestra.run.smithi092.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T01:51:45.511 INFO:teuthology.orchestra.run.smithi092.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T01:51:45.536 INFO:teuthology.orchestra.run.smithi092.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T01:51:45.547 INFO:teuthology.orchestra.run.smithi136.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.5 MB/s | 4.9 MB 00:01 2024-09-21T01:51:45.611 INFO:teuthology.orchestra.run.smithi092.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-21T01:51:45.637 INFO:teuthology.orchestra.run.smithi092.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-21T01:51:45.653 INFO:teuthology.orchestra.run.smithi092.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T01:51:45.679 INFO:teuthology.orchestra.run.smithi092.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-21T01:51:45.696 INFO:teuthology.orchestra.run.smithi092.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 856 kB/s | 36 kB 00:00 2024-09-21T01:51:45.712 INFO:teuthology.orchestra.run.smithi092.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T01:51:45.739 INFO:teuthology.orchestra.run.smithi136.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 658 kB/s | 910 kB 00:01 2024-09-21T01:51:45.746 INFO:teuthology.orchestra.run.smithi092.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T01:51:45.763 INFO:teuthology.orchestra.run.smithi092.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T01:51:45.788 INFO:teuthology.orchestra.run.smithi092.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-21T01:51:45.805 INFO:teuthology.orchestra.run.smithi092.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T01:51:45.831 INFO:teuthology.orchestra.run.smithi092.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-21T01:51:45.838 INFO:teuthology.orchestra.run.smithi099.stdout:(31/157): ceph-mgr-dashboard-19.3.0-5120.g18432 33 MB/s | 14 MB 00:00 2024-09-21T01:51:45.856 INFO:teuthology.orchestra.run.smithi136.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 445 kB/s | 137 kB 00:00 2024-09-21T01:51:45.896 INFO:teuthology.orchestra.run.smithi099.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T01:51:45.946 INFO:teuthology.orchestra.run.smithi099.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 990 kB/s | 49 kB 00:00 2024-09-21T01:51:46.048 INFO:teuthology.orchestra.run.smithi092.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-21T01:51:46.064 INFO:teuthology.orchestra.run.smithi092.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-09-21T01:51:46.114 INFO:teuthology.orchestra.run.smithi092.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-21T01:51:46.138 INFO:teuthology.orchestra.run.smithi099.stdout:(34/157): ceph-common-19.3.0-5120.g18432218.el9 2.8 MB/s | 20 MB 00:07 2024-09-21T01:51:46.140 INFO:teuthology.orchestra.run.smithi092.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-21T01:51:46.165 INFO:teuthology.orchestra.run.smithi092.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.0 MB/s | 308 kB 00:00 2024-09-21T01:51:46.189 INFO:teuthology.orchestra.run.smithi099.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 68 kB/s | 16 kB 00:00 2024-09-21T01:51:46.190 INFO:teuthology.orchestra.run.smithi092.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T01:51:46.224 INFO:teuthology.orchestra.run.smithi092.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-21T01:51:46.247 INFO:teuthology.orchestra.run.smithi099.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T01:51:46.249 INFO:teuthology.orchestra.run.smithi092.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-09-21T01:51:46.273 INFO:teuthology.orchestra.run.smithi099.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 2.1 MB/s | 285 kB 00:00 2024-09-21T01:51:46.283 INFO:teuthology.orchestra.run.smithi092.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.8 MB/s | 151 kB 00:00 2024-09-21T01:51:46.298 INFO:teuthology.orchestra.run.smithi099.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 367 kB/s | 18 kB 00:00 2024-09-21T01:51:46.316 INFO:teuthology.orchestra.run.smithi092.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-09-21T01:51:46.350 INFO:teuthology.orchestra.run.smithi092.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-21T01:51:46.375 INFO:teuthology.orchestra.run.smithi092.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T01:51:46.400 INFO:teuthology.orchestra.run.smithi092.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T01:51:46.426 INFO:teuthology.orchestra.run.smithi092.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 569 kB/s | 14 kB 00:00 2024-09-21T01:51:46.440 INFO:teuthology.orchestra.run.smithi099.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-21T01:51:46.496 INFO:teuthology.orchestra.run.smithi092.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-21T01:51:46.496 INFO:teuthology.orchestra.run.smithi092.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T01:51:46.518 INFO:teuthology.orchestra.run.smithi092.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.0 MB/s | 303 kB 00:00 2024-09-21T01:51:46.523 INFO:teuthology.orchestra.run.smithi099.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T01:51:46.544 INFO:teuthology.orchestra.run.smithi092.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-21T01:51:46.549 INFO:teuthology.orchestra.run.smithi099.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-21T01:51:46.577 INFO:teuthology.orchestra.run.smithi092.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T01:51:46.582 INFO:teuthology.orchestra.run.smithi099.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-21T01:51:46.602 INFO:teuthology.orchestra.run.smithi092.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T01:51:46.641 INFO:teuthology.orchestra.run.smithi099.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-09-21T01:51:46.666 INFO:teuthology.orchestra.run.smithi099.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-21T01:51:46.669 INFO:teuthology.orchestra.run.smithi092.stdout:(124/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-09-21T01:51:46.692 INFO:teuthology.orchestra.run.smithi099.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T01:51:46.695 INFO:teuthology.orchestra.run.smithi092.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-09-21T01:51:46.717 INFO:teuthology.orchestra.run.smithi099.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T01:51:46.720 INFO:teuthology.orchestra.run.smithi092.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-21T01:51:46.745 INFO:teuthology.orchestra.run.smithi092.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-21T01:51:46.750 INFO:teuthology.orchestra.run.smithi099.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T01:51:46.771 INFO:teuthology.orchestra.run.smithi092.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T01:51:46.796 INFO:teuthology.orchestra.run.smithi092.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-21T01:51:46.821 INFO:teuthology.orchestra.run.smithi092.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T01:51:46.846 INFO:teuthology.orchestra.run.smithi092.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T01:51:46.871 INFO:teuthology.orchestra.run.smithi092.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T01:51:46.905 INFO:teuthology.orchestra.run.smithi092.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-21T01:51:46.931 INFO:teuthology.orchestra.run.smithi092.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-21T01:51:46.951 INFO:teuthology.orchestra.run.smithi099.stdout:(48/157): ceph-mgr-diskprediction-local-19.3.0- 4.9 MB/s | 7.4 MB 00:01 2024-09-21T01:51:46.956 INFO:teuthology.orchestra.run.smithi092.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T01:51:46.981 INFO:teuthology.orchestra.run.smithi092.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T01:51:46.984 INFO:teuthology.orchestra.run.smithi099.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-21T01:51:47.006 INFO:teuthology.orchestra.run.smithi092.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T01:51:47.026 INFO:teuthology.orchestra.run.smithi099.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-21T01:51:47.031 INFO:teuthology.orchestra.run.smithi092.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T01:51:47.052 INFO:teuthology.orchestra.run.smithi099.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 325 kB/s | 22 kB 00:00 2024-09-21T01:51:47.057 INFO:teuthology.orchestra.run.smithi092.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T01:51:47.077 INFO:teuthology.orchestra.run.smithi099.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T01:51:47.082 INFO:teuthology.orchestra.run.smithi092.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T01:51:47.107 INFO:teuthology.orchestra.run.smithi092.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T01:51:47.110 INFO:teuthology.orchestra.run.smithi099.stdout:(53/157): boost-program-options-1.75.0-8.el9.x8 672 kB/s | 107 kB 00:00 2024-09-21T01:51:47.136 INFO:teuthology.orchestra.run.smithi099.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-21T01:51:47.158 INFO:teuthology.orchestra.run.smithi092.stdout:(142/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T01:51:47.178 INFO:teuthology.orchestra.run.smithi099.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-21T01:51:47.183 INFO:teuthology.orchestra.run.smithi092.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 742 kB/s | 74 kB 00:00 2024-09-21T01:51:47.203 INFO:teuthology.orchestra.run.smithi099.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-21T01:51:47.208 INFO:teuthology.orchestra.run.smithi092.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T01:51:47.228 INFO:teuthology.orchestra.run.smithi099.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T01:51:47.233 INFO:teuthology.orchestra.run.smithi092.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-21T01:51:47.259 INFO:teuthology.orchestra.run.smithi092.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-09-21T01:51:47.279 INFO:teuthology.orchestra.run.smithi099.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-21T01:51:47.284 INFO:teuthology.orchestra.run.smithi092.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T01:51:47.304 INFO:teuthology.orchestra.run.smithi099.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-21T01:51:47.317 INFO:teuthology.orchestra.run.smithi092.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T01:51:47.329 INFO:teuthology.orchestra.run.smithi099.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T01:51:47.363 INFO:teuthology.orchestra.run.smithi099.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-21T01:51:47.368 INFO:teuthology.orchestra.run.smithi092.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T01:51:47.382 INFO:teuthology.orchestra.run.smithi136.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 677 kB/s | 1.0 MB 00:01 2024-09-21T01:51:47.396 INFO:teuthology.orchestra.run.smithi099.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T01:51:47.413 INFO:teuthology.orchestra.run.smithi099.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2024-09-21T01:51:47.430 INFO:teuthology.orchestra.run.smithi099.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-09-21T01:51:47.455 INFO:teuthology.orchestra.run.smithi099.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-09-21T01:51:47.480 INFO:teuthology.orchestra.run.smithi099.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-21T01:51:47.506 INFO:teuthology.orchestra.run.smithi099.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-21T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T01:51:47.582 INFO:teuthology.orchestra.run.smithi136.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-09-21T01:51:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-21T01:51:47.673 INFO:teuthology.orchestra.run.smithi099.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-21T01:51:47.677 INFO:teuthology.orchestra.run.smithi092.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 7.7 MB/s | 2.3 MB 00:00 2024-09-21T01:51:47.715 INFO:teuthology.orchestra.run.smithi099.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-21T01:51:47.727 INFO:teuthology.orchestra.run.smithi092.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 822 kB/s | 41 kB 00:00 2024-09-21T01:51:47.749 INFO:teuthology.orchestra.run.smithi099.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 6.2 MB/s | 910 kB 00:00 2024-09-21T01:51:47.774 INFO:teuthology.orchestra.run.smithi099.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-21T01:51:47.794 INFO:teuthology.orchestra.run.smithi092.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.7 MB/s | 249 kB 00:00 2024-09-21T01:51:47.816 INFO:teuthology.orchestra.run.smithi099.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-21T01:51:47.841 INFO:teuthology.orchestra.run.smithi099.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T01:51:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:(154/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.0 MB/s | 3.2 MB 00:00 2024-09-21T01:51:47.883 INFO:teuthology.orchestra.run.smithi099.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T01:51:47.886 INFO:teuthology.orchestra.run.smithi092.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.2 MB/s | 1.8 MB 00:00 2024-09-21T01:51:47.919 INFO:teuthology.orchestra.run.smithi092.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-21T01:51:47.975 INFO:teuthology.orchestra.run.smithi099.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-21T01:51:48.000 INFO:teuthology.orchestra.run.smithi099.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-21T01:51:48.025 INFO:teuthology.orchestra.run.smithi099.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-21T01:51:48.051 INFO:teuthology.orchestra.run.smithi099.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T01:51:48.076 INFO:teuthology.orchestra.run.smithi099.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-21T01:51:48.101 INFO:teuthology.orchestra.run.smithi099.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T01:51:48.127 INFO:teuthology.orchestra.run.smithi099.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-21T01:51:48.161 INFO:teuthology.orchestra.run.smithi092.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.5 MB/s | 2.4 MB 00:00 2024-09-21T01:51:48.167 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:51:48.167 INFO:teuthology.orchestra.run.smithi092.stdout:Total 21 MB/s | 228 MB 00:10 2024-09-21T01:51:48.173 INFO:teuthology.orchestra.run.smithi136.stdout:(74/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-09-21T01:51:48.202 INFO:teuthology.orchestra.run.smithi099.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T01:51:48.227 INFO:teuthology.orchestra.run.smithi099.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T01:51:48.252 INFO:teuthology.orchestra.run.smithi099.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-21T01:51:48.278 INFO:teuthology.orchestra.run.smithi099.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi136.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-09-21T01:51:48.303 INFO:teuthology.orchestra.run.smithi099.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-21T01:51:48.328 INFO:teuthology.orchestra.run.smithi099.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T01:51:48.353 INFO:teuthology.orchestra.run.smithi099.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T01:51:48.379 INFO:teuthology.orchestra.run.smithi099.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T01:51:48.404 INFO:teuthology.orchestra.run.smithi099.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T01:51:48.429 INFO:teuthology.orchestra.run.smithi099.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T01:51:48.513 INFO:teuthology.orchestra.run.smithi099.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T01:51:48.538 INFO:teuthology.orchestra.run.smithi099.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-21T01:51:48.564 INFO:teuthology.orchestra.run.smithi099.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-21T01:51:48.589 INFO:teuthology.orchestra.run.smithi099.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T01:51:48.614 INFO:teuthology.orchestra.run.smithi099.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-21T01:51:48.639 INFO:teuthology.orchestra.run.smithi099.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-21T01:51:48.657 INFO:teuthology.orchestra.run.smithi136.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-21T01:51:48.664 INFO:teuthology.orchestra.run.smithi099.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T01:51:48.690 INFO:teuthology.orchestra.run.smithi099.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T01:51:48.715 INFO:teuthology.orchestra.run.smithi099.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T01:51:48.741 INFO:teuthology.orchestra.run.smithi099.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T01:51:48.766 INFO:teuthology.orchestra.run.smithi099.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-21T01:51:48.991 INFO:teuthology.orchestra.run.smithi099.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-21T01:51:49.008 INFO:teuthology.orchestra.run.smithi099.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-21T01:51:49.041 INFO:teuthology.orchestra.run.smithi136.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 647 kB/s | 248 kB 00:00 2024-09-21T01:51:49.067 INFO:teuthology.orchestra.run.smithi099.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-21T01:51:49.083 INFO:teuthology.orchestra.run.smithi099.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2024-09-21T01:51:49.117 INFO:teuthology.orchestra.run.smithi099.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-21T01:51:49.134 INFO:teuthology.orchestra.run.smithi099.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-21T01:51:49.149 INFO:teuthology.orchestra.run.smithi136.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-21T01:51:49.160 INFO:teuthology.orchestra.run.smithi099.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T01:51:49.185 INFO:teuthology.orchestra.run.smithi099.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T01:51:49.210 INFO:teuthology.orchestra.run.smithi099.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T01:51:49.254 INFO:teuthology.orchestra.run.smithi099.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T01:51:49.258 INFO:teuthology.orchestra.run.smithi136.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-21T01:51:49.302 INFO:teuthology.orchestra.run.smithi099.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T01:51:49.327 INFO:teuthology.orchestra.run.smithi099.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T01:51:49.340 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-21T01:51:49.353 INFO:teuthology.orchestra.run.smithi099.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T01:51:49.378 INFO:teuthology.orchestra.run.smithi099.stdout:(117/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-09-21T01:51:49.412 INFO:teuthology.orchestra.run.smithi099.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-21T01:51:49.487 INFO:teuthology.orchestra.run.smithi099.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T01:51:49.493 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-21T01:51:49.493 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-21T01:51:49.512 INFO:teuthology.orchestra.run.smithi099.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-09-21T01:51:49.537 INFO:teuthology.orchestra.run.smithi099.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-21T01:51:49.558 INFO:teuthology.orchestra.run.smithi136.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 574 kB/s | 172 kB 00:00 2024-09-21T01:51:49.563 INFO:teuthology.orchestra.run.smithi099.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-21T01:51:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-21T01:51:49.622 INFO:teuthology.orchestra.run.smithi099.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-21T01:51:49.647 INFO:teuthology.orchestra.run.smithi099.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-21T01:51:49.667 INFO:teuthology.orchestra.run.smithi136.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-21T01:51:49.672 INFO:teuthology.orchestra.run.smithi099.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-21T01:51:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-21T01:51:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T01:51:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T01:51:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T01:51:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T01:51:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T01:51:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-09-21T01:51:49.899 INFO:teuthology.orchestra.run.smithi099.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-21T01:51:49.925 INFO:teuthology.orchestra.run.smithi099.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T01:51:49.950 INFO:teuthology.orchestra.run.smithi099.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T01:51:49.975 INFO:teuthology.orchestra.run.smithi099.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T01:51:50.000 INFO:teuthology.orchestra.run.smithi099.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-09-21T01:51:50.026 INFO:teuthology.orchestra.run.smithi099.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T01:51:50.051 INFO:teuthology.orchestra.run.smithi099.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T01:51:50.076 INFO:teuthology.orchestra.run.smithi099.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T01:51:50.110 INFO:teuthology.orchestra.run.smithi099.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-21T01:51:50.152 INFO:teuthology.orchestra.run.smithi099.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-09-21T01:51:50.177 INFO:teuthology.orchestra.run.smithi099.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T01:51:50.202 INFO:teuthology.orchestra.run.smithi099.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T01:51:50.228 INFO:teuthology.orchestra.run.smithi099.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-21T01:51:50.312 INFO:teuthology.orchestra.run.smithi099.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-21T01:51:50.395 INFO:teuthology.orchestra.run.smithi099.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-09-21T01:51:50.437 INFO:teuthology.orchestra.run.smithi099.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-21T01:51:50.587 INFO:teuthology.orchestra.run.smithi099.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T01:51:50.722 INFO:teuthology.orchestra.run.smithi099.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 8.2 MB/s | 2.3 MB 00:00 2024-09-21T01:51:50.763 INFO:teuthology.orchestra.run.smithi099.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 983 kB/s | 41 kB 00:00 2024-09-21T01:51:50.830 INFO:teuthology.orchestra.run.smithi099.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.7 MB/s | 249 kB 00:00 2024-09-21T01:51:50.872 INFO:teuthology.orchestra.run.smithi099.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.4 MB/s | 1.8 MB 00:00 2024-09-21T01:51:50.914 INFO:teuthology.orchestra.run.smithi099.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-21T01:51:50.989 INFO:teuthology.orchestra.run.smithi099.stdout:(156/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.3 MB/s | 3.2 MB 00:00 2024-09-21T01:51:51.039 INFO:teuthology.orchestra.run.smithi099.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-21T01:51:51.046 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:51:51.046 INFO:teuthology.orchestra.run.smithi099.stdout:Total 18 MB/s | 228 MB 00:12 2024-09-21T01:51:51.707 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-21T01:51:51.764 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-21T01:51:52.248 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-21T01:51:52.402 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-21T01:51:52.402 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-21T01:51:54.205 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-21T01:51:54.205 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-21T01:51:54.258 INFO:teuthology.orchestra.run.smithi136.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 694 kB/s | 5.8 MB 00:08 2024-09-21T01:51:54.650 INFO:teuthology.orchestra.run.smithi136.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 640 kB/s | 250 kB 00:00 2024-09-21T01:51:55.160 INFO:teuthology.orchestra.run.smithi136.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 436 kB/s | 222 kB 00:00 2024-09-21T01:51:55.359 INFO:teuthology.orchestra.run.smithi136.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 389 kB/s | 77 kB 00:00 2024-09-21T01:51:55.467 INFO:teuthology.orchestra.run.smithi136.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-09-21T01:51:55.959 INFO:teuthology.orchestra.run.smithi136.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 548 kB/s | 269 kB 00:00 2024-09-21T01:51:56.284 INFO:teuthology.orchestra.run.smithi136.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 488 kB/s | 159 kB 00:00 2024-09-21T01:51:56.360 INFO:teuthology.orchestra.run.smithi136.stdout:(89/157): python3-babel-2.9.1-2.el9.noarch.rpm 696 kB/s | 6.0 MB 00:08 2024-09-21T01:51:56.468 INFO:teuthology.orchestra.run.smithi136.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-09-21T01:51:56.568 INFO:teuthology.orchestra.run.smithi136.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-09-21T01:51:56.760 INFO:teuthology.orchestra.run.smithi136.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 586 kB/s | 279 kB 00:00 2024-09-21T01:51:56.869 INFO:teuthology.orchestra.run.smithi136.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-21T01:51:56.977 INFO:teuthology.orchestra.run.smithi136.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-21T01:51:57.319 INFO:teuthology.orchestra.run.smithi136.stdout:(95/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 682 kB/s | 5.1 MB 00:07 2024-09-21T01:51:57.345 INFO:teuthology.orchestra.run.smithi136.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 601 kB/s | 221 kB 00:00 2024-09-21T01:51:57.428 INFO:teuthology.orchestra.run.smithi136.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-09-21T01:51:57.453 INFO:teuthology.orchestra.run.smithi136.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-21T01:51:57.562 INFO:teuthology.orchestra.run.smithi136.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-21T01:51:57.771 INFO:teuthology.orchestra.run.smithi136.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 459 kB/s | 96 kB 00:00 2024-09-21T01:51:57.896 INFO:teuthology.orchestra.run.smithi136.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 650 kB/s | 304 kB 00:00 2024-09-21T01:51:57.972 INFO:teuthology.orchestra.run.smithi136.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-21T01:51:58.005 INFO:teuthology.orchestra.run.smithi136.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 412 kB/s | 45 kB 00:00 2024-09-21T01:51:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-21T01:51:58.080 INFO:teuthology.orchestra.run.smithi136.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-21T01:51:58.139 INFO:teuthology.orchestra.run.smithi136.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T01:51:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:51:58.164 INFO:teuthology.orchestra.run.smithi136.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T01:51:58.190 INFO:teuthology.orchestra.run.smithi136.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-21T01:51:58.223 INFO:teuthology.orchestra.run.smithi136.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-21T01:51:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:51:58.274 INFO:teuthology.orchestra.run.smithi136.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-21T01:51:58.332 INFO:teuthology.orchestra.run.smithi136.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-09-21T01:51:58.358 INFO:teuthology.orchestra.run.smithi136.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T01:51:58.383 INFO:teuthology.orchestra.run.smithi136.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T01:51:58.443 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:51:58.458 INFO:teuthology.orchestra.run.smithi136.stdout:(113/157): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-21T01:51:58.492 INFO:teuthology.orchestra.run.smithi136.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T01:51:58.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:51:58.517 INFO:teuthology.orchestra.run.smithi136.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-21T01:51:58.524 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:51:58.543 INFO:teuthology.orchestra.run.smithi136.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-09-21T01:51:58.568 INFO:teuthology.orchestra.run.smithi136.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-21T01:51:58.576 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:51:58.593 INFO:teuthology.orchestra.run.smithi136.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T01:51:58.618 INFO:teuthology.orchestra.run.smithi136.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T01:51:58.644 INFO:teuthology.orchestra.run.smithi136.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T01:51:58.677 INFO:teuthology.orchestra.run.smithi136.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T01:51:58.703 INFO:teuthology.orchestra.run.smithi136.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T01:51:58.728 INFO:teuthology.orchestra.run.smithi136.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T01:51:58.753 INFO:teuthology.orchestra.run.smithi136.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-21T01:51:58.778 INFO:teuthology.orchestra.run.smithi136.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-21T01:51:58.804 INFO:teuthology.orchestra.run.smithi136.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-21T01:51:58.829 INFO:teuthology.orchestra.run.smithi136.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-21T01:51:58.846 INFO:teuthology.orchestra.run.smithi136.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 1.1 MB/s | 20 kB 00:00 2024-09-21T01:51:58.897 INFO:teuthology.orchestra.run.smithi136.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-21T01:51:58.922 INFO:teuthology.orchestra.run.smithi136.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T01:51:58.935 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:51:58.947 INFO:teuthology.orchestra.run.smithi136.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T01:51:58.964 INFO:teuthology.orchestra.run.smithi136.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T01:51:58.989 INFO:teuthology.orchestra.run.smithi136.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-21T01:51:59.023 INFO:teuthology.orchestra.run.smithi136.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-21T01:51:59.040 INFO:teuthology.orchestra.run.smithi136.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-21T01:51:59.057 INFO:teuthology.orchestra.run.smithi136.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-21T01:51:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:51:59.082 INFO:teuthology.orchestra.run.smithi136.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 679 kB/s | 28 kB 00:00 2024-09-21T01:51:59.107 INFO:teuthology.orchestra.run.smithi136.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-21T01:51:59.133 INFO:teuthology.orchestra.run.smithi136.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T01:51:59.149 INFO:teuthology.orchestra.run.smithi136.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-21T01:51:59.175 INFO:teuthology.orchestra.run.smithi136.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 856 kB/s | 36 kB 00:00 2024-09-21T01:51:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:51:59.200 INFO:teuthology.orchestra.run.smithi136.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T01:51:59.217 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:51:59.225 INFO:teuthology.orchestra.run.smithi136.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T01:51:59.251 INFO:teuthology.orchestra.run.smithi136.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T01:51:59.276 INFO:teuthology.orchestra.run.smithi136.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-21T01:51:59.292 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:51:59.301 INFO:teuthology.orchestra.run.smithi136.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-21T01:51:59.318 INFO:teuthology.orchestra.run.smithi136.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T01:51:59.322 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:51:59.352 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:51:59.368 INFO:teuthology.orchestra.run.smithi136.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-21T01:51:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:51:59.594 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:51:59.610 INFO:teuthology.orchestra.run.smithi136.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T01:51:59.622 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:51:59.728 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:51:59.756 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:51:59.794 INFO:teuthology.orchestra.run.smithi136.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.4 MB/s | 3.2 MB 00:00 2024-09-21T01:51:59.810 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:51:59.841 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:51:59.871 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:52:00.099 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:52:00.280 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:52:00.302 INFO:teuthology.orchestra.run.smithi136.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-09-21T01:52:00.397 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:52:00.411 INFO:teuthology.orchestra.run.smithi136.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 377 kB/s | 41 kB 00:00 2024-09-21T01:52:00.453 INFO:teuthology.orchestra.run.smithi136.stdout:(153/157): NetworkManager-libnm-1.51.0-1.el9.x8 2.7 MB/s | 1.8 MB 00:00 2024-09-21T01:52:00.508 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:00.528 INFO:teuthology.orchestra.run.smithi136.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-21T01:52:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-21T01:52:00.587 INFO:teuthology.orchestra.run.smithi136.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-21T01:52:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:52:00.834 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:00.834 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:52:00.941 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:52:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:52:00.977 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:52:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:52:01.013 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:52:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:52:01.077 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:52:01.095 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:52:01.147 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:52:01.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:52:01.231 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:52:01.264 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:52:01.291 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:52:01.370 INFO:teuthology.orchestra.run.smithi136.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-09-21T01:52:01.386 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:52:01.407 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:52:01.438 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:52:01.478 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:52:01.521 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:52:01.527 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:52:01.601 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:52:01.631 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:52:01.653 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:52:01.675 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:52:01.686 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:52:01.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:52:01.773 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:52:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:52:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:52:01.819 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:52:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:52:01.987 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:52:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:52:02.038 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:52:02.067 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:52:02.173 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:52:02.201 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:52:02.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:52:02.263 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:52:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:52:02.304 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:52:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:52:02.494 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:52:02.653 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:52:02.775 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:52:02.834 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:52:02.871 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:02.933 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:52:02.955 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:02.970 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:52:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:52:03.060 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:52:03.109 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:52:03.191 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:03.274 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:52:03.300 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:52:03.325 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:52:03.329 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:52:03.360 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:52:03.421 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:52:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:52:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:52:03.485 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:52:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:52:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:52:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:52:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:52:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:52:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:52:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:52:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:52:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:52:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:52:03.743 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:52:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:52:03.783 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:52:03.811 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:52:03.818 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:52:03.899 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:52:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:52:03.946 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:52:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:52:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:52:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:52:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:52:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:52:04.132 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:52:04.206 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:52:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:52:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:52:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:52:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:52:04.337 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:52:04.345 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:52:04.393 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:52:04.430 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:52:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:52:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:52:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:52:04.680 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:52:04.719 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:52:04.751 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:52:04.781 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:52:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:52:04.849 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:52:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:52:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:52:04.933 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:52:05.005 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:52:05.028 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:52:05.034 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:52:05.077 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:52:05.080 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:52:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:52:05.115 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:52:05.164 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:52:05.217 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:52:05.265 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:52:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:52:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:52:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:52:05.540 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:52:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:52:05.653 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:52:05.689 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:52:05.735 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:52:05.788 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:52:05.831 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:52:05.873 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:52:05.977 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:52:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:52:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:52:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:52:06.063 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:52:06.099 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:52:06.169 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:52:06.217 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:52:06.254 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:52:06.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:52:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:52:06.376 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:52:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:52:06.416 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:52:06.441 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:52:06.466 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:52:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:52:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:52:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:52:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:52:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:52:06.711 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:52:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:52:06.768 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:52:06.807 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:52:06.839 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:52:06.877 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:52:06.912 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:52:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:52:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:52:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:52:07.117 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:52:07.161 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:52:07.193 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:52:07.295 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:52:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:52:07.367 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:52:07.487 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:52:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:52:08.116 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:52:08.149 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:52:08.247 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:52:08.304 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:52:08.349 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:52:08.437 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:52:08.485 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:52:08.668 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:52:08.699 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:52:08.777 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:52:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:52:09.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:52:09.169 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:52:09.202 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:52:09.422 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:52:09.528 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:52:09.539 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:52:09.581 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:52:09.672 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:52:09.711 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:52:09.747 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:52:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:52:10.275 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:52:10.346 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:52:10.369 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:52:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:52:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:52:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:52:10.897 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:52:11.224 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:52:11.276 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:52:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:52:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:52:11.417 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:52:11.927 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:52:11.962 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:52:12.017 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:52:12.047 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:52:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:12.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:52:13.411 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:52:13.411 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:52:13.411 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:52:13.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:13.444 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:52:14.649 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:14.685 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:52:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:52:15.385 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:52:15.385 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:52:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:15.414 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:52:20.134 INFO:teuthology.orchestra.run.smithi136.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 706 kB/s | 16 MB 00:23 2024-09-21T01:52:20.140 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:52:20.140 INFO:teuthology.orchestra.run.smithi136.stdout:Total 5.2 MB/s | 228 MB 00:43 2024-09-21T01:52:21.331 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-09-21T01:52:21.483 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-09-21T01:52:21.483 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-09-21T01:52:23.363 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-09-21T01:52:23.363 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-21T01:52:25.210 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-21T01:52:25.211 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-21T01:52:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:52:25.405 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:52:25.405 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:25.405 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:52:25.405 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:52:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:52:27.035 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:27.035 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:52:27.035 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:52:27.035 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:52:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:27.259 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:52:27.267 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:52:27.267 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-09-21T01:52:27.268 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:52:27.282 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:27.282 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:52:27.282 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:52:27.282 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:52:27.282 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:27.449 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:52:27.466 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:52:27.467 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:27.467 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:52:27.467 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:27.752 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:52:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:52:27.775 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:27.775 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:52:27.775 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:52:27.775 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:52:27.775 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:27.890 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:52:27.955 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:52:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:52:28.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:52:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:52:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:52:28.426 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:52:28.557 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:52:28.602 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:52:28.643 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:52:28.670 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:52:29.060 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:29.087 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:52:29.127 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:52:29.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:52:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:52:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:52:29.319 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:52:29.342 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:52:29.342 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:29.342 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:52:29.342 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:52:29.343 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:52:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-09-21T01:52:29.811 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:52:29.812 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:52:29.834 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:52:29.834 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:29.834 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:52:29.834 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:52:29.835 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:52:29.835 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:29.850 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:52:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:52:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:52:29.941 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:52:29.964 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:52:29.998 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:52:30.005 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:52:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:52:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:52:30.287 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:52:30.356 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:52:30.412 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:52:30.452 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:52:30.473 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:52:30.589 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:52:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:52:30.616 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:52:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:52:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:52:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:52:30.690 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:52:30.720 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:52:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:52:30.901 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:52:30.951 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:52:30.979 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:52:31.084 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:52:31.114 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:52:31.132 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:52:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:52:31.176 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:52:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:52:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:52:31.222 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:52:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:52:31.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:52:31.416 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:52:31.543 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:52:31.655 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:52:31.754 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:31.823 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:32.049 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:52:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:52:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:52:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:52:32.284 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:52:32.338 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:52:32.370 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:52:32.414 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:52:32.424 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:52:32.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:52:32.474 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:52:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:52:32.543 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:52:32.596 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:52:32.627 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:52:32.662 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:52:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:52:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:52:32.825 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:52:32.853 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:52:32.922 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:52:32.972 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:52:33.045 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:52:33.178 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:52:33.232 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:52:33.449 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:52:33.480 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:52:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:52:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:52:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:52:33.799 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:52:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:52:33.824 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:33.824 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:52:33.824 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:52:33.824 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:52:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:52:33.866 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:52:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:52:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:52:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:52:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:52:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:52:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:52:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:52:34.150 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:52:34.225 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:52:34.315 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:52:34.381 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:52:34.453 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:52:34.484 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:52:34.485 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:52:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:52:34.541 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:52:34.568 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:52:34.604 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:52:34.647 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:52:34.690 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:52:34.826 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:52:34.873 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:52:34.914 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:52:35.016 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:52:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:52:35.053 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:52:35.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:52:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:52:35.077 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:35.077 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:52:35.077 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:52:35.077 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:52:35.077 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:35.096 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:52:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:52:35.122 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:52:35.143 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:52:35.143 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:35.143 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:52:35.143 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:35.147 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:52:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:52:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:52:35.210 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:35.210 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:52:35.210 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:52:35.210 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:52:35.210 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:35.221 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:52:35.349 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:52:35.379 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:52:35.407 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:52:35.441 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:52:35.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:52:35.498 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:52:35.524 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:52:35.529 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:52:35.538 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:52:35.559 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:52:35.561 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:52:35.561 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:35.561 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:52:35.561 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:52:35.561 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:52:35.561 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:35.594 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:52:35.627 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:52:35.636 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:52:35.668 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:52:35.703 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:52:35.791 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:52:35.802 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:52:35.835 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:52:35.842 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:52:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:52:35.859 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:35.859 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:52:35.859 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:52:35.859 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:52:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:35.873 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:52:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:52:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:52:35.977 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:52:36.021 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:52:36.032 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:52:36.075 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:52:36.749 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:52:36.880 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:52:36.914 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:52:37.012 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:52:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:52:37.060 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:52:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:52:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:52:37.103 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:37.103 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:52:37.104 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:52:37.104 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:52:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:37.105 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:52:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:52:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:52:37.170 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:37.170 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:52:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:37.200 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:52:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:52:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:52:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:52:37.245 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:37.245 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:52:37.245 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:52:37.246 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:52:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:37.427 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:52:37.456 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:52:37.533 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:52:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:52:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:52:37.588 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:37.588 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:52:37.588 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:52:37.588 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:52:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:52:38.296 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:52:39.514 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:52:39.559 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:52:39.595 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:52:39.599 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:52:39.639 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:52:39.640 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:52:39.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:52:39.672 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:52:39.699 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:52:39.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:52:39.954 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:52:39.984 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:52:40.001 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:52:40.022 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:52:40.043 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:52:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:52:40.134 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:52:40.165 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:52:40.201 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:52:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:52:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:52:40.700 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:52:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:52:40.730 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:52:40.801 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:52:40.823 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:52:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:52:41.552 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:52:41.593 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:52:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:52:41.641 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:52:41.673 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:52:41.690 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:52:41.718 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:52:41.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:52:41.798 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:52:41.823 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:52:41.849 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:52:41.849 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:52:41.903 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:52:41.917 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:52:41.917 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:52:41.948 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:52:41.978 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:52:41.992 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:52:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:52:42.017 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:52:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:52:42.022 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:52:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:52:42.211 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:52:42.212 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:52:42.287 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:52:42.351 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:52:42.672 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:52:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:52:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:52:43.046 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:43.078 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:52:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:52:43.642 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:52:43.674 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:52:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:52:43.767 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:52:43.767 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:52:43.768 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:52:43.768 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:52:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:52:43.801 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:52:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:52:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:52:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:52:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:52:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:52:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:52:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:52:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:52:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:52:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:52:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:52:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:52:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:52:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:52:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:52:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:52:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:52:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:52:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:52:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:52:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:52:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:52:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:52:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:52:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:52:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:52:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:52:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:52:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:52:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:52:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:52:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:52:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:52:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:52:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:52:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:52:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:52:50.640 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:52:50.640 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:52:50.640 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:52:50.640 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:52:50.640 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:52:50.640 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:52:50.641 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:52:50.642 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:52:50.643 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:52:50.644 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:52:50.645 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:52:50.646 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:52:50.647 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:52:50.648 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:52:50.649 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:52:50.650 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:52:50.651 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:52:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:52:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:52:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:52:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:52:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:52:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:52:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:52:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:52:50.670 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-21T01:52:51.092 DEBUG:teuthology.parallel:result is None 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.438 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.439 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.440 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:52:53.441 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.442 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:52:53.443 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.444 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:52:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:52:53.446 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:52:53.447 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:52:53.448 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:52:53.449 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:52:53.450 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:52:53.450 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:52:53.450 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:52:53.450 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-21T01:52:53.871 DEBUG:teuthology.parallel:result is None 2024-09-21T01:52:55.699 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:52:55.699 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-09-21T01:52:55.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:52:55.871 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:52:55.886 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:52:55.886 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:55.886 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:52:55.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:52:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:52:57.500 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:52:57.500 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:57.501 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:52:57.501 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:52:57.501 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:52:57.501 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:52:57.742 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:52:57.769 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:52:57.769 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:57.770 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:52:57.770 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:52:57.770 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:52:57.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:52:58.243 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:52:58.267 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:52:58.267 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:52:58.267 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:52:58.267 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:52:58.267 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:52:58.267 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:52:58.365 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:52:58.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:52:58.478 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:52:58.662 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:52:58.711 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:52:58.789 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:52:58.878 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:52:59.025 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:52:59.061 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:52:59.093 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:52:59.121 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:52:59.548 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:52:59.604 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:52:59.636 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:52:59.701 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:52:59.727 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:53:02.927 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:53:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:53:03.968 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:53:03.991 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:53:04.105 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:53:04.275 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:53:04.302 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:53:04.326 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:53:04.326 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:53:04.326 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:53:04.326 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:53:04.326 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:53:04.327 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:04.369 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:53:04.381 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:53:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:53:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:53:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:53:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:53:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:53:05.546 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:53:05.546 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:53:05.546 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:53:05.546 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:53:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:53:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:53:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:53:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:53:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:53:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:53:05.688 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:53:05.688 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:53:05.688 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:53:05.688 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:53:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:53:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:53:06.039 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:53:06.039 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:53:06.039 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:53:06.039 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:53:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:10.045 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:53:10.086 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:53:10.149 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:53:10.184 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:53:10.211 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:53:10.395 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:53:10.442 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:53:10.472 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:53:10.501 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:53:10.539 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:53:11.119 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:53:11.163 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:53:11.183 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:53:12.005 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:53:12.093 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:53:12.133 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:53:12.214 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:53:12.246 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:53:12.272 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:53:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:53:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:53:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:53:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:53:12.482 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:53:12.508 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:53:12.513 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:53:12.513 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:53:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:53:12.693 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:53:12.777 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:53:12.842 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:53:19.493 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:53:19.493 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:53:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:53:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:53:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:53:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:53:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:53:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:53:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:53:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:53:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:53:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.966 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.967 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.968 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:53:21.969 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:53:21.970 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:53:21.971 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.972 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:53:21.973 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:53:21.974 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.975 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:53:21.976 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:53:21.977 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:53:21.978 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:53:21.978 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:53:21.978 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-09-21T01:53:22.406 DEBUG:teuthology.parallel:result is None 2024-09-21T01:53:22.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:22.557 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:53:22.583 INFO:teuthology.orchestra.run.smithi092.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:53:22.583 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:53:22.583 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:53:22.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:22.724 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:53:22.750 INFO:teuthology.orchestra.run.smithi099.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:53:22.751 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:53:22.751 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:53:22.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:22.895 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:53:22.920 INFO:teuthology.orchestra.run.smithi136.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:53:22.920 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:53:22.920 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:53:22.922 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T01:53:22.922 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:53:22.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:53:22.955 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:53:22.955 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:53:22.989 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:53:22.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:53:23.018 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T01:53:23.019 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:53:23.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:53:23.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:53:23.115 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:53:23.115 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:53:23.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:53:23.212 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:53:23.212 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:53:23.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:53:23.306 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T01:53:23.306 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:53:23.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:53:23.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:53:23.397 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:53:23.397 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:53:23.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:53:23.487 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:53:23.487 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:53:23.513 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:53:23.579 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T01:53:23.579 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:53:23.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:53:23.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:53:23.669 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:53:23.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:53:23.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:53:23.758 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:53:23.758 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:53:23.784 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:53:23.850 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T01:53:23.938 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T01:53:23.938 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:23.939 INFO:tasks.cephadm:Cluster fsid is 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:53:23.939 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T01:53:23.939 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.92', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.136'} 2024-09-21T01:53:23.939 INFO:tasks.cephadm:First mon is mon.a on smithi092 2024-09-21T01:53:23.939 INFO:tasks.cephadm:First mgr is x 2024-09-21T01:53:23.940 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T01:53:23.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-09-21T01:53:23.965 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-09-21T01:53:23.990 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-09-21T01:53:24.017 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T01:53:24.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:24.166 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T01:53:24.314 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:24.316 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T01:53:24.316 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T01:53:24.316 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:24.821 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:53 /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:24.821 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.351 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:53 /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.351 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.861 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:53 /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.861 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.881 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.900 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:53:25.935 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T01:53:25.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:53:25.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:53:25.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:53:27.440 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:53:27.445 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:53:27.626 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:54:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-21T01:54:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:54:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:54:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-09-21T01:54:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:54:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-21T01:54:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-21T01:54:10.456 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T01:54:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:54:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:54:10.456 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-09-21T01:54:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:54:10.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T01:54:10.457 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T01:54:12.296 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T01:54:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:54:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:54:12.296 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-09-21T01:54:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:54:12.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-21T01:54:12.297 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T01:54:12.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-09-21T01:54:12.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-09-21T01:54:12.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-09-21T01:54:12.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-09-21T01:54:12.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-09-21T01:54:12.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-09-21T01:54:12.493 INFO:tasks.cephadm:Writing seed config... 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T01:54:12.494 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T01:54:12.495 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T01:54:12.496 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T01:54:12.496 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T01:54:12.496 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T01:54:12.496 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T01:54:12.496 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:54:12.496 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T01:54:12.513 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 49513f9c-77bc-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T01:54:12.513 DEBUG:teuthology.orchestra.run.smithi092:mon.a> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T01:54:12.556 DEBUG:teuthology.orchestra.run.smithi092:mgr.x> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T01:54:12.599 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T01:54:12.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.92 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:54:12.803 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:54:12.803 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.92', '--skip-admin-label'] 2024-09-21T01:54:12.804 INFO:teuthology.orchestra.run.smithi092.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T01:54:12.804 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying podman|docker is present... 2024-09-21T01:54:12.827 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-21T01:54:12.827 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying lvm2 is present... 2024-09-21T01:54:12.827 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying time synchronization is in place... 2024-09-21T01:54:12.835 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T01:54:12.835 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T01:54:12.841 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T01:54:12.841 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-21T01:54:12.848 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-09-21T01:54:12.854 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-09-21T01:54:12.855 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-09-21T01:54:12.855 INFO:teuthology.orchestra.run.smithi092.stdout:Repeating the final host check... 2024-09-21T01:54:12.879 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 5.2.2 2024-09-21T01:54:12.879 INFO:teuthology.orchestra.run.smithi092.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T01:54:12.879 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl is present 2024-09-21T01:54:12.879 INFO:teuthology.orchestra.run.smithi092.stdout:lvcreate is present 2024-09-21T01:54:12.885 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T01:54:12.885 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T01:54:12.891 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T01:54:12.892 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-21T01:54:12.898 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-09-21T01:54:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-09-21T01:54:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-09-21T01:54:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:Host looks OK 2024-09-21T01:54:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:Cluster fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:Acquiring lock 140436620446064 on /run/cephadm/49513f9c-77bc-11ef-baf5-efdc52797490.lock 2024-09-21T01:54:12.905 INFO:teuthology.orchestra.run.smithi092.stdout:Lock 140436620446064 acquired on /run/cephadm/49513f9c-77bc-11ef-baf5-efdc52797490.lock 2024-09-21T01:54:12.906 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 3300 ... 2024-09-21T01:54:12.906 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 6789 ... 2024-09-21T01:54:12.906 INFO:teuthology.orchestra.run.smithi092.stdout:Base mon IP(s) is [172.21.15.92:3300, 172.21.15.92:6789], mon addrv is [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-21T01:54:12.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.92 metric 100 2024-09-21T01:54:12.909 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.92 metric 100 2024-09-21T01:54:12.912 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T01:54:12.912 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T01:54:12.912 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T01:54:12.914 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T01:54:12.914 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T01:54:12.914 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T01:54:12.914 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T01:54:12.914 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-21T01:54:12.914 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T01:54:12.915 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-09-21T01:54:12.915 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-09-21T01:54:12.915 INFO:teuthology.orchestra.run.smithi092.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T01:54:12.915 INFO:teuthology.orchestra.run.smithi092.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T01:54:12.916 INFO:teuthology.orchestra.run.smithi092.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:54:13.358 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T01:54:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:54:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T01:54:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T01:54:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T01:54:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T01:54:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T01:54:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T01:54:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T01:54:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T01:54:15.278 INFO:teuthology.orchestra.run.smithi092.stdout:stat: stdout 167 167 2024-09-21T01:54:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial keys... 2024-09-21T01:54:16.301 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQBHJ+5mxzKRJBAARH0/Mp6USckeBc427oapDQ== 2024-09-21T01:54:17.165 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQBIJ+5mIdkAJRAA3hGaVBrPzNysXOjBAw236w== 2024-09-21T01:54:18.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQBJJ+5m19v3HRAA55OzOpUxoFeT0Yb2jP784w== 2024-09-21T01:54:18.040 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial monmap... 2024-09-21T01:54:18.962 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T01:54:18.962 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T01:54:18.962 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:18.962 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool for a [v2:172.21.15.92:3300,v1:172.21.15.92:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = quincy 2024-09-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: set fsid to 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mon... 2024-09-21T01:54:20.026 INFO:teuthology.orchestra.run.smithi092.stdout:create mon.a on 2024-09-21T01:54:20.236 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T01:54:20.399 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T01:54:20.596 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-49513f9c-77bc-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-49513f9c-77bc-11ef-baf5-efdc52797490.target. 2024-09-21T01:54:20.596 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-49513f9c-77bc-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-49513f9c-77bc-11ef-baf5-efdc52797490.target. 2024-09-21T01:54:20.846 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a 2024-09-21T01:54:20.846 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T01:54:21.040 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-49513f9c-77bc-11ef-baf5-efdc52797490.target.wants/ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-49513f9c-77bc-11ef-baf5-efdc52797490@.service. 2024-09-21T01:54:21.099 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 systemd[1]: Starting Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:54:21.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 podman[30805]: 2024-09-21 01:54:21.196172005 +0000 UTC m=+0.028646660 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:54:21.390 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 podman[30805]: 2024-09-21 01:54:21.292462289 +0000 UTC m=+0.124936926 container create a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T01:54:21.576 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:54:21.576 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-21T01:54:21.584 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:54:21.584 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-21T01:54:21.584 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-21T01:54:21.584 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T01:54:21.584 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon to start... 2024-09-21T01:54:21.584 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon... 2024-09-21T01:54:21.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 podman[30805]: 2024-09-21 01:54:21.486440663 +0000 UTC m=+0.318915305 container init a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T01:54:21.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 podman[30805]: 2024-09-21 01:54:21.489923815 +0000 UTC m=+0.322398458 container start a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T01:54:21.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 ceph-mon[30823]: mkfs 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:21.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 ceph-mon[30823]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:54:21.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 bash[30805]: a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 2024-09-21T01:54:21.642 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:21 smithi092 systemd[1]: Started Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:54:22.547 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T01:54:22.547 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout id: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:22.547 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T01:54:22.547 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout services: 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.575883s) 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout data: 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:mon is available 2024-09-21T01:54:22.548 INFO:teuthology.orchestra.run.smithi092.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: monmap epoch 1 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: last_changed 2024-09-21T01:54:18.399933+0000 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: min_mon_release 19 (squid) 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: election_strategy: 1 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: fsmap 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: mgrmap e1: no daemons active 2024-09-21T01:54:22.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:22 smithi092 ceph-mon[30823]: from='client.? 172.21.15.92:0/3933259036' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T01:54:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T01:54:23.622 INFO:teuthology.orchestra.run.smithi092.stdout:Generating new minimal ceph.conf... 2024-09-21T01:54:23.717 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:23 smithi092 ceph-mon[30823]: from='client.? 172.21.15.92:0/1565973794' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T01:54:23.717 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:23 smithi092 ceph-mon[30823]: from='client.? 172.21.15.92:0/1565973794' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T01:54:24.656 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:24 smithi092 ceph-mon[30823]: from='client.? 172.21.15.92:0/2148905738' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:54:24.659 INFO:teuthology.orchestra.run.smithi092.stdout:Restarting the monitor... 2024-09-21T01:54:24.935 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:24 smithi092 systemd[1]: Stopping Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:54:25.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a[30819]: 2024-09-21T01:54:24.934+0000 7f6fabeaa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:54:25.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a[30819]: 2024-09-21T01:54:24.934+0000 7f6fabeaa640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T01:54:25.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 podman[31068]: 2024-09-21 01:54:25.057704239 +0000 UTC m=+0.226584624 container died a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T01:54:25.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 podman[31068]: 2024-09-21 01:54:25.385187855 +0000 UTC m=+0.554068240 container cleanup a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T01:54:25.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 bash[31068]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a 2024-09-21T01:54:25.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 podman[31080]: 2024-09-21 01:54:25.596701323 +0000 UTC m=+0.534031058 container remove a3162dbad95c969f082d28d6d7eb0cf044147b8d89d542f6941ac82e8f7a2037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T01:54:25.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T01:54:25.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 systemd[1]: Stopped Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:54:25.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 systemd[1]: Starting Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 podman[31171]: 2024-09-21 01:54:25.830465495 +0000 UTC m=+0.015724314 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:25 smithi092 podman[31171]: 2024-09-21 01:54:25.928716828 +0000 UTC m=+0.113975638 container create 082f4472a4da4395e63f54c692064907d4a8c4d89713730cea5497c983850d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 podman[31171]: 2024-09-21 01:54:26.140407552 +0000 UTC m=+0.325666373 container init 082f4472a4da4395e63f54c692064907d4a8c4d89713730cea5497c983850d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 podman[31171]: 2024-09-21 01:54:26.143427324 +0000 UTC m=+0.328686137 container start 082f4472a4da4395e63f54c692064907d4a8c4d89713730cea5497c983850d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: pidfile_write: ignore empty --pid-file 2024-09-21T01:54:26.178 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: load: jerasure load: lrc 2024-09-21T01:54:26.227 INFO:teuthology.orchestra.run.smithi092.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T01:54:26.441 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: RocksDB version: 7.9.2 2024-09-21T01:54:26.441 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Git sha 0 2024-09-21T01:54:26.441 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T01:54:26.441 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: DB SUMMARY 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: DB Session ID: RZ7X419XQ3DX6GMSKI9F 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: CURRENT file: CURRENT 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95192 ; 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.error_if_exists: 0 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.create_if_missing: 0 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.paranoid_checks: 1 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.env: 0x55ba6164cd60 2024-09-21T01:54:26.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.info_log: 0x55ba66bd5360 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.statistics: (nil) 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.use_fsync: 0 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_log_file_size: 0 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.allow_fallocate: 1 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T01:54:26.443 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.use_direct_reads: 0 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.db_log_dir: 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.wal_dir: 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T01:54:26.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T01:54:26.445 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T01:54:26.445 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.write_buffer_manager: 0x55ba66bd9d60 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.unordered_write: 0 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T01:54:26.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.row_cache: None 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.wal_filter: None 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.two_write_queues: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.wal_compression: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.atomic_flush: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T01:54:26.447 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.log_readahead_size: 0 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_background_jobs: 2 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_background_compactions: -1 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_subcompactions: 1 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T01:54:26.448 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_open_files: -1 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_background_flushes: -1 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Compression algorithms supported: 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kZSTD supported: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kXpressCompression supported: 0 2024-09-21T01:54:26.449 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kBZip2Compression supported: 0 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kLZ4Compression supported: 1 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kZlibCompression supported: 1 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: kSnappyCompression supported: 1 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.merge_operator: 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_filter: None 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T01:54:26.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ba66bd4fa0) 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_type: 0 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_index_type: 0 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_shortening: 1 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: checksum: 4 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: no_block_cache: 0 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache: 0x55ba62b99610 2024-09-21T01:54:26.451 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_options: 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: capacity : 536870912 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_shard_bits : 4 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: strict_capacity_limit : 0 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_compressed: (nil) 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: persistent_cache: (nil) 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size: 4096 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size_deviation: 10 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_restart_interval: 16 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_block_restart_interval: 1 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: metadata_block_size: 4096 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: partition_filters: 0 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: use_delta_encoding: 1 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: filter_policy: bloomfilter 2024-09-21T01:54:26.452 INFO:journalctl@ceph.mon.a.smithi092.stdout: whole_key_filtering: 1 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: verify_compression: 0 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: format_version: 5 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_index_compression: 1 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_align: 0 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: max_auto_readahead_size: 262144 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: prepopulate_block_cache: 0 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: initial_auto_readahead_size: 8192 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression: NoCompression 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T01:54:26.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.num_levels: 7 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:54:26.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T01:54:26.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T01:54:26.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.inplace_update_support: 0 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.bloom_locality: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.max_successive_merges: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.ttl: 2592000 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.enable_blob_files: false 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.min_blob_size: 0 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T01:54:26.458 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8f008047-9e97-4df9-9cd3-aa3e5363e1fb 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726883666180331, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726883666181541, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91335, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89383, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11708, "raw_average_key_size": 47, "raw_value_size": 82791, "raw_average_value_size": 339, "num_data_blocks": 12, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726883666, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8f008047-9e97-4df9-9cd3-aa3e5363e1fb", "db_session_id": "RZ7X419XQ3DX6GMSKI9F", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726883666181642, "job": 1, "event": "recovery_finished"} 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T01:54:26.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ba66bf8a80 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: DB pointer 0x55ba6799e000 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** DB Stats ** 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:54:26.460 INFO:journalctl@ceph.mon.a.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: L0 2/0 91.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: Sum 2/0 91.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T01:54:26.461 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative compaction: 0.00 GB write, 25.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval compaction: 0.00 GB write, 25.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: Block cache BinnedLRUCache@0x55ba62b99610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: starting mon.a rank 0 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???) e1 preinit fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).mds e1 new map 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).mds e1 print_map 2024-09-21T01:54:26.462 INFO:journalctl@ceph.mon.a.smithi092.stdout: e1 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout: btime 2024-09-21T01:54:21:537800+0000 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout: legacy client fscid: -1 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout: No filesystems configured 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 bash[31171]: 082f4472a4da4395e63f54c692064907d4a8c4d89713730cea5497c983850d2f 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 systemd[1]: Started Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: monmap epoch 1 2024-09-21T01:54:26.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: last_changed 2024-09-21T01:54:18.399933+0000 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: min_mon_release 19 (squid) 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: election_strategy: 1 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: fsmap 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T01:54:26.464 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:26 smithi092 ceph-mon[31190]: mgrmap e1: no daemons active 2024-09-21T01:54:27.282 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T01:54:27.304 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:54:27.304 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mgr... 2024-09-21T01:54:27.305 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T01:54:27.305 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T01:54:27.554 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x 2024-09-21T01:54:27.555 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T01:54:27.705 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-49513f9c-77bc-11ef-baf5-efdc52797490.target.wants/ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-49513f9c-77bc-11ef-baf5-efdc52797490@.service. 2024-09-21T01:54:27.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:27 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1664733590' entity='client.admin' 2024-09-21T01:54:27.933 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:27 smithi092 systemd[1]: Starting Ceph mgr.x for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:54:28.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:27 smithi092 podman[31399]: 2024-09-21 01:54:27.932226816 +0000 UTC m=+0.108222276 container create 024a4f715ef9ece78f6764b43ee2b02d2c896b0ee3c85d23ea1dbaf319358a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:54:28.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:27 smithi092 podman[31399]: 2024-09-21 01:54:27.839383105 +0000 UTC m=+0.015378540 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:54:28.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:28 smithi092 podman[31399]: 2024-09-21 01:54:28.134947002 +0000 UTC m=+0.310942421 container init 024a4f715ef9ece78f6764b43ee2b02d2c896b0ee3c85d23ea1dbaf319358a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T01:54:28.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:28 smithi092 podman[31399]: 2024-09-21 01:54:28.137840133 +0000 UTC m=+0.313835563 container start 024a4f715ef9ece78f6764b43ee2b02d2c896b0ee3c85d23ea1dbaf319358a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T01:54:28.201 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:28 smithi092 bash[31399]: 024a4f715ef9ece78f6764b43ee2b02d2c896b0ee3c85d23ea1dbaf319358a85 2024-09-21T01:54:28.215 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:54:28.215 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-21T01:54:28.222 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:54:28.222 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-21T01:54:28.222 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-21T01:54:28.222 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T01:54:28.231 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:54:28.231 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-21T01:54:28.238 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:54:28.238 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-21T01:54:28.239 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-21T01:54:28.239 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T01:54:28.239 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr to start... 2024-09-21T01:54:28.239 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr... 2024-09-21T01:54:28.453 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:28 smithi092 systemd[1]: Started Ceph mgr.x for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:54:29.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:28 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1206985958' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:54:33.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: Activating manager daemon x 2024-09-21T01:54:33.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: mgrmap e2: x(active, starting, since 0.00356098s) 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: Manager daemon x is now available 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' 2024-09-21T01:54:33.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:33 smithi092 ceph-mon[31190]: from='mgr.14100 172.21.15.92:0/4205546709' entity='mgr.x' 2024-09-21T01:54:34.505 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:34.509 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:54:34.510 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:54:34.513 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:54:21:537800+0000", 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:54:34.514 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:54:21.538611+0000", 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:54:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (1/15)... 2024-09-21T01:54:35.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:35 smithi092 ceph-mon[31190]: mgrmap e3: x(active, since 1.00886s) 2024-09-21T01:54:36.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:36 smithi092 ceph-mon[31190]: mgrmap e4: x(active, since 2s) 2024-09-21T01:54:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:37 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3534904874' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:54:38.240 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:38.240 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:54:38.240 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-21T01:54:38.241 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:54:38.242 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:54:38.245 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:54:38.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:54:21:537800+0000", 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:54:38.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:54:21.538611+0000", 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:54:38.248 INFO:teuthology.orchestra.run.smithi092.stdout:mgr is available 2024-09-21T01:54:39.150 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:38 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2891811519' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T01:54:39.429 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-09-21T01:54:39.430 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T01:54:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling cephadm module... 2024-09-21T01:54:40.391 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:40 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1103319239' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T01:54:40.391 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: ignoring --setuser ceph since I am not root 2024-09-21T01:54:40.392 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: ignoring --setgroup ceph since I am not root 2024-09-21T01:54:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:41 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1103319239' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T01:54:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:41 smithi092 ceph-mon[31190]: mgrmap e5: x(active, since 6s) 2024-09-21T01:54:42.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:42 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/28176468' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T01:54:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: Active manager daemon x restarted 2024-09-21T01:54:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: Activating manager daemon x 2024-09-21T01:54:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T01:54:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: mgrmap e6: x(active, starting, since 0.00348189s) 2024-09-21T01:54:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:54:45.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:54:45.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:54:45.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:54:45.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:54:45.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:45 smithi092 ceph-mon[31190]: Manager daemon x is now available 2024-09-21T01:54:45.983 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-09-21T01:54:45.984 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 5... 2024-09-21T01:54:46.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:46.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:46.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: Found migration_current of "None". Setting to last migration. 2024-09-21T01:54:46.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:46.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:46.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:46.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:46.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:54:46.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:54:46.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:46 smithi092 ceph-mon[31190]: mgrmap e7: x(active, since 1.00576s) 2024-09-21T01:54:46.940 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:54:46.940 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T01:54:46.940 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T01:54:46.940 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:54:46.940 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 5 is available 2024-09-21T01:54:46.940 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying orchestrator module is enabled... 2024-09-21T01:54:47.554 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T01:54:47.223+0000 7f2ba27e7640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:54:47] ENGINE Error in HTTPServer.serve 2024-09-21T01:54:47.554 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: Traceback (most recent call last): 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self._connections.run(self.expiration_interval) 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self._run(expiration_interval) 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: s = self.context.wrap_socket( 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: return self.sslsocket_class._create( 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self.do_handshake() 2024-09-21T01:54:47.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:54:47.556 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self._sslobj.do_handshake() 2024-09-21T01:54:47.556 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:54:47.556 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T01:54:48.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-mon[31190]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T01:54:48.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-mon[31190]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T01:54:48.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-mon[31190]: [21/Sep/2024:01:54:47] ENGINE Bus STARTING 2024-09-21T01:54:48.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:48.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:47 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2414003802' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T01:54:48.814 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T01:54:48.814 INFO:teuthology.orchestra.run.smithi092.stdout:Setting orchestrator backend to cephadm... 2024-09-21T01:54:48.936 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:48 smithi092 ceph-mon[31190]: [21/Sep/2024:01:54:47] ENGINE Serving on https://172.21.15.92:7150 2024-09-21T01:54:48.936 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:48 smithi092 ceph-mon[31190]: [21/Sep/2024:01:54:47] ENGINE Error in HTTPServer.serve 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:54:48.937 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:48 smithi092 ceph-mon[31190]: [21/Sep/2024:01:54:47] ENGINE Serving on http://172.21.15.92:8765 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:48 smithi092 ceph-mon[31190]: [21/Sep/2024:01:54:47] ENGINE Bus STARTED 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:48 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2414003802' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T01:54:48.938 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:48 smithi092 ceph-mon[31190]: mgrmap e8: x(active, since 2s) 2024-09-21T01:54:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:50 smithi092 ceph-mon[31190]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:54:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:50 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:50 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:51.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T01:54:51.260 INFO:teuthology.orchestra.run.smithi092.stdout:Generating ssh key... 2024-09-21T01:54:51.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-mon[31190]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: Generating public/private rsa key pair. 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: Your identification has been saved in /tmp/tmpign02fye/key 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: Your public key has been saved in /tmp/tmpign02fye/key.pub 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: The key fingerprint is: 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: SHA256:rIPcbnpisAZKkdNgYC4GBn9apA9caSKWmIaacf9dMBg ceph-49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: The key's randomart image is: 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: +---[RSA 3072]----+ 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |O+. o.Eo | 2024-09-21T01:54:52.230 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |@Oo+o . o | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |*BO+o o | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |=+ B. . . | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: | + .. .S. | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |.... o... | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |o. oo + | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: |. o o.o. | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: | . ..=. | 2024-09-21T01:54:52.231 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:54:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: +----[SHA256]-----+ 2024-09-21T01:54:53.094 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:52 smithi092 ceph-mon[31190]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:54:53.094 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:52 smithi092 ceph-mon[31190]: Generating ssh key... 2024-09-21T01:54:53.095 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:52 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:53.095 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:52 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:53.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:54:53.681 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T01:54:53.681 INFO:teuthology.orchestra.run.smithi092.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T01:54:53.687 INFO:teuthology.orchestra.run.smithi092.stdout:Adding host smithi092... 2024-09-21T01:54:53.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:53 smithi092 ceph-mon[31190]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:54:55.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:55 smithi092 ceph-mon[31190]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:54:55.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:55 smithi092 ceph-mon[31190]: Deploying cephadm binary to smithi092 2024-09-21T01:54:57.974 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Added host 'smithi092' with addr '172.21.15.92' 2024-09-21T01:54:57.974 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mon service... 2024-09-21T01:54:58.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:58 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:54:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:58 smithi092 ceph-mon[31190]: Added host smithi092 2024-09-21T01:54:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:58 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:54:59.373 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T01:54:59.373 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mgr service... 2024-09-21T01:54:59.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:59 smithi092 ceph-mon[31190]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:54:59.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:59 smithi092 ceph-mon[31190]: Saving service mon spec with placement count:5 2024-09-21T01:54:59.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:54:59 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:55:00.606 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T01:55:01.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:00 smithi092 ceph-mon[31190]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:01.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:00 smithi092 ceph-mon[31190]: Saving service mgr spec with placement count:2 2024-09-21T01:55:01.241 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:00 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:55:02.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:02 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/692876378' entity='client.admin' 2024-09-21T01:55:03.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:03 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:55:03.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:03 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:55:03.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:03 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/397506850' entity='client.admin' 2024-09-21T01:55:04.376 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling the dashboard module... 2024-09-21T01:55:05.148 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:04 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:55:05.148 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:04 smithi092 ceph-mon[31190]: from='mgr.14116 172.21.15.92:0/1677723344' entity='mgr.x' 2024-09-21T01:55:06.116 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2369092157' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T01:55:06.117 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:05 smithi092 ceph-mon[31190]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:06.117 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: ignoring --setuser ceph since I am not root 2024-09-21T01:55:06.117 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: ignoring --setgroup ceph since I am not root 2024-09-21T01:55:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:06 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2369092157' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T01:55:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:06 smithi092 ceph-mon[31190]: mgrmap e9: x(active, since 20s) 2024-09-21T01:55:08.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:07 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/757082478' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-09-21T01:55:08.038 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 9... 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: Active manager daemon x restarted 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: Activating manager daemon x 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: mgrmap e10: x(active, starting, since 0.00435323s) 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:55:11.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:10 smithi092 ceph-mon[31190]: Manager daemon x is now available 2024-09-21T01:55:12.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-mon[31190]: [21/Sep/2024:01:55:11] ENGINE Bus STARTING 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-mon[31190]: [21/Sep/2024:01:55:11] ENGINE Serving on http://172.21.15.92:8765 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-mon[31190]: mgrmap e11: x(active, since 1.00649s) 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T01:55:11.987+0000 7fc186c30640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:55:11] ENGINE Error in HTTPServer.serve 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: Traceback (most recent call last): 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self._connections.run(self.expiration_interval) 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self._run(expiration_interval) 2024-09-21T01:55:12.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: s = self.context.wrap_socket( 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: return self.sslsocket_class._create( 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self.do_handshake() 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: self._sslobj.do_handshake() 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:55:12.306 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:55:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T01:55:12.628 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:55:12.628 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T01:55:12.628 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T01:55:12.628 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:55:12.628 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 9 is available 2024-09-21T01:55:12.628 INFO:teuthology.orchestra.run.smithi092.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T01:55:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:12 smithi092 ceph-mon[31190]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T01:55:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:12 smithi092 ceph-mon[31190]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T01:55:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:12 smithi092 ceph-mon[31190]: [21/Sep/2024:01:55:11] ENGINE Serving on https://172.21.15.92:7150 2024-09-21T01:55:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:12 smithi092 ceph-mon[31190]: [21/Sep/2024:01:55:11] ENGINE Bus STARTED 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:12 smithi092 ceph-mon[31190]: [21/Sep/2024:01:55:11] ENGINE Error in HTTPServer.serve 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: Traceback (most recent call last): 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._run(expiration_interval) 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: s = self.context.wrap_socket( 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:55:13.305 INFO:journalctl@ceph.mon.a.smithi092.stdout: return self.sslsocket_class._create( 2024-09-21T01:55:13.306 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:55:13.306 INFO:journalctl@ceph.mon.a.smithi092.stdout: self.do_handshake() 2024-09-21T01:55:13.306 INFO:journalctl@ceph.mon.a.smithi092.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:55:13.306 INFO:journalctl@ceph.mon.a.smithi092.stdout: self._sslobj.do_handshake() 2024-09-21T01:55:13.306 INFO:journalctl@ceph.mon.a.smithi092.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:55:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:13 smithi092 ceph-mon[31190]: mgrmap e12: x(active, since 2s) 2024-09-21T01:55:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:13 smithi092 ceph-mon[31190]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:13 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:14.752 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUSBhEM84yYIn4JW5R7Bf1i9n/w1gwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAxNTUxNFoXDTM0\nMDkyMjAxNTUxNFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjkyMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAxAWz1rcuoenFATTIhYz9sWVLBqu6Jlu+T1n3\nU/yQTrFGoLPV7p5Z2NCWR6FC2INtQUoKX+QZkJM2INkBxCBH10govclpnDhwrVsh\nMvik0opCUOIGq/TqYVng/MmLGlO4yCKC1dHrsN91jYnruE2vcwwNIWiQsv/XbzPs\nZYIFSzgJiiQgafzc+Yjb0wF97QldMLYqiFnFh5MtNKWuLecPtQx+7QDHmmVHLzqw\nxYEjOO+cHYu8prU1sJG3urj3qRPK2bHTg7E/K/JqroDFfu9xML6wCVcTT3EGsjjq\nvcH6X3R9alt15L5HIaFj2vr/ZRD6m8m9SvVG9Zn5LhWh3njwkajt9xTH7Pyd0or0\n5PsWoBTmkLXCzx0kaJjsuP5DwbHf7UgzlV9oWyjNZEoRl85NK+t7qEu86dq0tUtj\nWNO7JYJpr4VbUU+c2nIfQH/p1jK1nISRwPHMiRrcI8tzNsV98OAwhBHkNbgdesEb\nGE3chCjkCxdM1RlU8vZ0yTwvy6HtuI+nag2BfHEdXcIie8Dv6WoYaKnWQCUyg7Pz\nQl71jYXgJ8oig9zrIDmw3EyUfzrPhSfeffZa5nbBiS38fHY3vSpEveh/s2xyq2U8\nR/ay8glFdKx3p0Xrj7lYOyBryMaZdYrJdKd6GkeFjMGAEHwz1cTI8yDH5PUhWHow\n8Aa5y7ECAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDkyghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPXDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAX\n1Xew8EqtROT8hlHi6tKVWAIK4zZCx7a9RP2k4lFfKGSmarXjZI4tcOlQEpnJJzuk\norEPxlg0CAw3pubwZ3khXFxdthVhq3QKpRnVA8Sky8JlsETl5A+z3TQ8qA9jWGn0\nkB6rRixxN2Ehw33uDjU0Z0HQ5vOM7i7b/+INYEhetMXV3qbxuS0bEhqrJs/NDdoR\ney3IDV3l5rc7KgUhDverY/Tc1C7yL/Nb+UaoWL7efGl00JvP2e5q2KcvxiPFbYq9\n4x7TR0QULEV0R4Ay71kBX3xn4g1D0JvTjWL+jubpeGFgH9VMDqcNBZS4YRTJvtRa\nK/lfBRiHm45dZDf0Izkh5ZTu46lS63CXVabJyuMEBJMuXQ5B1cxK5NIZUUA1Ikj+\njCyjLyi43XETD2Bhe0UmK7UTaoMvjQoeazQ1typWyIdpBsxEbNIpQ47/CzcwyXF3\nvr48Rxt/fYUE02Mz67IFHh90PYqzuny2lrswGu0LNI7uXJuYGE/Jg0wyh70lj5VI\njnZkiZfQRb+xgt6LNerbUB7iISLyuSGkcMWRD2t+sCKep6Ky71yjkj0osiBHppBc\nWRTBZE9mDgxE/EY9r728vXtUnQXV6m769X8lo0XBOHWI2oVKCnT6Dwrz0QCph6MO\n8VNMWSMO8mp5+QO3c+/3jqJUagygh6Qlnrb/KoFHYQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAxAWz1rcuoenFATTIhYz9sWVLBqu6Jlu+T1n3U/yQTrFGoLPV\n7p5Z2NCWR6FC2INtQUoKX+QZkJM2INkBxCBH10govclpnDhwrVshMvik0opCUOIG\nq/TqYVng/MmLGlO4yCKC1dHrsN91jYnruE2vcwwNIWiQsv/XbzPsZYIFSzgJiiQg\nafzc+Yjb0wF97QldMLYqiFnFh5MtNKWuLecPtQx+7QDHmmVHLzqwxYEjOO+cHYu8\nprU1sJG3urj3qRPK2bHTg7E/K/JqroDFfu9xML6wCVcTT3EGsjjqvcH6X3R9alt1\n5L5HIaFj2vr/ZRD6m8m9SvVG9Zn5LhWh3njwkajt9xTH7Pyd0or05PsWoBTmkLXC\nzx0kaJjsuP5DwbHf7UgzlV9oWyjNZEoRl85NK+t7qEu86dq0tUtjWNO7JYJpr4Vb\nUU+c2nIfQH/p1jK1nISRwPHMiRrcI8tzNsV98OAwhBHkNbgdesEbGE3chCjkCxdM\n1RlU8vZ0yTwvy6HtuI+nag2BfHEdXcIie8Dv6WoYaKnWQCUyg7PzQl71jYXgJ8oi\ng9zrIDmw3EyUfzrPhSfeffZa5nbBiS38fHY3vSpEveh/s2xyq2U8R/ay8glFdKx3\np0Xrj7lYOyBryMaZdYrJdKd6GkeFjMGAEHwz1cTI8yDH5PUhWHow8Aa5y7ECAwEA\nAQKCAgACoQzzcUuH0mnZBUs3N1fjvDcQZNc/UnWv/MXrc3TACXKoDyFsbOVJmiqZ\n1VjfLIwXlxWiiZ2qKb9NG3wBlGlCQu6A9SWceclRGerNEzaquD4BdbafMc6+9YSl\n/1IFTWAqFwRoN8OQgL7D4jiuJSSYR24Zp5xhOjs6e5jNF7xYcOz31Fo28bRJCIPi\nRZ/ZwQll8NuLLqEOm3O8xAgdB39Xb054gU0sDfWxdXkPfRbq2ZPfdZNXLREGSD4N\nnBN+SgyrdM4gcw0k2sCBjKqZdD6xZ61Wwrs4koyDcW38k5B3HUndVlKzebXchpyf\nQESSp3KlCkSmr7l+68zCyHl+UWLwmy+i67AmnkTn2cmMkSzQX/JYUi8+sQEj09un\nfp8CJyQ+lcmPEtr5QVIy5QMXNdk9dret4NxrPLr4/74i6GkQTiVkEj91FYZAT4V+\nS9dZ+KjgBa0mjs8M766bnBbyTqhtrJ/LnnpwEwI7SQvMJn/nF+KaLMcdpMgvpZ7Q\nv9LMmZucLIfeS9JOTdtymP4PufhFI6szl142faucTV6xsfYEcMZXUKBZQjweZRPv\nWwkvXtWiHB5d1rMJiBVsjZfS07ZOq3Zgjg4d3OckpTvLo7m8zuWDEn86UJLxKmbW\nfsl9YDeaXAeUJ2gkIyayho4LqUCkPKOn42pZZgmIcHMPAC3vnQKCAQEA5YfMcJys\nfgv7Nvjx2RzEJRWr+tGFBfNgz8MjfFbZiDTPidLDOXDM2mybEn695ST2Xaw3pLnw\nMHvFO1pti5Ls+uyQLHplnZNL/EP7RNg+4iaaxzJNLceTiSgMed79ileRHhagay9J\n3sNML0YfVgx95le6YD1INCcujwbjmvtvO17ozLqEadbCaR7hhWfzA15f5cTfe80Z\nK64LKsfEvGZSKBA8w9rvnAvZj2nArQQNdv/jYo4E7Zd7AM0MAH9rAaTiQJjmL3Qr\n3jY8U+Q08PNAiI3ufZRPWbGuCnXY4SZp58Q/s4wE8oyEbY7vQsPtlDPJ2HEZFUGl\n8qIopvhy7hHltwKCAQEA2qCsuyf1hf1GFrUTP9zdwQII+yRK5EYMcjuJxxgTg4u9\n6prRcfeGEMtRKi5PAcgCTtxfT9giZSMsbqX/zPiKTn6D7iF6sxrJpffdc6wAvYbx\n2ifB1HFf8xxFIadUbnwxb4scXQGl0kYALn61UFWH7G4CQsVz4NwYVmcXPuWaUPHl\nkDy/STUHdB3Oh604DtNTXTNsePEFOY+Lw2tA3cPUU/k/fl5UTUkaoXHLP+FKcRkK\nZfCq0Swy7Ms9GZMUgMRBNIGbmfpexrwg59wLAdpU/rsHMytVpouBLrzYBHHiA4kH\nWXE5P8N9Asj8ExnaTPjqDr2h0J/t7rZiNviTc9UZ1wKCAQEAuefgnUE4sKrRoPfH\nItuavpKsBe2XeXdjbUOXRUkuzZRAmFe9yZNkSHF4tGU1NxhV9w7azq/PK+XDnMwk\nN80b5iyepXfJP/SiscL6DAu9+vvblwVV34NrKsbmO/txjZBZOG6JJedaRHQYC54X\nGWHqwxtBGFvqVFt76VpiFRAbSpIXqmJTh0nrkqrxbFUCTjMtJvPrMuGuxmMkJlVB\nMnsJheuI2/VC5zft0auM5FWI9l5RdGyk7y8m8EIju294Yw+CNWHh5oOIWGXDPQt0\nGchb935FsS7jijlQRqamr82mL5MTKUB6bV2EkRwZDBGXJtc/X/BSqZXPbBMNLBfF\nNrAGRwKCAQBkakoLyCWv/g10HoUR3wsHPD0NxLSVNks/KHN2u2szDzsShztNwieU\njR3hZtWo9smAmR052gaqN0aQJ7Re24UC2n8UH9JwcwxNK5ROLaFFa7anwXz4a35r\ndf+WnnlcKbS3C8jiOwCDnN02fcIffLlqcXieXHyFufF/tefqXqDLLt+GqTAemrmq\nrooOrRPDU3LFqvnWbrU8o97IyflIIkriqudrzj2gJcSShGabF4oOejbZbXm3izp0\nokCU/4X6QqnESQdD6MghNYVTmTCBEFvbh+Gdsfir/wU9h0UcfYYWo5h5j1SBIfPI\nFLIav9QRaH5weM7auKoa6zAfGXr4R9ePAoIBAAn0oGRUtAFc7smVAH9NEa4YaBRh\nlnAxSraISBe14c25rB0cA0RupTo9C9SAyuKgdL5X2nAuIR8RJY3nSfbdY8cXeAIs\nPtYAM/+HKlHmXp6nwtEyUFGa5lq5qxBq0v+bk5bArvZZvAFgLPGb/AI/NND3hVYc\ncy3vZmL25ORyRKWhe/lLztp0QzAcZk9UEO6FOPeN7GWhmLljOEvzTPayyPajurlF\njuo52JPNOocedrM48Yvz4vgBbLKhKhM8z81Q2h1zdf3dJKNzPKR8uV55gXNZLkUH\nC4oGA4xRKRqHD6QJ3x3ATViZ9aMvxaArakrv7ZZL3jywKzB3LQRDRYPOa2k=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T01:55:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:15 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:16.053 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T01:55:16.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:16 smithi092 ceph-mon[31190]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:16.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:17.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:17 smithi092 ceph-mon[31190]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:17.952 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:17.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T01:55:17.955 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial admin user... 2024-09-21T01:55:19.344 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1JWVV7KNOXGX4AaZL9O0/OPPHkUCF9KA9RlzbVJ0UxxlbWT/YtW4a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726883718, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T01:55:19.344 INFO:teuthology.orchestra.run.smithi092.stdout:Fetching dashboard port number... 2024-09-21T01:55:20.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:19 smithi092 ceph-mon[31190]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:20.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:20.576 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T01:55:20.585 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:55:20.586 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-09-21T01:55:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:55:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-09-21T01:55:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-09-21T01:55:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T01:55:20.595 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph Dashboard is now available at: 2024-09-21T01:55:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: URL: https://smithi092.front.sepia.ceph.com:8443/ 2024-09-21T01:55:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: User: admin 2024-09-21T01:55:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: Password: 396mu2vgtc 2024-09-21T01:55:20.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:20.596 INFO:teuthology.orchestra.run.smithi092.stdout:Saving cluster configuration to /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config directory 2024-09-21T01:55:20.892 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:20 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2407830282' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T01:55:21.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T01:55:21.890 INFO:teuthology.orchestra.run.smithi092.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T01:55:21.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: ceph telemetry on 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout:For more information see: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T01:55:21.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:21.892 INFO:teuthology.orchestra.run.smithi092.stdout:Bootstrap complete. 2024-09-21T01:55:21.929 INFO:tasks.cephadm:Fetching config... 2024-09-21T01:55:21.929 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:55:21.929 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T01:55:21.944 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T01:55:21.944 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:55:21.944 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T01:55:22.000 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T01:55:22.000 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:55:22.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T01:55:22.065 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T01:55:22.065 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:55:22.065 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T01:55:22.121 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T01:55:22.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:55:22.196 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:22.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:55:22.248 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:22.261 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:55:22.302 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGh4iyaboGQB46HehgDDCDVNetxS/o74hY9za7zW5mtDoe6Dn8r9ZfxI7m09UzIQQEJcIbGBM27R8rBEu3AkIYtHRRsEUlv6m+xNR6/pwNJQDhqd2pg/hOzl61CsZqsfKYbZfHDWcZ9KPDUZeTXavKg5r+7cHb3IeUTBdfyR/tjNE0RH87O8I6NZkGAI8aMcTjFhqMI7va8F+w3+IpwiJxJs19OloF5UVplZ6JApLEhiJKkciKyYCUxecmZm4q1QDD/fK6+k0jxtHQ3OG4fSwXXLWhjRR0wqvz5ZjdARQvt7eMvpsr0lBSVSg3aSx5S4Bei+7N4hiQbX37PEBev7Ku1mifxaQ8srm3fnSjIF5H6dnhoAjk4bCqyc8C+jTNqjLiRxnLTPnzKN5oPTL5SxAJcssBeMMyDq3t6joxzBiLFrnbg/uzwpJyoCar3NcnhrhQfFuUEdsytfV46tvUNs1otsBBiUiFn/aak+vkazcqnFrI3Ty9bYxp506SvjgFJgs= ceph-49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:22.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T01:55:22.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:22 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3046518909' entity='client.admin' 2024-09-21T01:55:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:24.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/705738367' entity='client.admin' 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: mgrmap e13: x(active, since 12s) 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:24.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:24.319 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T01:55:24.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T01:55:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:26.696 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2024-09-21T01:55:26.696 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:55:26.697 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2024-09-21T01:55:26.716 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:55:26.716 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:26.774 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2024-09-21T01:55:26.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch host add smithi099 2024-09-21T01:55:26.796 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: Updating smithi092:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: Updating smithi092:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:26.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:27.012 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:27 smithi092 ceph-mon[31190]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:29.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:28 smithi092 ceph-mon[31190]: Deploying cephadm binary to smithi099 2024-09-21T01:55:30.743 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi099' with addr '172.21.15.99' 2024-09-21T01:55:31.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T01:55:31.664 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:32.037 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:32.037 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:31 smithi092 ceph-mon[31190]: Added host smithi099 2024-09-21T01:55:32.037 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:32.037 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:32.037 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:32.319 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:32.319 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2024-09-21T01:55:32.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:32 smithi092 ceph-mon[31190]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:32.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:32.901 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-09-21T01:55:32.901 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:55:32.901 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-09-21T01:55:32.920 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:55:32.920 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:32.978 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-09-21T01:55:32.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch host add smithi136 2024-09-21T01:55:33.218 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:34.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:33 smithi092 ceph-mon[31190]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:55:35.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:34 smithi092 ceph-mon[31190]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:35.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:34 smithi092 ceph-mon[31190]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:36.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: Deploying cephadm binary to smithi136 2024-09-21T01:55:36.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:35 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:36.972 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-09-21T01:55:37.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:36 smithi092 ceph-mon[31190]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:37.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:36 smithi092 ceph-mon[31190]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:37.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:36 smithi092 ceph-mon[31190]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:37.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:36 smithi092 ceph-mon[31190]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:37.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T01:55:37.770 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:38.182 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:37 smithi092 ceph-mon[31190]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:38.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:38.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:37 smithi092 ceph-mon[31190]: Added host smithi136 2024-09-21T01:55:38.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:38.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:38.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:55:38.471 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-09-21T01:55:39.088 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T01:55:39.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T01:55:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:55:39.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:39.444 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:39 smithi092 ceph-mon[31190]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:55:40.287 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-09-21T01:55:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:40 smithi092 ceph-mon[31190]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:40 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3216989727' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:55:40.904 INFO:tasks.cephadm:Adding mon.a on smithi092 2024-09-21T01:55:40.904 INFO:tasks.cephadm:Adding mon.b on smithi099 2024-09-21T01:55:40.904 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-09-21T01:55:40.904 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c' 2024-09-21T01:55:41.148 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3216989727' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:55:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:55:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:41.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:41 smithi092 ceph-mon[31190]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-21T01:55:41.867 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-09-21T01:55:42.418 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T01:55:42.421 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T01:55:42.423 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:55:42.423 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c;count:3 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:42.647 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:42.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:42.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:42.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:55:42.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:42.648 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:42 smithi092 ceph-mon[31190]: Deploying daemon mon.c on smithi136 2024-09-21T01:55:43.053 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:44.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:44 smithi092 ceph-mon[31190]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:44.917 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:55:44.917 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","modified":"2024-09-21T01:54:18.399933Z","created":"2024-09-21T01:54:18.399933Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T01:55:44.917 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-09-21T01:55:45.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:45 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/1994101076' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:55:46.095 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(???) e0 preinit fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: e1 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: btime 2024-09-21T01:54:21:537800+0000 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: legacy client fscid: -1 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout: No filesystems configured 2024-09-21T01:55:46.096 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T01:55:46.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Deploying cephadm binary to smithi136 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:46.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Added host smithi136 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:46.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3216989727' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3216989727' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:46.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c;count:3 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: Deploying daemon mon.c on smithi136 2024-09-21T01:55:46.538 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:46.539 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/1994101076' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:55:46.539 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 ceph-mon[34754]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T01:55:46.539 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 bash[34735]: e245b3464d87d7d51c148d131ef09e94124e2d3aead4bd364d2bcb93099fa188 2024-09-21T01:55:46.539 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:46 smithi136 systemd[1]: Started Ceph mon.c for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:55:46.657 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:55:46.657 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:55:46.913 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: mon.a calling monitor election 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: mon.c calling monitor election 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:51.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: monmap epoch 2 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: last_changed 2024-09-21T01:55:46.103392+0000 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: min_mon_release 19 (squid) 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: election_strategy: 1 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: fsmap 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:55:51.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: mgrmap e13: x(active, since 40s) 2024-09-21T01:55:51.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: overall HEALTH_OK 2024-09-21T01:55:51.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:51.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:51.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:51.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:55:51.536 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: mon.a calling monitor election 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: mon.c calling monitor election 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: monmap epoch 2 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: last_changed 2024-09-21T01:55:46.103392+0000 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: min_mon_release 19 (squid) 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: election_strategy: 1 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: fsmap 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: mgrmap e13: x(active, since 40s) 2024-09-21T01:55:51.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: overall HEALTH_OK 2024-09-21T01:55:51.557 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:51.557 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:51.557 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:51.557 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:55:51.557 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:52.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:55:52.007 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","modified":"2024-09-21T01:55:46.103392Z","created":"2024-09-21T01:54:18.399933Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:55:52.007 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-09-21T01:55:52.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:52 smithi136 ceph-mon[34754]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:52.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:52 smithi136 ceph-mon[34754]: Deploying daemon mon.b on smithi099 2024-09-21T01:55:52.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:52 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/1092857819' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:55:52.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:52 smithi092 ceph-mon[31190]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:52 smithi092 ceph-mon[31190]: Deploying daemon mon.b on smithi099 2024-09-21T01:55:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:52 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/1092857819' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:55:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:53.597 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:55:53.598 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:55:53.854 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:55:54.262 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:54 smithi136 ceph-mon[34754]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:54.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 systemd[1]: Starting Ceph mon.b for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:55:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:54 smithi092 ceph-mon[31190]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:55:54.566 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","modified":"2024-09-21T01:55:46.103392Z","created":"2024-09-21T01:54:18.399933Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:55:54.566 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-09-21T01:55:54.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 podman[34385]: 2024-09-21 01:55:54.419558168 +0000 UTC m=+0.015567472 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:55:54.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 podman[34385]: 2024-09-21 01:55:54.520476176 +0000 UTC m=+0.116485480 container create 8ed0cbe597e5f7f0a7eb399cfb6d87e665b5784d400564380fd441043593d439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 podman[34385]: 2024-09-21 01:55:54.781313489 +0000 UTC m=+0.377322793 container init 8ed0cbe597e5f7f0a7eb399cfb6d87e665b5784d400564380fd441043593d439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 podman[34385]: 2024-09-21 01:55:54.784598327 +0000 UTC m=+0.380607633 container start 8ed0cbe597e5f7f0a7eb399cfb6d87e665b5784d400564380fd441043593d439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pidfile_write: ignore empty --pid-file 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: load: jerasure load: lrc 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: RocksDB version: 7.9.2 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Git sha 0 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T01:55:55.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: DB SUMMARY 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: DB Session ID: WMC0EOT3TK9LUGDUCCDP 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: CURRENT file: CURRENT 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.error_if_exists: 0 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.create_if_missing: 0 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.paranoid_checks: 1 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T01:55:55.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.env: 0x55e8ed48ed60 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.info_log: 0x55e8f210d360 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.statistics: (nil) 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.use_fsync: 0 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_log_file_size: 0 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.allow_fallocate: 1 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T01:55:55.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.use_direct_reads: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.db_log_dir: 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.wal_dir: 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T01:55:55.085 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.write_buffer_manager: 0x55e8f2111d60 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.unordered_write: 0 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T01:55:55.086 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.row_cache: None 2024-09-21T01:55:55.087 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.wal_filter: None 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.two_write_queues: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.wal_compression: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.atomic_flush: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T01:55:55.088 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.log_readahead_size: 0 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_background_jobs: 2 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_background_compactions: -1 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_subcompactions: 1 2024-09-21T01:55:55.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_open_files: -1 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T01:55:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_background_flushes: -1 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Compression algorithms supported: 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kZSTD supported: 0 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kXpressCompression supported: 0 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kBZip2Compression supported: 0 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kLZ4Compression supported: 1 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kZlibCompression supported: 1 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: kSnappyCompression supported: 1 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T01:55:55.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.merge_operator: 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_filter: None 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e8f210cfa0) 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_type: 0 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_index_type: 0 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_shortening: 1 2024-09-21T01:55:55.092 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: checksum: 4 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: no_block_cache: 0 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache: 0x55e8ee0d1610 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_options: 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: capacity : 536870912 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_shard_bits : 4 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: strict_capacity_limit : 0 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_compressed: (nil) 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: persistent_cache: (nil) 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size: 4096 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size_deviation: 10 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_restart_interval: 16 2024-09-21T01:55:55.093 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_block_restart_interval: 1 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: metadata_block_size: 4096 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: partition_filters: 0 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: use_delta_encoding: 1 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: filter_policy: bloomfilter 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: whole_key_filtering: 1 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: verify_compression: 0 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: format_version: 5 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_index_compression: 1 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_align: 0 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: max_auto_readahead_size: 262144 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: prepopulate_block_cache: 0 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: initial_auto_readahead_size: 8192 2024-09-21T01:55:55.094 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression: NoCompression 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.num_levels: 7 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T01:55:55.095 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T01:55:55.096 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T01:55:55.098 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.inplace_update_support: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.bloom_locality: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.max_successive_merges: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T01:55:55.099 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.ttl: 2592000 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.enable_blob_files: false 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.min_blob_size: 0 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T01:55:55.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fa4d2d6e-fe7c-4ca1-970d-000027db62fa 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726883754822710, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726883754823329, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726883754, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fa4d2d6e-fe7c-4ca1-970d-000027db62fa", "db_session_id": "WMC0EOT3TK9LUGDUCCDP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726883754823456, "job": 1, "event": "recovery_finished"} 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e8f2130a80 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: DB pointer 0x55e8f2ed6000 2024-09-21T01:55:55.101 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** DB Stats ** 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:55:55.102 INFO:journalctl@ceph.mon.b.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:55:55.103 INFO:journalctl@ceph.mon.b.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache BinnedLRUCache@0x55e8ee0d1610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: using public_addr v2:172.21.15.99:0/0 -> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] 2024-09-21T01:55:55.104 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: starting mon.b rank -1 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(???) e0 preinit fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: e1 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: btime 2024-09-21T01:54:21:537800+0000 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: legacy client fscid: -1 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout: No filesystems configured 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T01:55:55.105 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Deploying cephadm binary to smithi136 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.106 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.107 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Added host smithi136 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3216989727' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3216989727' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"} : dispatch 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c;count:3 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.109 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Deploying daemon mon.c on smithi136 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/1994101076' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.a calling monitor election 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.c calling monitor election 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: monmap epoch 2 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: last_changed 2024-09-21T01:55:46.103392+0000 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: min_mon_release 19 (squid) 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: election_strategy: 1 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:55:55.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: fsmap 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mgrmap e13: x(active, since 40s) 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: overall HEALTH_OK 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: Deploying daemon mon.b on smithi099 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/1092857819' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:55:55.112 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 ceph-mon[34404]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T01:55:55.113 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 bash[34385]: 8ed0cbe597e5f7f0a7eb399cfb6d87e665b5784d400564380fd441043593d439 2024-09-21T01:55:55.113 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:54 smithi099 systemd[1]: Started Ceph mon.b for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:55:56.120 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:55:56.120 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:55:56.380 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:56:00.249 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","modified":"2024-09-21T01:55:54.851472Z","created":"2024-09-21T01:54:18.399933Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T01:56:00.250 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-09-21T01:56:00.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:56:00.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: mon.a calling monitor election 2024-09-21T01:56:00.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: mon.c calling monitor election 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: mon.b calling monitor election 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: monmap epoch 3 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:56:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: last_changed 2024-09-21T01:55:54.851472+0000 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: min_mon_release 19 (squid) 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: election_strategy: 1 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: fsmap 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: mgrmap e13: x(active, since 48s) 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: overall HEALTH_OK 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:55:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: mon.a calling monitor election 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: mon.c calling monitor election 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: mon.b calling monitor election 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: monmap epoch 3 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: last_changed 2024-09-21T01:55:54.851472+0000 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: min_mon_release 19 (squid) 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: election_strategy: 1 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: fsmap 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: mgrmap e13: x(active, since 48s) 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: overall HEALTH_OK 2024-09-21T01:56:00.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:55:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:56:00.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: mon.a calling monitor election 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: mon.c calling monitor election 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: mon.b calling monitor election 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: monmap epoch 3 2024-09-21T01:56:00.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: last_changed 2024-09-21T01:55:54.851472+0000 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: created 2024-09-21T01:54:18.399933+0000 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: min_mon_release 19 (squid) 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: election_strategy: 1 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: fsmap 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: mgrmap e13: x(active, since 48s) 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: overall HEALTH_OK 2024-09-21T01:56:00.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:00.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:55:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:01.097 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:00 smithi136 ceph-mon[34754]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:01.097 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:00 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:01.097 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:00 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/2959540105' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:56:01.097 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:00 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:01.098 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T01:56:01.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T01:56:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:00 smithi092 ceph-mon[31190]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:00 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:00 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/2959540105' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:56:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:00 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:00 smithi099 ceph-mon[34404]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:00 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:00 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/2959540105' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:56:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:00 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:56:01.351 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:02.024 INFO:teuthology.orchestra.run.smithi092.stdout:# minimal ceph.conf for 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:56:02.025 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-21T01:56:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T01:56:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: mon_host = [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-09-21T01:56:02.130 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T01:56:01.850+0000 7fc1bfe70640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-21T01:56:02.131 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: Updating smithi092:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.132 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.132 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:01 smithi092 ceph-mon[31190]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: Updating smithi092:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:01 smithi136 ceph-mon[34754]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: Updating smithi136:/etc/ceph/ceph.conf 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: Updating smithi092:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: Updating smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:01 smithi099 ceph-mon[34404]: Updating smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:56:02.675 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T01:56:02.675 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:56:02.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:56:02.702 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:56:02.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:56:02.769 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:56:02.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:56:02.798 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:56:02.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:56:02.864 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:56:02.864 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:56:02.892 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:56:02.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:56:02.957 INFO:tasks.cephadm:Adding mgr.x on smithi092 2024-09-21T01:56:02.958 INFO:tasks.cephadm:Adding mgr.y on smithi099 2024-09-21T01:56:02.958 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-09-21T01:56:02.958 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi092=x;smithi099=y;smithi136=z' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/455342323' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:56:03.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:03.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:03.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:03.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:02 smithi092 ceph-mon[31190]: Reconfiguring daemon mon.a on smithi092 2024-09-21T01:56:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/455342323' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:03.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:02 smithi099 ceph-mon[34404]: Reconfiguring daemon mon.a on smithi092 2024-09-21T01:56:03.233 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:56:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/455342323' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:03.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:02 smithi136 ceph-mon[34754]: Reconfiguring daemon mon.a on smithi092 2024-09-21T01:56:03.833 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-09-21T01:56:04.407 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T01:56:04.410 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T01:56:04.412 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T01:56:04.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T01:56:04.414 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T01:56:04.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T01:56:04.457 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T01:56:04.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T01:56:04.501 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T01:56:04.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T01:56:04.554 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T01:56:04.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T01:56:04.603 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T01:56:04.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T01:56:04.661 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T01:56:04.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T01:56:04.722 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T01:56:04.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T01:56:04.777 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T01:56:04.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T01:56:04.828 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T01:56:04.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T01:56:04.878 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T01:56:04.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T01:56:04.936 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T01:56:04.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T01:56:04.996 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T01:56:04.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T01:56:05.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi099=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: Saving service mgr spec with placement smithi092=x;smithi099=y;smithi136=z;count:3 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:05.018 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:04 smithi092 ceph-mon[31190]: Reconfiguring daemon mgr.x on smithi092 2024-09-21T01:56:05.020 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T01:56:05.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T01:56:05.072 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T01:56:05.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T01:56:05.119 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T01:56:05.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T01:56:05.167 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T01:56:05.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T01:56:05.219 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T01:56:05.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T01:56:05.269 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T01:56:05.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi099=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: Saving service mgr spec with placement smithi092=x;smithi099=y;smithi136=z;count:3 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:04 smithi136 ceph-mon[34754]: Reconfiguring daemon mgr.x on smithi092 2024-09-21T01:56:05.320 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T01:56:05.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T01:56:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi099=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: Saving service mgr spec with placement smithi092=x;smithi099=y;smithi136=z;count:3 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:05.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:04 smithi099 ceph-mon[34404]: Reconfiguring daemon mgr.x on smithi092 2024-09-21T01:56:05.371 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T01:56:05.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T01:56:05.410 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:05.436 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T01:56:05.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T01:56:05.524 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T01:56:05.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T01:56:05.571 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T01:56:05.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T01:56:05.619 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T01:56:05.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T01:56:05.675 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T01:56:05.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T01:56:05.720 DEBUG:tasks.cephadm:set 26 configs 2024-09-21T01:56:05.721 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi092') 2024-09-21T01:56:06.294 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.321 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.323 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.344 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:56:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:06.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:06.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:06.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:06 smithi136 ceph-mon[34754]: Reconfiguring daemon mon.b on smithi099 2024-09-21T01:56:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:56:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:06.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:06.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:06 smithi092 ceph-mon[31190]: Reconfiguring daemon mon.b on smithi099 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:06.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:06 smithi099 ceph-mon[34404]: Reconfiguring daemon mon.b on smithi099 2024-09-21T01:56:07.203 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.203 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.206 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.207 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.207 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.214 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:07.233 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi092') 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2474962716' entity='client.admin' 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:07.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:07 smithi092 ceph-mon[31190]: Reconfiguring daemon mon.c on smithi136 2024-09-21T01:56:07.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2474962716' entity='client.admin' 2024-09-21T01:56:07.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:07.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:07.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:56:07.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:07.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:07.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:07.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:07 smithi136 ceph-mon[34754]: Reconfiguring daemon mon.c on smithi136 2024-09-21T01:56:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2474962716' entity='client.admin' 2024-09-21T01:56:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:56:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:56:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:56:07.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:07.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:07 smithi099 ceph-mon[34404]: Reconfiguring daemon mon.c on smithi136 2024-09-21T01:56:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2293326778' entity='client.admin' 2024-09-21T01:56:09.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:08 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1919048560' entity='client.admin' 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:09.159 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.160 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:09.160 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:56:09.160 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:09.160 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:09.160 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2293326778' entity='client.admin' 2024-09-21T01:56:09.160 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:08 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1919048560' entity='client.admin' 2024-09-21T01:56:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2293326778' entity='client.admin' 2024-09-21T01:56:09.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:08 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1919048560' entity='client.admin' 2024-09-21T01:56:09.877 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:09 smithi136 systemd[1]: Starting Ceph mgr.z for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:56:10.198 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:09 smithi136 ceph-mon[34754]: Deploying daemon mgr.z on smithi136 2024-09-21T01:56:10.198 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:09 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/538268037' entity='client.admin' 2024-09-21T01:56:10.199 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:09 smithi136 podman[36003]: 2024-09-21 01:56:09.852109915 +0000 UTC m=+0.015291769 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:56:10.199 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:10 smithi136 podman[36003]: 2024-09-21 01:56:10.004754658 +0000 UTC m=+0.167936496 container create 4917858bb7a6c3e3a7c42142a26e9733d5ea2ea5937ccd65d199c3624679f791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-z, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:56:10.199 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:10 smithi136 podman[36003]: 2024-09-21 01:56:10.198838893 +0000 UTC m=+0.362020727 container init 4917858bb7a6c3e3a7c42142a26e9733d5ea2ea5937ccd65d199c3624679f791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T01:56:10.253 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:09 smithi092 ceph-mon[31190]: Deploying daemon mgr.z on smithi136 2024-09-21T01:56:10.253 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:09 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/538268037' entity='client.admin' 2024-09-21T01:56:10.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:09 smithi099 ceph-mon[34404]: Deploying daemon mgr.z on smithi136 2024-09-21T01:56:10.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:09 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/538268037' entity='client.admin' 2024-09-21T01:56:10.533 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:10 smithi136 podman[36003]: 2024-09-21 01:56:10.202334835 +0000 UTC m=+0.365516672 container start 4917858bb7a6c3e3a7c42142a26e9733d5ea2ea5937ccd65d199c3624679f791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T01:56:10.533 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mgr[36022]: -- 172.21.15.136:0/4273545832 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bccc1690e0 con 0x55bcd0f0c000 2024-09-21T01:56:10.533 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:10 smithi136 bash[36003]: 4917858bb7a6c3e3a7c42142a26e9733d5ea2ea5937ccd65d199c3624679f791 2024-09-21T01:56:10.533 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 01:56:10 smithi136 systemd[1]: Started Ceph mgr.z for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:56:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2845848474' entity='client.admin' 2024-09-21T01:56:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:56:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:10 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2017806095' entity='client.admin' 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2845848474' entity='client.admin' 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:11.156 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:56:11.157 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:11.157 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:11.157 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:10 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2017806095' entity='client.admin' 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2845848474' entity='client.admin' 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:56:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:56:11.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:56:11.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:11.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:10 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2017806095' entity='client.admin' 2024-09-21T01:56:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 -- 172.21.15.92:0/3219175293 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24b403fb20 msgr2=0x7f24b4042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 --2- 172.21.15.92:0/3219175293 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24b403fb20 0x7f24b4042030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24c40058c0 tx=0x7f24c4005850 comp rx=0 tx=0).stop 2024-09-21T01:56:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 -- 172.21.15.92:0/3219175293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c81040a0 msgr2=0x7f24c819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 --2- 172.21.15.92:0/3219175293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c81040a0 0x7f24c819a030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c000b260 comp rx=0 tx=0).stop 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 -- 172.21.15.92:0/3219175293 shutdown_connections 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 --2- 172.21.15.92:0/3219175293 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24b403fb20 0x7f24b4042030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 --2- 172.21.15.92:0/3219175293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c81040a0 0x7f24c819a030 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 --2- 172.21.15.92:0/3219175293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24c8103700 0x7f24c8199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 --2- 172.21.15.92:0/3219175293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c8102670 0x7f24c8198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 -- 172.21.15.92:0/3219175293 >> 172.21.15.92:0/3219175293 conn(0x7f24c80fdfe0 msgr2=0x7f24c8106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 -- 172.21.15.92:0/3219175293 shutdown_connections 2024-09-21T01:56:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:11.447+0000 7f24d0341640 1 -- 172.21.15.92:0/3219175293 wait complete. 2024-09-21T01:56:11.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:11 smithi099 ceph-mon[34404]: Deploying daemon mgr.y on smithi099 2024-09-21T01:56:11.971 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:11 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3219175293' entity='client.admin' 2024-09-21T01:56:11.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:11 smithi092 ceph-mon[31190]: Deploying daemon mgr.y on smithi099 2024-09-21T01:56:11.979 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:11 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3219175293' entity='client.admin' 2024-09-21T01:56:12.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:11 smithi136 ceph-mon[34754]: Deploying daemon mgr.y on smithi099 2024-09-21T01:56:12.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:11 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3219175293' entity='client.admin' 2024-09-21T01:56:12.331 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 systemd[1]: Starting Ceph mgr.y for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T01:56:12.618 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 podman[35207]: 2024-09-21 01:56:12.251864848 +0000 UTC m=+0.015267921 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:56:12.618 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 podman[35207]: 2024-09-21 01:56:12.357134704 +0000 UTC m=+0.120537834 container create 9c2cd6367ad0a9d1909c7a45481ea513fe6270d7542084f78e22dcb9dfedf44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T01:56:12.618 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 podman[35207]: 2024-09-21 01:56:12.542733104 +0000 UTC m=+0.306136176 container init 9c2cd6367ad0a9d1909c7a45481ea513fe6270d7542084f78e22dcb9dfedf44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T01:56:12.618 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 podman[35207]: 2024-09-21 01:56:12.545419025 +0000 UTC m=+0.308822086 container start 9c2cd6367ad0a9d1909c7a45481ea513fe6270d7542084f78e22dcb9dfedf44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:56:12.618 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mgr[35225]: -- 172.21.15.99:0/3901533910 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2f700b0e0 con 0x55c2fbdae000 2024-09-21T01:56:12.618 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 bash[35207]: 9c2cd6367ad0a9d1909c7a45481ea513fe6270d7542084f78e22dcb9dfedf44f 2024-09-21T01:56:12.888 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 01:56:12 smithi099 systemd[1]: Started Ceph mgr.y for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T01:56:13.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:12 smithi092 ceph-mon[31190]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:13.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.013 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.014 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 -- 172.21.15.92:0/3332067216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b81036d0 msgr2=0x7f23b8103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/3332067216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b81036d0 0x7f23b8103ae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009900 tx=0x7f23a802a480 comp rx=0 tx=0).stop 2024-09-21T01:56:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 -- 172.21.15.92:0/3332067216 shutdown_connections 2024-09-21T01:56:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/3332067216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 0x7f23b810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/3332067216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b81036d0 0x7f23b8103ae0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/3332067216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b8102640 0x7f23b8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.012+0000 7f23bfbd8640 1 -- 172.21.15.92:0/3332067216 >> 172.21.15.92:0/3332067216 conn(0x7f23b80fdfb0 msgr2=0x7f23b8100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.013+0000 7f23bfbd8640 1 -- 172.21.15.92:0/3332067216 shutdown_connections 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.013+0000 7f23bfbd8640 1 -- 172.21.15.92:0/3332067216 wait complete. 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.013+0000 7f23bfbd8640 1 Processor -- start 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.013+0000 7f23bfbd8640 1 -- start start 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.013+0000 7f23bfbd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 0x7f23b806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.013+0000 7f23bfbd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b81036d0 0x7f23b819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bfbd8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 0x7f23b81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bfbd8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23b810e290 con 0x7f23b8102640 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bfbd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23b810e110 con 0x7f23b81036d0 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bfbd8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23b810e410 con 0x7f23b8104070 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 0x7f23b806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 0x7f23b806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45326/0 (socket says 172.21.15.92:45326) 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 -- 172.21.15.92:0/2644630883 learned_addr learned my addr 172.21.15.92:0/2644630883 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd173640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b81036d0 0x7f23b819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23be175640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 0x7f23b81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 -- 172.21.15.92:0/2644630883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 msgr2=0x7f23b81a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 0x7f23b81a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 -- 172.21.15.92:0/2644630883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b81036d0 msgr2=0x7f23b819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b81036d0 0x7f23b819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 -- 172.21.15.92:0/2644630883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81a1a80 con 0x7f23b8102640 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23be175640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 0x7f23b81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bd974640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 0x7f23b806b6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e9f0 tx=0x7f23b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b400cd80 con 0x7f23b8102640 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b40108e0 con 0x7f23b8102640 2024-09-21T01:56:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4018980 con 0x7f23b8102640 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b81a1d70 con 0x7f23b8102640 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.014+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b81a21f0 con 0x7f23b8102640 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.015+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f23b4010430 con 0x7f23b8102640 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.015+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b8109800 con 0x7f23b8102640 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.016+0000 7f23aeffd640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f23a403fb20 0x7f23a4042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.016+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23b4054920 con 0x7f23b8102640 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.016+0000 7f23bd173640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f23a403fb20 0x7f23a4042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.016+0000 7f23bd173640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f23a403fb20 0x7f23a4042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23a8000c00 tx=0x7f23a8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.023+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f23b400b2c0 con 0x7f23b8102640 2024-09-21T01:56:13.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.217+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f23b802e640 con 0x7f23b8102640 2024-09-21T01:56:13.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.221+0000 7f23aeffd640 1 -- 172.21.15.92:0/2644630883 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f23b4010a90 con 0x7f23b8102640 2024-09-21T01:56:13.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f23a403fb20 msgr2=0x7f23a4042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f23a403fb20 0x7f23a4042030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23a8000c00 tx=0x7f23a8031400 comp rx=0 tx=0).stop 2024-09-21T01:56:13.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 msgr2=0x7f23b806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 0x7f23b806b6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e9f0 tx=0x7f23b400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 shutdown_connections 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f23a403fb20 0x7f23a4042030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b8104070 0x7f23b81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b81036d0 0x7f23b819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 --2- 172.21.15.92:0/2644630883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b8102640 0x7f23b806b6c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 >> 172.21.15.92:0/2644630883 conn(0x7f23b80fdfb0 msgr2=0x7f23b81069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 shutdown_connections 2024-09-21T01:56:13.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.224+0000 7f23bfbd8640 1 -- 172.21.15.92:0/2644630883 wait complete. 2024-09-21T01:56:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:12 smithi136 ceph-mon[34754]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mon[34404]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:13.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.921+0000 7f415605d640 1 -- 172.21.15.92:0/2589299030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150104810 msgr2=0x7f4150104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.921+0000 7f415605d640 1 --2- 172.21.15.92:0/2589299030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150104810 0x7f4150104c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4144009900 tx=0x7f414402a480 comp rx=0 tx=0).stop 2024-09-21T01:56:13.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 -- 172.21.15.92:0/2589299030 shutdown_connections 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 --2- 172.21.15.92:0/2589299030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41501051b0 0x7f415010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 --2- 172.21.15.92:0/2589299030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150104810 0x7f4150104c20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 --2- 172.21.15.92:0/2589299030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4150103780 0x7f4150103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 -- 172.21.15.92:0/2589299030 >> 172.21.15.92:0/2589299030 conn(0x7f41500ff0f0 msgr2=0x7f41501015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 -- 172.21.15.92:0/2589299030 shutdown_connections 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 -- 172.21.15.92:0/2589299030 wait complete. 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.922+0000 7f415605d640 1 Processor -- start 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 -- start start 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 0x7f415019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4150104810 0x7f415019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41501051b0 0x7f415019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f415010f3d0 con 0x7f4150103780 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f415010f250 con 0x7f41501051b0 2024-09-21T01:56:13.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f415605d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f415010f550 con 0x7f4150104810 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 0x7f415019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 0x7f415019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45302/0 (socket says 172.21.15.92:45302) 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 -- 172.21.15.92:0/3196222701 learned_addr learned my addr 172.21.15.92:0/3196222701 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414effd640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4150104810 0x7f415019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414ffff640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41501051b0 0x7f415019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 -- 172.21.15.92:0/3196222701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4150104810 msgr2=0x7f415019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4150104810 0x7f415019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 -- 172.21.15.92:0/3196222701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41501051b0 msgr2=0x7f415019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41501051b0 0x7f415019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 -- 172.21.15.92:0/3196222701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415019ba10 con 0x7f4150103780 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414effd640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4150104810 0x7f415019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.923+0000 7f414f7fe640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 0x7f415019a050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f414000e9f0 tx=0x7f414000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.924+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414000cd80 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.924+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415019bd00 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.924+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41400054d0 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.924+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140010690 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.924+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41501a2400 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.925+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f41400108f0 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.925+0000 7f414cff9640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f413803fb20 0x7f4138042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.925+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f414005b060 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.925+0000 7f414effd640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f413803fb20 0x7f4138042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.925+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4150103bf0 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.925+0000 7f414effd640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f413803fb20 0x7f4138042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4144031440 tx=0x7f4144005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:12.930+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41400046c0 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.122+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f4150063fc0 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.125+0000 7f414cff9640 1 -- 172.21.15.92:0/3196222701 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f4140019d40 con 0x7f4150103780 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f413803fb20 msgr2=0x7f4138042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f413803fb20 0x7f4138042030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4144031440 tx=0x7f4144005d50 comp rx=0 tx=0).stop 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 msgr2=0x7f415019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 0x7f415019a050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f414000e9f0 tx=0x7f414000edf0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 shutdown_connections 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f413803fb20 0x7f4138042030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41501051b0 0x7f415019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4150104810 0x7f415019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 --2- 172.21.15.92:0/3196222701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4150103780 0x7f415019a050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 >> 172.21.15.92:0/3196222701 conn(0x7f41500ff0f0 msgr2=0x7f4150107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 shutdown_connections 2024-09-21T01:56:13.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.128+0000 7f415605d640 1 -- 172.21.15.92:0/3196222701 wait complete. 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 -- 172.21.15.92:0/1719734824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f406b980 msgr2=0x7f17f4106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 --2- 172.21.15.92:0/1719734824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f406b980 0x7f17f4106390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009900 tx=0x7f17e402a410 comp rx=0 tx=0).stop 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 -- 172.21.15.92:0/1719734824 shutdown_connections 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 --2- 172.21.15.92:0/1719734824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17f4109470 0x7f17f410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 --2- 172.21.15.92:0/1719734824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41068d0 0x7f17f4108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 --2- 172.21.15.92:0/1719734824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f406b980 0x7f17f4106390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 -- 172.21.15.92:0/1719734824 >> 172.21.15.92:0/1719734824 conn(0x7f17f40fdfb0 msgr2=0x7f17f4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 -- 172.21.15.92:0/1719734824 shutdown_connections 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.929+0000 7f17fa61e640 1 -- 172.21.15.92:0/1719734824 wait complete. 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 Processor -- start 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 -- start start 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17f406b980 0x7f17f41039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 0x7f17f4102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f4109470 0x7f17f41027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410f380 con 0x7f17f41068d0 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410f200 con 0x7f17f4109470 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17fa61e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17f410f500 con 0x7f17f406b980 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 0x7f17f4102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 0x7f17f4102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54076/0 (socket says 172.21.15.92:54076) 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17f37fe640 1 -- 172.21.15.92:0/477784066 learned_addr learned my addr 172.21.15.92:0/477784066 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.930+0000 7f17f8bbb640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f4109470 0x7f17f41027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f37fe640 1 -- 172.21.15.92:0/477784066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17f406b980 msgr2=0x7f17f41039c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f37fe640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17f406b980 0x7f17f41039c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f37fe640 1 -- 172.21.15.92:0/477784066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f4109470 msgr2=0x7f17f41027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f37fe640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f4109470 0x7f17f41027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f37fe640 1 -- 172.21.15.92:0/477784066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f4102ed0 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f37fe640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 0x7f17f4102290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d950 tx=0x7f17e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8017030 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e8004020 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80047a0 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f41a4000 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.931+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f41a4460 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.932+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f40ffad0 con 0x7f17f41068d0 2024-09-21T01:56:13.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.933+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f17e8004940 con 0x7f17f41068d0 2024-09-21T01:56:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.933+0000 7f17f17fa640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f17dc03fb20 0x7f17dc042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.933+0000 7f17f3fff640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f17dc03fb20 0x7f17dc042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.933+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f17e8053fd0 con 0x7f17f41068d0 2024-09-21T01:56:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.934+0000 7f17f3fff640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f17dc03fb20 0x7f17dc042030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17e402a930 tx=0x7f17e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.939+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17e8004240 con 0x7f17f41068d0 2024-09-21T01:56:14.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.142+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f17f402e640 con 0x7f17f41068d0 2024-09-21T01:56:14.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.145+0000 7f17f17fa640 1 -- 172.21.15.92:0/477784066 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v20)=0 v20) ==== 108+0+0 (secure 0 0 0) 0x7f17e801ad70 con 0x7f17f41068d0 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f17dc03fb20 msgr2=0x7f17dc042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f17dc03fb20 0x7f17dc042030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17e402a930 tx=0x7f17e4000bc0 comp rx=0 tx=0).stop 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 msgr2=0x7f17f4102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 0x7f17f4102290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d950 tx=0x7f17e800de10 comp rx=0 tx=0).stop 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 shutdown_connections 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f17dc03fb20 0x7f17dc042030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f4109470 0x7f17f41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f41068d0 0x7f17f4102290 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 --2- 172.21.15.92:0/477784066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17f406b980 0x7f17f41039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 >> 172.21.15.92:0/477784066 conn(0x7f17f40fdfb0 msgr2=0x7f17f4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 shutdown_connections 2024-09-21T01:56:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7f17fa61e640 1 -- 172.21.15.92:0/477784066 wait complete. 2024-09-21T01:56:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:14.255 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3196222701' entity='client.admin' 2024-09-21T01:56:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2644630883' entity='client.admin' 2024-09-21T01:56:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:14.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:14.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:14.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:14 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/213493142' entity='client.admin' 2024-09-21T01:56:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3196222701' entity='client.admin' 2024-09-21T01:56:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2644630883' entity='client.admin' 2024-09-21T01:56:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:14 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/213493142' entity='client.admin' 2024-09-21T01:56:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3196222701' entity='client.admin' 2024-09-21T01:56:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2644630883' entity='client.admin' 2024-09-21T01:56:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:14 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/213493142' entity='client.admin' 2024-09-21T01:56:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:15 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/477784066' entity='client.admin' 2024-09-21T01:56:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:15 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/752926250' entity='client.admin' 2024-09-21T01:56:15.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:15 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/477784066' entity='client.admin' 2024-09-21T01:56:15.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:15 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/752926250' entity='client.admin' 2024-09-21T01:56:15.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 -- 172.21.15.92:0/2965448416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec06c340 msgr2=0x7fa6ec1063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2965448416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec06c340 0x7fa6ec1063c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009ca0 tx=0x7fa6dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 -- 172.21.15.92:0/2965448416 shutdown_connections 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2965448416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec10b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2965448416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 0x7fa6ec108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2965448416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec06c340 0x7fa6ec1063c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 -- 172.21.15.92:0/2965448416 >> 172.21.15.92:0/2965448416 conn(0x7fa6ec0fdfe0 msgr2=0x7fa6ec100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 -- 172.21.15.92:0/2965448416 shutdown_connections 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.554+0000 7fa6f3311640 1 -- 172.21.15.92:0/2965448416 wait complete. 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 Processor -- start 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 -- start start 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ec06c340 0x7fa6ec1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 0x7fa6ec1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10f3b0 con 0x7fa6ec1094a0 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10f230 con 0x7fa6ec106900 2024-09-21T01:56:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f3311640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10f530 con 0x7fa6ec06c340 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f08ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 0x7fa6ec1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec102800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54122/0 (socket says 172.21.15.92:54122) 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 -- 172.21.15.92:0/2499949359 learned_addr learned my addr 172.21.15.92:0/2499949359 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f10ad640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ec06c340 0x7fa6ec1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 -- 172.21.15.92:0/2499949359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ec06c340 msgr2=0x7fa6ec1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ec06c340 0x7fa6ec1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 -- 172.21.15.92:0/2499949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 msgr2=0x7fa6ec1022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 0x7fa6ec1022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f18ae640 1 -- 172.21.15.92:0/2499949359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec102f00 con 0x7fa6ec1094a0 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.555+0000 7fa6f08ac640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 0x7fa6ec1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.556+0000 7fa6f18ae640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec102800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c970 tx=0x7fa6e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.556+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4007bb0 con 0x7fa6ec1094a0 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.556+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e4007d50 con 0x7fa6ec1094a0 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.556+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec1a4060 con 0x7fa6ec1094a0 2024-09-21T01:56:15.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.556+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4004e60 con 0x7fa6ec1094a0 2024-09-21T01:56:15.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.556+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1a44c0 con 0x7fa6ec1094a0 2024-09-21T01:56:15.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.557+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec0ffb00 con 0x7fa6ec1094a0 2024-09-21T01:56:15.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.557+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fa6e400f450 con 0x7fa6ec1094a0 2024-09-21T01:56:15.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.557+0000 7fa6ea7fc640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6d403fb20 0x7fa6d4042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.557+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6e4054490 con 0x7fa6ec1094a0 2024-09-21T01:56:15.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.557+0000 7fa6f10ad640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6d403fb20 0x7fa6d4042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.558+0000 7fa6f10ad640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6d403fb20 0x7fa6d4042030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0094e0 tx=0x7fa6dc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.562+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa6e4014030 con 0x7fa6ec1094a0 2024-09-21T01:56:15.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:15 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/477784066' entity='client.admin' 2024-09-21T01:56:15.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:15 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/752926250' entity='client.admin' 2024-09-21T01:56:15.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.744+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fa6ec02e640 con 0x7fa6ec1094a0 2024-09-21T01:56:15.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.748+0000 7fa6ea7fc640 1 -- 172.21.15.92:0/2499949359 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fa6e400dde0 con 0x7fa6ec1094a0 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6d403fb20 msgr2=0x7fa6d4042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6d403fb20 0x7fa6d4042030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0094e0 tx=0x7fa6dc004a40 comp rx=0 tx=0).stop 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 msgr2=0x7fa6ec102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec102800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c970 tx=0x7fa6e400ce30 comp rx=0 tx=0).stop 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 shutdown_connections 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6d403fb20 0x7fa6d4042030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ec1094a0 0x7fa6ec102800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6ec106900 0x7fa6ec1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 --2- 172.21.15.92:0/2499949359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ec06c340 0x7fa6ec1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 >> 172.21.15.92:0/2499949359 conn(0x7fa6ec0fdfe0 msgr2=0x7fa6ec104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 shutdown_connections 2024-09-21T01:56:15.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:15.750+0000 7fa6f3311640 1 -- 172.21.15.92:0/2499949359 wait complete. 2024-09-21T01:56:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 -- 172.21.15.92:0/1479966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd01037e0 msgr2=0x7fccd0103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 --2- 172.21.15.92:0/1479966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd01037e0 0x7fccd0103c50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009900 tx=0x7fccc002a480 comp rx=0 tx=0).stop 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 -- 172.21.15.92:0/1479966778 shutdown_connections 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 --2- 172.21.15.92:0/1479966778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd0105210 0x7fccd010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 --2- 172.21.15.92:0/1479966778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd0104870 0x7fccd0104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 --2- 172.21.15.92:0/1479966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd01037e0 0x7fccd0103c50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 -- 172.21.15.92:0/1479966778 >> 172.21.15.92:0/1479966778 conn(0x7fccd00ff150 msgr2=0x7fccd0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 -- 172.21.15.92:0/1479966778 shutdown_connections 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.148+0000 7fccd679c640 1 -- 172.21.15.92:0/1479966778 wait complete. 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 Processor -- start 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 -- start start 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd01037e0 0x7fccd0199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd0104870 0x7fccd019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 0x7fccd019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010f3b0 con 0x7fccd0105210 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010f230 con 0x7fccd01037e0 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd679c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fccd010f530 con 0x7fccd0104870 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 0x7fccd019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 0x7fccd019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54104/0 (socket says 172.21.15.92:54104) 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fcccffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd01037e0 0x7fccd0199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 -- 172.21.15.92:0/752926250 learned_addr learned my addr 172.21.15.92:0/752926250 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccc7fff640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd0104870 0x7fccd019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 -- 172.21.15.92:0/752926250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd0104870 msgr2=0x7fccd019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd0104870 0x7fccd019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 -- 172.21.15.92:0/752926250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd01037e0 msgr2=0x7fccd0199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd01037e0 0x7fccd0199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fccd4d39640 1 -- 172.21.15.92:0/752926250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd019b9f0 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.149+0000 7fcccffff640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd01037e0 0x7fccd0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.150+0000 7fccd4d39640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 0x7fccd019b1d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccc800eb10 tx=0x7fccc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.150+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8004b20 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.150+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd019bce0 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.150+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc8004cc0 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.150+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8010690 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.150+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd01a23e0 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.151+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fccc80108f0 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.151+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd0103c50 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.152+0000 7fcccdffb640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccbc03fb20 0x7fccbc042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.152+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fccc8054a70 con 0x7fccd0105210 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.152+0000 7fcccffff640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccbc03fb20 0x7fccbc042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.152+0000 7fcccffff640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccbc03fb20 0x7fccbc042030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:15.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.157+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fccc8016330 con 0x7fccd0105210 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.358+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fccd010a9c0 con 0x7fccd0105210 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.362+0000 7fcccdffb640 1 -- 172.21.15.92:0/752926250 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v21)=0 v21) ==== 116+0+0 (secure 0 0 0) 0x7fccc801b160 con 0x7fccd0105210 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.364+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccbc03fb20 msgr2=0x7fccbc042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.364+0000 7fccd679c640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccbc03fb20 0x7fccbc042030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc0031400 comp rx=0 tx=0).stop 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.364+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 msgr2=0x7fccd019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.364+0000 7fccd679c640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 0x7fccd019b1d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccc800eb10 tx=0x7fccc800c2d0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 shutdown_connections 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccbc03fb20 0x7fccbc042030 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccd0105210 0x7fccd019b1d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd0104870 0x7fccd019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 --2- 172.21.15.92:0/752926250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccd01037e0 0x7fccd0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 >> 172.21.15.92:0/752926250 conn(0x7fccd00ff150 msgr2=0x7fccd0107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 shutdown_connections 2024-09-21T01:56:15.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:14.365+0000 7fccd679c640 1 -- 172.21.15.92:0/752926250 wait complete. 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 -- 172.21.15.92:0/1048007485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5001040c0 msgr2=0x7fe50010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 --2- 172.21.15.92:0/1048007485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5001040c0 0x7fe50010aa60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009ca0 tx=0x7fe4f802a950 comp rx=0 tx=0).stop 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 -- 172.21.15.92:0/1048007485 shutdown_connections 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 --2- 172.21.15.92:0/1048007485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5001040c0 0x7fe50010aa60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 --2- 172.21.15.92:0/1048007485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500103720 0x7fe500103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 --2- 172.21.15.92:0/1048007485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500102690 0x7fe500102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 -- 172.21.15.92:0/1048007485 >> 172.21.15.92:0/1048007485 conn(0x7fe5000fe000 msgr2=0x7fe5001004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 -- 172.21.15.92:0/1048007485 shutdown_connections 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.731+0000 7fe50518b640 1 -- 172.21.15.92:0/1048007485 wait complete. 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.732+0000 7fe50518b640 1 Processor -- start 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.732+0000 7fe50518b640 1 -- start start 2024-09-21T01:56:16.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500102690 0x7fe500198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 0x7fe500199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5001040c0 0x7fe50019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe50010e330 con 0x7fe500103720 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe50010e1b0 con 0x7fe5001040c0 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe50010e4b0 con 0x7fe500102690 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 0x7fe500199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 0x7fe500199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54054/0 (socket says 172.21.15.92:54054) 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 -- 172.21.15.92:0/213493142 learned_addr learned my addr 172.21.15.92:0/213493142 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fed9d640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500102690 0x7fe500198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4ff59e640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5001040c0 0x7fe50019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 -- 172.21.15.92:0/213493142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500102690 msgr2=0x7fe500198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500102690 0x7fe500198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 -- 172.21.15.92:0/213493142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5001040c0 msgr2=0x7fe50019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5001040c0 0x7fe50019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 -- 172.21.15.92:0/213493142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50019a8d0 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4fe59c640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 0x7fe500199510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400e9f0 tx=0x7fe4f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f400cd80 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f40108e0 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50019abc0 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.733+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4018980 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.734+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50006bcf0 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.734+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7fe4f4018b20 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.735+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe500102b00 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.735+0000 7fe4e7fff640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4e803fb20 0x7fe4e8042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.735+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe4f4054950 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.735+0000 7fe4fed9d640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4e803fb20 0x7fe4e8042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.735+0000 7fe4fed9d640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4e803fb20 0x7fe4e8042030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009bd0 tx=0x7fe4f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.740+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4f4018e10 con 0x7fe500103720 2024-09-21T01:56:16.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.938+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fe500109380 con 0x7fe500103720 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.942+0000 7fe4e7fff640 1 -- 172.21.15.92:0/213493142 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v19)=0 v19) ==== 116+0+0 (secure 0 0 0) 0x7fe4f4010d70 con 0x7fe500103720 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4e803fb20 msgr2=0x7fe4e8042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4e803fb20 0x7fe4e8042030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009bd0 tx=0x7fe4f0005f90 comp rx=0 tx=0).stop 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 msgr2=0x7fe500199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 0x7fe500199510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400e9f0 tx=0x7fe4f400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 shutdown_connections 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4e803fb20 0x7fe4e8042030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5001040c0 0x7fe50019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500103720 0x7fe500199510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 --2- 172.21.15.92:0/213493142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500102690 0x7fe500198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 >> 172.21.15.92:0/213493142 conn(0x7fe5000fe000 msgr2=0x7fe500106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 shutdown_connections 2024-09-21T01:56:16.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:13.945+0000 7fe50518b640 1 -- 172.21.15.92:0/213493142 wait complete. 2024-09-21T01:56:16.263 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:16.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: Standby manager daemon z started 2024-09-21T01:56:16.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:56:16.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:56:16.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:56:16.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:56:16.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:16 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2499949359' entity='client.admin' 2024-09-21T01:56:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: Standby manager daemon z started 2024-09-21T01:56:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:56:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:56:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:56:16.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:56:16.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:16 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2499949359' entity='client.admin' 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: Standby manager daemon z started 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.136:0/603829570' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:56:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:16 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2499949359' entity='client.admin' 2024-09-21T01:56:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.753+0000 7f84a92f8640 1 -- 172.21.15.92:0/2498972003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 msgr2=0x7f84a4104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.753+0000 7f84a92f8640 1 --2- 172.21.15.92:0/2498972003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a4104bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c02a480 comp rx=0 tx=0).stop 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 -- 172.21.15.92:0/2498972003 shutdown_connections 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 --2- 172.21.15.92:0/2498972003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4105180 0x7f84a410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 --2- 172.21.15.92:0/2498972003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a4104bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 --2- 172.21.15.92:0/2498972003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4103750 0x7f84a4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 -- 172.21.15.92:0/2498972003 >> 172.21.15.92:0/2498972003 conn(0x7f84a40ff0c0 msgr2=0x7f84a4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 -- 172.21.15.92:0/2498972003 shutdown_connections 2024-09-21T01:56:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 -- 172.21.15.92:0/2498972003 wait complete. 2024-09-21T01:56:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 Processor -- start 2024-09-21T01:56:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.754+0000 7f84a92f8640 1 -- start start 2024-09-21T01:56:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4103750 0x7f84a419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4105180 0x7f84a419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410f3f0 con 0x7f84a41047e0 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410f270 con 0x7f84a4103750 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410f570 con 0x7f84a4105180 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54152/0 (socket says 172.21.15.92:54152) 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 -- 172.21.15.92:0/4186628063 learned_addr learned my addr 172.21.15.92:0/4186628063 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a2ffd640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4103750 0x7f84a419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a37fe640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4105180 0x7f84a419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 -- 172.21.15.92:0/4186628063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4105180 msgr2=0x7f84a419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4105180 0x7f84a419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 -- 172.21.15.92:0/4186628063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4103750 msgr2=0x7f84a419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4103750 0x7f84a419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 -- 172.21.15.92:0/4186628063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419ba80 con 0x7f84a41047e0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f849a7fc640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a419a6c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f848c031440 tx=0x7f848c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c03f070 con 0x7f84a41047e0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c031c50 con 0x7f84a41047e0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.755+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419bd10 con 0x7f84a41047e0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.756+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c035030 con 0x7f84a41047e0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.756+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a22a0 con 0x7f84a41047e0 2024-09-21T01:56:16.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.757+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a410a930 con 0x7f84a41047e0 2024-09-21T01:56:16.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.757+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f848c042070 con 0x7f84a41047e0 2024-09-21T01:56:16.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.758+0000 7f84a0ff9640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f849007afd0 0x7f849007d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.758+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f848c0bd670 con 0x7f84a41047e0 2024-09-21T01:56:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.758+0000 7f84a2ffd640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f849007afd0 0x7f849007d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.758+0000 7f84a2ffd640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f849007afd0 0x7f849007d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8494005fd0 tx=0x7f849400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.762+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f848c08a0d0 con 0x7f84a41047e0 2024-09-21T01:56:16.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.944+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f84a402e640 con 0x7f84a41047e0 2024-09-21T01:56:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.948+0000 7f84a0ff9640 1 -- 172.21.15.92:0/4186628063 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f848c08ef80 con 0x7f84a41047e0 2024-09-21T01:56:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f849007afd0 msgr2=0x7f849007d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f849007afd0 0x7f849007d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8494005fd0 tx=0x7f849400b390 comp rx=0 tx=0).stop 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 msgr2=0x7f84a419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a419a6c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f848c031440 tx=0x7f848c003c10 comp rx=0 tx=0).stop 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 shutdown_connections 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f849007afd0 0x7f849007d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4105180 0x7f84a419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a41047e0 0x7f84a419a6c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 --2- 172.21.15.92:0/4186628063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4103750 0x7f84a419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 >> 172.21.15.92:0/4186628063 conn(0x7f84a40ff0c0 msgr2=0x7f84a4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 shutdown_connections 2024-09-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.951+0000 7f84a92f8640 1 -- 172.21.15.92:0/4186628063 wait complete. 2024-09-21T01:56:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:17 smithi092 ceph-mon[31190]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-21T01:56:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:56:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:17 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2365500471' entity='client.admin' 2024-09-21T01:56:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:17 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4186628063' entity='client.admin' 2024-09-21T01:56:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:17 smithi136 ceph-mon[34754]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-21T01:56:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:56:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:17 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2365500471' entity='client.admin' 2024-09-21T01:56:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:17 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4186628063' entity='client.admin' 2024-09-21T01:56:17.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:17 smithi099 ceph-mon[34404]: mgrmap e14: x(active, since 65s), standbys: z 2024-09-21T01:56:17.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:56:17.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:17.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:17 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2365500471' entity='client.admin' 2024-09-21T01:56:17.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:17 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4186628063' entity='client.admin' 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 -- 172.21.15.92:0/2736825506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de01068e0 msgr2=0x7f1de0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 --2- 172.21.15.92:0/2736825506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de01068e0 0x7f1de0108d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009900 tx=0x7f1dd002a410 comp rx=0 tx=0).stop 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 -- 172.21.15.92:0/2736825506 shutdown_connections 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 --2- 172.21.15.92:0/2736825506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0109480 0x7f1de010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 --2- 172.21.15.92:0/2736825506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de01068e0 0x7f1de0108d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 --2- 172.21.15.92:0/2736825506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de006c340 0x7f1de01063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 -- 172.21.15.92:0/2736825506 >> 172.21.15.92:0/2736825506 conn(0x7f1de00fdfa0 msgr2=0x7f1de0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 -- 172.21.15.92:0/2736825506 shutdown_connections 2024-09-21T01:56:17.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.743+0000 7f1de7e61640 1 -- 172.21.15.92:0/2736825506 wait complete. 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 Processor -- start 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 -- start start 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 0x7f1de01039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de01068e0 0x7f1de01022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de0109480 0x7f1de01027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010f390 con 0x7f1de006c340 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010f210 con 0x7f1de01068e0 2024-09-21T01:56:17.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de7e61640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010f510 con 0x7f1de0109480 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 0x7f1de01039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 0x7f1de01039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54174/0 (socket says 172.21.15.92:54174) 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 -- 172.21.15.92:0/350103602 learned_addr learned my addr 172.21.15.92:0/350103602 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de53fc640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de01068e0 0x7f1de01022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de63fe640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de0109480 0x7f1de01027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 -- 172.21.15.92:0/350103602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de0109480 msgr2=0x7f1de01027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de0109480 0x7f1de01027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 -- 172.21.15.92:0/350103602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de01068e0 msgr2=0x7f1de01022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de01068e0 0x7f1de01022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.744+0000 7f1de5bfd640 1 -- 172.21.15.92:0/350103602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de0102ee0 con 0x7f1de006c340 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.745+0000 7f1de5bfd640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 0x7f1de01039d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00d770 tx=0x7f1ddc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.745+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc017070 con 0x7f1de006c340 2024-09-21T01:56:17.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.745+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de01a4050 con 0x7f1de006c340 2024-09-21T01:56:17.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.745+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ddc0054d0 con 0x7f1de006c340 2024-09-21T01:56:17.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.745+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc004c30 con 0x7f1de006c340 2024-09-21T01:56:17.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.745+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de01a44b0 con 0x7f1de006c340 2024-09-21T01:56:17.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.746+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de0106d30 con 0x7f1de006c340 2024-09-21T01:56:17.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.746+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f1ddc004e10 con 0x7f1de006c340 2024-09-21T01:56:17.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.747+0000 7f1dd6ffd640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1dcc07afd0 0x7f1dcc07d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:17.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.747+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ddc09c7d0 con 0x7f1de006c340 2024-09-21T01:56:17.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.747+0000 7f1de53fc640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1dcc07afd0 0x7f1dcc07d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:17.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.747+0000 7f1de53fc640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1dcc07afd0 0x7f1dcc07d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0000c00 tx=0x7f1dd0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:17.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.751+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ddc069230 con 0x7f1de006c340 2024-09-21T01:56:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.934+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f1de002e8c0 con 0x7f1de006c340 2024-09-21T01:56:17.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.938+0000 7f1dd6ffd640 1 -- 172.21.15.92:0/350103602 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f1ddc06e0e0 con 0x7f1de006c340 2024-09-21T01:56:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1dcc07afd0 msgr2=0x7f1dcc07d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1dcc07afd0 0x7f1dcc07d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0000c00 tx=0x7f1dd0031400 comp rx=0 tx=0).stop 2024-09-21T01:56:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 msgr2=0x7f1de01039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 0x7f1de01039d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00d770 tx=0x7f1ddc00dc30 comp rx=0 tx=0).stop 2024-09-21T01:56:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 shutdown_connections 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1dcc07afd0 0x7f1dcc07d4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de0109480 0x7f1de01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de01068e0 0x7f1de01022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.940+0000 7f1de7e61640 1 --2- 172.21.15.92:0/350103602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de006c340 0x7f1de01039d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.941+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 >> 172.21.15.92:0/350103602 conn(0x7f1de00fdfa0 msgr2=0x7f1de0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.941+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 shutdown_connections 2024-09-21T01:56:17.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:17.941+0000 7f1de7e61640 1 -- 172.21.15.92:0/350103602 wait complete. 2024-09-21T01:56:18.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2367616646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d68104070 msgr2=0x7f4d6810aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2367616646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d68104070 0x7f4d6810aa10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009ca0 tx=0x7f4d6002a950 comp rx=0 tx=0).stop 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2367616646 shutdown_connections 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2367616646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d68104070 0x7f4d6810aa10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2367616646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d681036d0 0x7f4d68103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2367616646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d68102640 0x7f4d68102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2367616646 >> 172.21.15.92:0/2367616646 conn(0x7f4d680fdfb0 msgr2=0x7f4d68100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2367616646 shutdown_connections 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.065+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2367616646 wait complete. 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 Processor -- start 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 -- start start 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d68102640 0x7f4d68198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 0x7f4d681994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d68104070 0x7f4d6819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810e290 con 0x7f4d681036d0 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810e110 con 0x7f4d68104070 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d6d6ea640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810e410 con 0x7f4d68102640 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d667fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 0x7f4d681994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d667fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 0x7f4d681994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54140/0 (socket says 172.21.15.92:54140) 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d667fc640 1 -- 172.21.15.92:0/2365500471 learned_addr learned my addr 172.21.15.92:0/2365500471 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d66ffd640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d68102640 0x7f4d68198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.066+0000 7f4d677fe640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d68104070 0x7f4d6819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d667fc640 1 -- 172.21.15.92:0/2365500471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d68102640 msgr2=0x7f4d68198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d667fc640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d68102640 0x7f4d68198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d667fc640 1 -- 172.21.15.92:0/2365500471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d68104070 msgr2=0x7f4d6819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d667fc640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d68104070 0x7f4d6819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d667fc640 1 -- 172.21.15.92:0/2365500471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6819a8b0 con 0x7f4d681036d0 2024-09-21T01:56:18.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d66ffd640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d68102640 0x7f4d68198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d667fc640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 0x7f4d681994f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000d770 tx=0x7f4d5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50017070 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d500054d0 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6819aba0 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50004c30 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.067+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681a12a0 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.068+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d68109800 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.068+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 51898+0+0 (secure 0 0 0) 0x7f4d50010460 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.068+0000 7f4d5ffff640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4d5403fb20 0x7f4d54042030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.068+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4d50053f40 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.068+0000 7f4d66ffd640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4d5403fb20 0x7f4d54042030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.069+0000 7f4d66ffd640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4d5403fb20 0x7f4d54042030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005d00 tx=0x7f4d5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.073+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d50009c30 con 0x7f4d681036d0 2024-09-21T01:56:18.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.154+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 14) ==== 103177+0+0 (secure 0 0 0) 0x7f4d50022cf0 con 0x7f4d681036d0 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.262+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f4d6802e640 con 0x7f4d681036d0 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.267+0000 7f4d5ffff640 1 -- 172.21.15.92:0/2365500471 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7f4d5006d1f0 con 0x7f4d681036d0 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.269+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4d5403fb20 msgr2=0x7f4d54042030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4d5403fb20 0x7f4d54042030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005d00 tx=0x7f4d5800b410 comp rx=0 tx=0).stop 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 msgr2=0x7f4d681994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 0x7f4d681994f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000d770 tx=0x7f4d5000dc30 comp rx=0 tx=0).stop 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 shutdown_connections 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4d5403fb20 0x7f4d54042030 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d68104070 0x7f4d6819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d681036d0 0x7f4d681994f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 --2- 172.21.15.92:0/2365500471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d68102640 0x7f4d68198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 >> 172.21.15.92:0/2365500471 conn(0x7f4d680fdfb0 msgr2=0x7f4d681069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 shutdown_connections 2024-09-21T01:56:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:16.270+0000 7f4d6d6ea640 1 -- 172.21.15.92:0/2365500471 wait complete. 2024-09-21T01:56:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: Standby manager daemon y started 2024-09-21T01:56:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:56:18.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:56:18.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:56:18.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:56:18.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:18 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/350103602' entity='client.admin' 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: Standby manager daemon y started 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:56:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:18 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/350103602' entity='client.admin' 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: Standby manager daemon y started 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: from='mgr.? 172.21.15.99:0/2261587838' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:56:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:18 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/350103602' entity='client.admin' 2024-09-21T01:56:19.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:19 smithi092 ceph-mon[31190]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-21T01:56:19.468 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:56:19.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:19 smithi099 ceph-mon[34404]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-21T01:56:19.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:56:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:19 smithi136 ceph-mon[34754]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-21T01:56:19.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1855453163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e841036d0 msgr2=0x7f9e84103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1855453163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e841036d0 0x7f9e84103ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009900 tx=0x7f9e7802a480 comp rx=0 tx=0).stop 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1855453163 shutdown_connections 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1855453163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e84104070 0x7f9e8410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1855453163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e841036d0 0x7f9e84103ae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1855453163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e84102640 0x7f9e84102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1855453163 >> 172.21.15.92:0/1855453163 conn(0x7f9e840fdfb0 msgr2=0x7f9e84100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1855453163 shutdown_connections 2024-09-21T01:56:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.102+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1855453163 wait complete. 2024-09-21T01:56:20.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 Processor -- start 2024-09-21T01:56:20.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 -- start start 2024-09-21T01:56:20.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e84102640 0x7f9e84199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e841036d0 0x7f9e8419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 0x7f9e8419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8410e2c0 con 0x7f9e84104070 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8410e140 con 0x7f9e84102640 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e8a1db640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8410e440 con 0x7f9e841036d0 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 0x7f9e8419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 0x7f9e8419b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54192/0 (socket says 172.21.15.92:54192) 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.103+0000 7f9e83fff640 1 -- 172.21.15.92:0/1626012877 learned_addr learned my addr 172.21.15.92:0/1626012877 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e837fe640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e84102640 0x7f9e84199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e82ffd640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e841036d0 0x7f9e8419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e83fff640 1 -- 172.21.15.92:0/1626012877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e841036d0 msgr2=0x7f9e8419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e83fff640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e841036d0 0x7f9e8419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e83fff640 1 -- 172.21.15.92:0/1626012877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e84102640 msgr2=0x7f9e84199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e83fff640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e84102640 0x7f9e84199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e83fff640 1 -- 172.21.15.92:0/1626012877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8419ba10 con 0x7f9e84104070 2024-09-21T01:56:20.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e82ffd640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e841036d0 0x7f9e8419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e83fff640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 0x7f9e8419b1f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00cad0 tx=0x7f9e7c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00ae20 con 0x7f9e84104070 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c005020 con 0x7f9e84104070 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8419bd00 con 0x7f9e84104070 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00f6b0 con 0x7f9e84104070 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.104+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e841a2400 con 0x7f9e84104070 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.106+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e841a2630 con 0x7f9e84104070 2024-09-21T01:56:20.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.107+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f9e7c0051d0 con 0x7f9e84104070 2024-09-21T01:56:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.107+0000 7f9e80ff9640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9e6c0b6500 0x7f9e6c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.108+0000 7f9e837fe640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9e6c0b6500 0x7f9e6c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.108+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9e7c0e5590 con 0x7f9e84104070 2024-09-21T01:56:20.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.108+0000 7f9e837fe640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9e6c0b6500 0x7f9e6c0b8a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e74005d00 tx=0x7f9e7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:20.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.112+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e7c014030 con 0x7f9e84104070 2024-09-21T01:56:20.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.296+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f9e841a2800 con 0x7f9e84104070 2024-09-21T01:56:20.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.300+0000 7f9e80ff9640 1 -- 172.21.15.92:0/1626012877 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v26)=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f9e7c0adaf0 con 0x7f9e84104070 2024-09-21T01:56:20.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9e6c0b6500 msgr2=0x7f9e6c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9e6c0b6500 0x7f9e6c0b8a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e74005d00 tx=0x7f9e7400b410 comp rx=0 tx=0).stop 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 msgr2=0x7f9e8419b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 0x7f9e8419b1f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00cad0 tx=0x7f9e7c00cf90 comp rx=0 tx=0).stop 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 shutdown_connections 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9e6c0b6500 0x7f9e6c0b8a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e84104070 0x7f9e8419b1f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e841036d0 0x7f9e8419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 --2- 172.21.15.92:0/1626012877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e84102640 0x7f9e84199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 >> 172.21.15.92:0/1626012877 conn(0x7f9e840fdfb0 msgr2=0x7f9e840ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 shutdown_connections 2024-09-21T01:56:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:20.302+0000 7f9e8a1db640 1 -- 172.21.15.92:0/1626012877 wait complete. 2024-09-21T01:56:20.543 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:20 smithi092 ceph-mon[31190]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:20.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:20 smithi099 ceph-mon[34404]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:20 smithi136 ceph-mon[34754]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:21.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:21 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1626012877' entity='client.admin' 2024-09-21T01:56:21.646 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:21 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1626012877' entity='client.admin' 2024-09-21T01:56:21.651 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:21.686 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi092') 2024-09-21T01:56:21.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:21 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1626012877' entity='client.admin' 2024-09-21T01:56:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:21.831 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi092') 2024-09-21T01:56:22.446 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:22 smithi092 ceph-mon[31190]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:22.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:22 smithi099 ceph-mon[34404]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:22.684 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:22 smithi136 ceph-mon[34754]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.466+0000 7f3ab5ced640 1 -- 172.21.15.92:0/869410531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 msgr2=0x7f3ab010b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.466+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/869410531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab010b9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa802a410 comp rx=0 tx=0).stop 2024-09-21T01:56:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 -- 172.21.15.92:0/869410531 shutdown_connections 2024-09-21T01:56:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/869410531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab010b9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/869410531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0106be0 0x7f3ab0109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/869410531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0101b40 0x7f3ab0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 -- 172.21.15.92:0/869410531 >> 172.21.15.92:0/869410531 conn(0x7f3ab00fd770 msgr2=0x7f3ab00ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 -- 172.21.15.92:0/869410531 shutdown_connections 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 -- 172.21.15.92:0/869410531 wait complete. 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.467+0000 7f3ab5ced640 1 Processor -- start 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 -- start start 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0101b40 0x7f3ab0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0106be0 0x7f3ab019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab010f3a0 con 0x7f3ab0109540 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab010f220 con 0x7f3ab0106be0 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3ab5ced640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab010f520 con 0x7f3ab0101b40 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:23.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaf7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0101b40 0x7f3ab0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaeffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0106be0 0x7f3ab019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab019b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52744/0 (socket says 172.21.15.92:52744) 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaf7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0101b40 0x7f3ab0199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:60270/0 (socket says 172.21.15.92:60270) 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaffff640 1 -- 172.21.15.92:0/3474592711 learned_addr learned my addr 172.21.15.92:0/3474592711 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaeffd640 1 -- 172.21.15.92:0/3474592711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0101b40 msgr2=0x7f3ab0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaeffd640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0101b40 0x7f3ab0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaeffd640 1 -- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 msgr2=0x7f3ab019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaeffd640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaeffd640 1 -- 172.21.15.92:0/3474592711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab019b9c0 con 0x7f3ab0106be0 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.468+0000 7f3aaffff640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.469+0000 7f3aaeffd640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0106be0 0x7f3ab019a600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400e9f0 tx=0x7f3aa400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.469+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400ccc0 con 0x7f3ab0106be0 2024-09-21T01:56:23.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.469+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab019bcb0 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.469+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa40108e0 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.469+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4018880 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.469+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab01a23b0 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.470+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab00ff290 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.471+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f3aa4010430 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.472+0000 7f3aacff9640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a980b6480 0x7f3a980b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.472+0000 7f3aaf7fe640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a980b6480 0x7f3a980b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.472+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3aa40e5b90 con 0x7f3ab0106be0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.473+0000 7f3aaf7fe640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a980b6480 0x7f3a980b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00099a0 tx=0x7f3aa0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:23.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.475+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3aa40b24b0 con 0x7f3ab0106be0 2024-09-21T01:56:23.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.659+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f3ab002e640 con 0x7f3ab0106be0 2024-09-21T01:56:23.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.667+0000 7f3aacff9640 1 -- 172.21.15.92:0/3474592711 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v27)=0 v27) ==== 127+0+0 (secure 0 0 0) 0x7f3aa40b73f0 con 0x7f3ab0106be0 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a980b6480 msgr2=0x7f3a980b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a980b6480 0x7f3a980b8990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00099a0 tx=0x7f3aa0009500 comp rx=0 tx=0).stop 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0106be0 msgr2=0x7f3ab019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0106be0 0x7f3ab019a600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400e9f0 tx=0x7f3aa400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 shutdown_connections 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a980b6480 0x7f3a980b8990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ab0109540 0x7f3ab019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0106be0 0x7f3ab019a600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 --2- 172.21.15.92:0/3474592711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ab0101b40 0x7f3ab0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 >> 172.21.15.92:0/3474592711 conn(0x7f3ab00fd770 msgr2=0x7f3ab01079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:23.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 shutdown_connections 2024-09-21T01:56:23.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:23.669+0000 7f3ab5ced640 1 -- 172.21.15.92:0/3474592711 wait complete. 2024-09-21T01:56:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 -- 172.21.15.92:0/1250946734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 msgr2=0x7f2b68102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/1250946734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68102b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009900 tx=0x7f2b6402a480 comp rx=0 tx=0).stop 2024-09-21T01:56:24.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 -- 172.21.15.92:0/1250946734 shutdown_connections 2024-09-21T01:56:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/1250946734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b681040c0 0x7f2b6810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/1250946734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b68103720 0x7f2b68103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/1250946734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68102b00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.678+0000 7f2b6ff19640 1 -- 172.21.15.92:0/1250946734 >> 172.21.15.92:0/1250946734 conn(0x7f2b680fe000 msgr2=0x7f2b681004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 -- 172.21.15.92:0/1250946734 shutdown_connections 2024-09-21T01:56:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 -- 172.21.15.92:0/1250946734 wait complete. 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 Processor -- start 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 -- start start 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b68103720 0x7f2b6819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b681040c0 0x7f2b6819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6810e2d0 con 0x7f2b68102690 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6810e150 con 0x7f2b68103720 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.679+0000 7f2b6ff19640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6810e450 con 0x7f2b681040c0 2024-09-21T01:56:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52770/0 (socket says 172.21.15.92:52770) 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 -- 172.21.15.92:0/2500131343 learned_addr learned my addr 172.21.15.92:0/2500131343 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6d4b4640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b68103720 0x7f2b6819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6e4b6640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b681040c0 0x7f2b6819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 -- 172.21.15.92:0/2500131343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b681040c0 msgr2=0x7f2b6819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b681040c0 0x7f2b6819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 -- 172.21.15.92:0/2500131343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b68103720 msgr2=0x7f2b6819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b68103720 0x7f2b6819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 -- 172.21.15.92:0/2500131343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6819ba20 con 0x7f2b68102690 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6dcb5640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68199ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2b64006010 tx=0x7f2b64031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6403f030 con 0x7f2b68102690 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b64003aa0 con 0x7f2b68102690 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6819bcb0 con 0x7f2b68102690 2024-09-21T01:56:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64035070 con 0x7f2b68102690 2024-09-21T01:56:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.680+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b681a2410 con 0x7f2b68102690 2024-09-21T01:56:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.681+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b68102b00 con 0x7f2b68102690 2024-09-21T01:56:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.683+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f2b64042030 con 0x7f2b68102690 2024-09-21T01:56:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.683+0000 7f2b5effd640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2b540b6500 0x7f2b540b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.683+0000 7f2b6d4b4640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2b540b6500 0x7f2b540b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.684+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2b641061a0 con 0x7f2b68102690 2024-09-21T01:56:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.684+0000 7f2b6d4b4640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2b540b6500 0x7f2b540b8a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005fd0 tx=0x7f2b5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:24.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.687+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2b640d2ae0 con 0x7f2b68102690 2024-09-21T01:56:24.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.871+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f2b681a2680 con 0x7f2b68102690 2024-09-21T01:56:24.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:24 smithi092 ceph-mon[31190]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:24.873 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:24 smithi092 ceph-mon[31190]: from='client.? ' entity='client.admin' 2024-09-21T01:56:24.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.876+0000 7f2b5effd640 1 -- 172.21.15.92:0/2500131343 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v28)=0 v28) ==== 119+0+0 (secure 0 0 0) 0x7f2b640d7a20 con 0x7f2b68102690 2024-09-21T01:56:24.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2b540b6500 msgr2=0x7f2b540b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2b540b6500 0x7f2b540b8a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005fd0 tx=0x7f2b5800b390 comp rx=0 tx=0).stop 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 msgr2=0x7f2b68199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68199ff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2b64006010 tx=0x7f2b64031a50 comp rx=0 tx=0).stop 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 shutdown_connections 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2b540b6500 0x7f2b540b8a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2b681040c0 0x7f2b6819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b68103720 0x7f2b6819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 --2- 172.21.15.92:0/2500131343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b68102690 0x7f2b68199ff0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 >> 172.21.15.92:0/2500131343 conn(0x7f2b680fe000 msgr2=0x7f2b680ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 shutdown_connections 2024-09-21T01:56:24.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.879+0000 7f2b6ff19640 1 -- 172.21.15.92:0/2500131343 wait complete. 2024-09-21T01:56:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 -- 172.21.15.92:0/2853418360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa698103770 msgr2=0x7fa698103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 --2- 172.21.15.92:0/2853418360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa698103770 0x7fa698103be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa694009900 tx=0x7fa69402a480 comp rx=0 tx=0).stop 2024-09-21T01:56:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 -- 172.21.15.92:0/2853418360 shutdown_connections 2024-09-21T01:56:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 --2- 172.21.15.92:0/2853418360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6981051a0 0x7fa69810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 --2- 172.21.15.92:0/2853418360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698104800 0x7fa698104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 --2- 172.21.15.92:0/2853418360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa698103770 0x7fa698103be0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 -- 172.21.15.92:0/2853418360 >> 172.21.15.92:0/2853418360 conn(0x7fa6980ff0e0 msgr2=0x7fa698101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 -- 172.21.15.92:0/2853418360 shutdown_connections 2024-09-21T01:56:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 -- 172.21.15.92:0/2853418360 wait complete. 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.888+0000 7fa69ff97640 1 Processor -- start 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 -- start start 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698103770 0x7fa698199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa698104800 0x7fa69819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 0x7fa69819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa69810f3b0 con 0x7fa6981051a0 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa69810f230 con 0x7fa698103770 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69ff97640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa69810f530 con 0x7fa698104800 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 0x7fa69819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69d532640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa698104800 0x7fa69819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69dd33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698103770 0x7fa698199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69d532640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa698104800 0x7fa69819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:60302/0 (socket says 172.21.15.92:60302) 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 0x7fa69819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52792/0 (socket says 172.21.15.92:52792) 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69d532640 1 -- 172.21.15.92:0/515526708 learned_addr learned my addr 172.21.15.92:0/515526708 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 -- 172.21.15.92:0/515526708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa698104800 msgr2=0x7fa69819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa698104800 0x7fa69819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 -- 172.21.15.92:0/515526708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698103770 msgr2=0x7fa698199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698103770 0x7fa698199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69e534640 1 -- 172.21.15.92:0/515526708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69819b9d0 con 0x7fa6981051a0 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.889+0000 7fa69dd33640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698103770 0x7fa698199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.890+0000 7fa69e534640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 0x7fa69819b1b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa69000e940 tx=0x7fa69000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.890+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690004d20 con 0x7fa6981051a0 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.890+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa690005020 con 0x7fa6981051a0 2024-09-21T01:56:24.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.890+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900116d0 con 0x7fa6981051a0 2024-09-21T01:56:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.890+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69819bcc0 con 0x7fa6981051a0 2024-09-21T01:56:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.890+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981a23c0 con 0x7fa6981051a0 2024-09-21T01:56:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.891+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69810a950 con 0x7fa6981051a0 2024-09-21T01:56:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.892+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fa690005460 con 0x7fa6981051a0 2024-09-21T01:56:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.894+0000 7fa68effd640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6840b6500 0x7fa6840b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.894+0000 7fa69dd33640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6840b6500 0x7fa6840b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.894+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6900e5940 con 0x7fa6981051a0 2024-09-21T01:56:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.894+0000 7fa69dd33640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6840b6500 0x7fa6840b8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa694000c00 tx=0x7fa694031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:24.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:24.897+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa690016030 con 0x7fa6981051a0 2024-09-21T01:56:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:24 smithi136 ceph-mon[34754]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:24 smithi136 ceph-mon[34754]: from='client.? ' entity='client.admin' 2024-09-21T01:56:25.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:24 smithi099 ceph-mon[34404]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:25.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:24 smithi099 ceph-mon[34404]: from='client.? ' entity='client.admin' 2024-09-21T01:56:25.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.089+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fa69802e640 con 0x7fa6981051a0 2024-09-21T01:56:25.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.093+0000 7fa68effd640 1 -- 172.21.15.92:0/515526708 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v29)=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fa6900b2300 con 0x7fa6981051a0 2024-09-21T01:56:25.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6840b6500 msgr2=0x7fa6840b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6840b6500 0x7fa6840b8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa694000c00 tx=0x7fa694031400 comp rx=0 tx=0).stop 2024-09-21T01:56:25.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 msgr2=0x7fa69819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 0x7fa69819b1b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa69000e940 tx=0x7fa69000ee00 comp rx=0 tx=0).stop 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 shutdown_connections 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6840b6500 0x7fa6840b8a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6981051a0 0x7fa69819b1b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa698104800 0x7fa69819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 --2- 172.21.15.92:0/515526708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698103770 0x7fa698199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 >> 172.21.15.92:0/515526708 conn(0x7fa6980ff0e0 msgr2=0x7fa698107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.095+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 shutdown_connections 2024-09-21T01:56:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.096+0000 7fa69ff97640 1 -- 172.21.15.92:0/515526708 wait complete. 2024-09-21T01:56:25.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 -- 172.21.15.92:0/4060399266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 msgr2=0x7f82c8103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 --2- 172.21.15.92:0/4060399266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c8103be0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc02a480 comp rx=0 tx=0).stop 2024-09-21T01:56:25.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 -- 172.21.15.92:0/4060399266 shutdown_connections 2024-09-21T01:56:25.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 --2- 172.21.15.92:0/4060399266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c81051a0 0x7f82c810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 --2- 172.21.15.92:0/4060399266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c8104800 0x7f82c8104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 --2- 172.21.15.92:0/4060399266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c8103be0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 -- 172.21.15.92:0/4060399266 >> 172.21.15.92:0/4060399266 conn(0x7f82c80ff0c0 msgr2=0x7f82c8101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:25.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 -- 172.21.15.92:0/4060399266 shutdown_connections 2024-09-21T01:56:25.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.623+0000 7f82cef84640 1 -- 172.21.15.92:0/4060399266 wait complete. 2024-09-21T01:56:25.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 Processor -- start 2024-09-21T01:56:25.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 -- start start 2024-09-21T01:56:25.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:25.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c8104800 0x7f82c819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:25.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c81051a0 0x7f82c819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:25.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f460 con 0x7f82c8103770 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f2e0 con 0x7f82c8104800 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82cef84640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f5e0 con 0x7f82c81051a0 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.624+0000 7f82ccd20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52812/0 (socket says 172.21.15.92:52812) 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 -- 172.21.15.92:0/3444531851 learned_addr learned my addr 172.21.15.92:0/3444531851 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82c7fff640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c8104800 0x7f82c819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82cd521640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c81051a0 0x7f82c819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 -- 172.21.15.92:0/3444531851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c81051a0 msgr2=0x7f82c819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c81051a0 0x7f82c819b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 -- 172.21.15.92:0/3444531851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c8104800 msgr2=0x7f82c819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c8104800 0x7f82c819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 -- 172.21.15.92:0/3444531851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c819ba30 con 0x7f82c8103770 2024-09-21T01:56:25.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82c7fff640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c8104800 0x7f82c819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:56:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82ccd20640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c819a000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82bc031440 tx=0x7f82bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc03f070 con 0x7f82c8103770 2024-09-21T01:56:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82bc031c50 con 0x7f82c8103770 2024-09-21T01:56:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c819bcc0 con 0x7f82c8103770 2024-09-21T01:56:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.625+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc035030 con 0x7f82c8103770 2024-09-21T01:56:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.626+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c81a22d0 con 0x7f82c8103770 2024-09-21T01:56:25.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.627+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c810a950 con 0x7f82c8103770 2024-09-21T01:56:25.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.628+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f82bc042070 con 0x7f82c8103770 2024-09-21T01:56:25.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.629+0000 7f82c5ffb640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82b80b6480 0x7f82b80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:25.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.629+0000 7f82c7fff640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82b80b6480 0x7f82b80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:25.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.629+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f82bc1066c0 con 0x7f82c8103770 2024-09-21T01:56:25.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.629+0000 7f82c7fff640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82b80b6480 0x7f82b80b8990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:25.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.632+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82bc0cec20 con 0x7f82c8103770 2024-09-21T01:56:25.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.811+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f82c802e640 con 0x7f82c8103770 2024-09-21T01:56:25.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.816+0000 7f82c5ffb640 1 -- 172.21.15.92:0/3444531851 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v30)=0 v30) ==== 120+0+0 (secure 0 0 0) 0x7f82bc0d3000 con 0x7f82c8103770 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82b80b6480 msgr2=0x7f82b80b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82b80b6480 0x7f82b80b8990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4008040 comp rx=0 tx=0).stop 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 msgr2=0x7f82c819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c819a000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82bc031440 tx=0x7f82bc003c10 comp rx=0 tx=0).stop 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 shutdown_connections 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82b80b6480 0x7f82b80b8990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c81051a0 0x7f82c819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82c8104800 0x7f82c819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 --2- 172.21.15.92:0/3444531851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c8103770 0x7f82c819a000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:25.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 >> 172.21.15.92:0/3444531851 conn(0x7f82c80ff0c0 msgr2=0x7f82c8107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 shutdown_connections 2024-09-21T01:56:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:25.818+0000 7f82cef84640 1 -- 172.21.15.92:0/3444531851 wait complete. 2024-09-21T01:56:26.133 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2500131343' entity='client.admin' 2024-09-21T01:56:26.133 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/515526708' entity='client.admin' 2024-09-21T01:56:26.134 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3444531851' entity='client.admin' 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.161 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi092') 2024-09-21T01:56:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2500131343' entity='client.admin' 2024-09-21T01:56:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/515526708' entity='client.admin' 2024-09-21T01:56:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3444531851' entity='client.admin' 2024-09-21T01:56:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2500131343' entity='client.admin' 2024-09-21T01:56:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/515526708' entity='client.admin' 2024-09-21T01:56:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3444531851' entity='client.admin' 2024-09-21T01:56:27.257 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:27.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:27.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:27.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:27.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:27.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:27.964 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi092') 2024-09-21T01:56:28.123 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:28.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3842552068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4281036d0 msgr2=0x7fb428103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3842552068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4281036d0 0x7fb428103ae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009900 tx=0x7fb41c02a480 comp rx=0 tx=0).stop 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3842552068 shutdown_connections 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3842552068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb428104070 0x7fb42810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3842552068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4281036d0 0x7fb428103ae0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3842552068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3842552068 >> 172.21.15.92:0/3842552068 conn(0x7fb4280fdfb0 msgr2=0x7fb428100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3842552068 shutdown_connections 2024-09-21T01:56:28.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.140+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3842552068 wait complete. 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 Processor -- start 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 -- start start 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4281036d0 0x7fb428199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 0x7fb42819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb42810e2e0 con 0x7fb428104070 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb42810e160 con 0x7fb4281036d0 2024-09-21T01:56:28.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42e9fd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb42810e460 con 0x7fb428102640 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42cf9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 0x7fb42819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb427fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42cf9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 0x7fb42819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52824/0 (socket says 172.21.15.92:52824) 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb427fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:60326/0 (socket says 172.21.15.92:60326) 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.141+0000 7fb42cf9a640 1 -- 172.21.15.92:0/3943376990 learned_addr learned my addr 172.21.15.92:0/3943376990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb4277fe640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4281036d0 0x7fb428199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb427fff640 1 -- 172.21.15.92:0/3943376990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4281036d0 msgr2=0x7fb428199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb427fff640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4281036d0 0x7fb428199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb427fff640 1 -- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 msgr2=0x7fb42819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb427fff640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 0x7fb42819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb427fff640 1 -- 172.21.15.92:0/3943376990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42819a8d0 con 0x7fb428102640 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb42cf9a640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 0x7fb42819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb427fff640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428198ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb41800eb00 tx=0x7fb418009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800bb70 con 0x7fb428102640 2024-09-21T01:56:28.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42819abc0 con 0x7fb428102640 2024-09-21T01:56:28.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41800bd10 con 0x7fb428102640 2024-09-21T01:56:28.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418017410 con 0x7fb428102640 2024-09-21T01:56:28.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.142+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42806b030 con 0x7fb428102640 2024-09-21T01:56:28.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.144+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb428102ab0 con 0x7fb428102640 2024-09-21T01:56:28.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.145+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fb41801e020 con 0x7fb428102640 2024-09-21T01:56:28.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.146+0000 7fb4257fa640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4100b6480 0x7fb4100b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:28.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.146+0000 7fb4277fe640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4100b6480 0x7fb4100b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:28.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.146+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4180e6fe0 con 0x7fb428102640 2024-09-21T01:56:28.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.146+0000 7fb4277fe640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4100b6480 0x7fb4100b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:28.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.150+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4180af520 con 0x7fb428102640 2024-09-21T01:56:28.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.339+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fb42806ba50 con 0x7fb428102640 2024-09-21T01:56:28.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.341+0000 7fb4257fa640 1 -- 172.21.15.92:0/3943376990 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v30)=0 v30) ==== 108+0+0 (secure 0 0 0) 0x7fb4180b3900 con 0x7fb428102640 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4100b6480 msgr2=0x7fb4100b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4100b6480 0x7fb4100b8990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c031400 comp rx=0 tx=0).stop 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 msgr2=0x7fb428198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428198ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb41800eb00 tx=0x7fb418009e40 comp rx=0 tx=0).stop 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 shutdown_connections 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4100b6480 0x7fb4100b8990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb428104070 0x7fb42819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4281036d0 0x7fb428199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 --2- 172.21.15.92:0/3943376990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb428102640 0x7fb428198ea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:28.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 >> 172.21.15.92:0/3943376990 conn(0x7fb4280fdfb0 msgr2=0x7fb4281069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:28.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 shutdown_connections 2024-09-21T01:56:28.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:28.343+0000 7fb42e9fd640 1 -- 172.21.15.92:0/3943376990 wait complete. 2024-09-21T01:56:29.097 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi092') 2024-09-21T01:56:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:28 smithi136 ceph-mon[34754]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:29.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:28 smithi092 ceph-mon[31190]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:28 smithi099 ceph-mon[34404]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:30.135 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:30.136 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:30.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 -- 172.21.15.92:0/1467908161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18105750 msgr2=0x7f3a18107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1467908161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18105750 0x7f3a18107c30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009900 tx=0x7f3a0802a410 comp rx=0 tx=0).stop 2024-09-21T01:56:30.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 -- 172.21.15.92:0/1467908161 shutdown_connections 2024-09-21T01:56:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1467908161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18108370 0x7f3a1810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1467908161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18105750 0x7f3a18107c30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1467908161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a18102cf0 0x7f3a18105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 -- 172.21.15.92:0/1467908161 >> 172.21.15.92:0/1467908161 conn(0x7f3a180fd770 msgr2=0x7f3a180ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 -- 172.21.15.92:0/1467908161 shutdown_connections 2024-09-21T01:56:30.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 -- 172.21.15.92:0/1467908161 wait complete. 2024-09-21T01:56:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.138+0000 7f3a1f480640 1 Processor -- start 2024-09-21T01:56:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 -- start start 2024-09-21T01:56:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 0x7f3a18199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18105750 0x7f3a1819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a18108370 0x7f3a1819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810f400 con 0x7f3a18102cf0 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810f280 con 0x7f3a18105750 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1f480640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810f580 con 0x7f3a18108370 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 0x7f3a18199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 0x7f3a18199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52844/0 (socket says 172.21.15.92:52844) 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 -- 172.21.15.92:0/1104116477 learned_addr learned my addr 172.21.15.92:0/1104116477 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1da1d640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a18108370 0x7f3a1819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1ca1b640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18105750 0x7f3a1819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 -- 172.21.15.92:0/1104116477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a18108370 msgr2=0x7f3a1819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a18108370 0x7f3a1819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 -- 172.21.15.92:0/1104116477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18105750 msgr2=0x7f3a1819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18105750 0x7f3a1819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.139+0000 7f3a1d21c640 1 -- 172.21.15.92:0/1104116477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1819b9d0 con 0x7f3a18102cf0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.140+0000 7f3a1d21c640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 0x7f3a18199fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400d950 tx=0x7f3a1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.140+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14017030 con 0x7f3a18102cf0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.140+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a14004020 con 0x7f3a18102cf0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.140+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1819bcc0 con 0x7f3a18102cf0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.140+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a140047a0 con 0x7f3a18102cf0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.140+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a181a23c0 con 0x7f3a18102cf0 2024-09-21T01:56:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.141+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a180ff290 con 0x7f3a18102cf0 2024-09-21T01:56:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.144+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f3a14009a30 con 0x7f3a18102cf0 2024-09-21T01:56:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.145+0000 7f3a0e7fc640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a040b6480 0x7f3a040b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.145+0000 7f3a1ca1b640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a040b6480 0x7f3a040b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.145+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a140e5670 con 0x7f3a18102cf0 2024-09-21T01:56:30.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.145+0000 7f3a1ca1b640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a040b6480 0x7f3a040b8990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3a08004770 tx=0x7f3a080314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:30.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.147+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a140adbd0 con 0x7f3a18102cf0 2024-09-21T01:56:30.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.337+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f3a1802e640 con 0x7f3a18102cf0 2024-09-21T01:56:30.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.342+0000 7f3a0e7fc640 1 -- 172.21.15.92:0/1104116477 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v31)=0 v31) ==== 119+0+0 (secure 0 0 0) 0x7f3a140b1fb0 con 0x7f3a18102cf0 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a040b6480 msgr2=0x7f3a040b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a040b6480 0x7f3a040b8990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3a08004770 tx=0x7f3a080314a0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 msgr2=0x7f3a18199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 0x7f3a18199fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400d950 tx=0x7f3a1400de10 comp rx=0 tx=0).stop 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 shutdown_connections 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a040b6480 0x7f3a040b8990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a18108370 0x7f3a1819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a18105750 0x7f3a1819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 --2- 172.21.15.92:0/1104116477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a18102cf0 0x7f3a18199fa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 >> 172.21.15.92:0/1104116477 conn(0x7f3a180fd770 msgr2=0x7f3a18103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 shutdown_connections 2024-09-21T01:56:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.344+0000 7f3a1f480640 1 -- 172.21.15.92:0/1104116477 wait complete. 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.801+0000 7f20a6e9d640 1 -- 172.21.15.92:0/975290973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a01036e0 msgr2=0x7f20a0103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.801+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/975290973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a01036e0 0x7f20a0103af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2090009900 tx=0x7f209002a480 comp rx=0 tx=0).stop 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 -- 172.21.15.92:0/975290973 shutdown_connections 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/975290973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a0104080 0x7f20a010aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/975290973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a01036e0 0x7f20a0103af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/975290973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a0102650 0x7f20a0102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 -- 172.21.15.92:0/975290973 >> 172.21.15.92:0/975290973 conn(0x7f20a00fdfe0 msgr2=0x7f20a0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 -- 172.21.15.92:0/975290973 shutdown_connections 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 -- 172.21.15.92:0/975290973 wait complete. 2024-09-21T01:56:30.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 Processor -- start 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 -- start start 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.802+0000 7f20a6e9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a0102650 0x7f20a019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a6e9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a01036e0 0x7f20a019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a6e9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 0x7f20a019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a6e9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010e280 con 0x7f20a0104080 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a6e9d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010e100 con 0x7f20a0102650 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a6e9d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010e400 con 0x7f20a01036e0 2024-09-21T01:56:30.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 0x7f20a019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 0x7f20a019b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52858/0 (socket says 172.21.15.92:52858) 2024-09-21T01:56:30.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 -- 172.21.15.92:0/2859415779 learned_addr learned my addr 172.21.15.92:0/2859415779 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f2097fff640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a01036e0 0x7f20a019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a4c39640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a0102650 0x7f20a019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 -- 172.21.15.92:0/2859415779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a01036e0 msgr2=0x7f20a019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a01036e0 0x7f20a019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 -- 172.21.15.92:0/2859415779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a0102650 msgr2=0x7f20a019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a0102650 0x7f20a019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 -- 172.21.15.92:0/2859415779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a019ba40 con 0x7f20a0104080 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a543a640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 0x7f20a019b220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2098009fd0 tx=0x7f209800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098017070 con 0x7f20a0104080 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2098003e20 con 0x7f20a0104080 2024-09-21T01:56:30.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098010460 con 0x7f20a0104080 2024-09-21T01:56:30.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.803+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a019bd30 con 0x7f20a0104080 2024-09-21T01:56:30.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.804+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a01a2430 con 0x7f20a0104080 2024-09-21T01:56:30.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.805+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a0102ac0 con 0x7f20a0104080 2024-09-21T01:56:30.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.806+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f2098003970 con 0x7f20a0104080 2024-09-21T01:56:30.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.807+0000 7f2095ffb640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20880b6500 0x7f20880b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.807+0000 7f20a4c39640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20880b6500 0x7f20880b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.807+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f20980e5250 con 0x7f20a0104080 2024-09-21T01:56:30.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.807+0000 7f20a4c39640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20880b6500 0x7f20880b8a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f209c009bd0 tx=0x7f209c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:30.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.811+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f20980b1b90 con 0x7f20a0104080 2024-09-21T01:56:30.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.980+0000 7f367397c640 1 -- 172.21.15.92:0/1887404697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c1036d0 msgr2=0x7f366c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.980+0000 7f367397c640 1 --2- 172.21.15.92:0/1887404697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c1036d0 0x7f366c103ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3668009900 tx=0x7f366802a480 comp rx=0 tx=0).stop 2024-09-21T01:56:30.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 -- 172.21.15.92:0/1887404697 shutdown_connections 2024-09-21T01:56:30.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 --2- 172.21.15.92:0/1887404697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c104070 0x7f366c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 --2- 172.21.15.92:0/1887404697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c1036d0 0x7f366c103ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 --2- 172.21.15.92:0/1887404697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c102640 0x7f366c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 -- 172.21.15.92:0/1887404697 >> 172.21.15.92:0/1887404697 conn(0x7f366c0fdfb0 msgr2=0x7f366c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 -- 172.21.15.92:0/1887404697 shutdown_connections 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 -- 172.21.15.92:0/1887404697 wait complete. 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.981+0000 7f367397c640 1 Processor -- start 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 -- start start 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c102640 0x7f366c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1036d0 0x7f366c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 0x7f366c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f366c10e270 con 0x7f366c104070 2024-09-21T01:56:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f366c10e0f0 con 0x7f366c102640 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f367397c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f366c10e3f0 con 0x7f366c1036d0 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 0x7f366c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 0x7f366c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52880/0 (socket says 172.21.15.92:52880) 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 -- 172.21.15.92:0/3458800640 learned_addr learned my addr 172.21.15.92:0/3458800640 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3670f17640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1036d0 0x7f366c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671718640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c102640 0x7f366c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 -- 172.21.15.92:0/3458800640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1036d0 msgr2=0x7f366c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1036d0 0x7f366c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 -- 172.21.15.92:0/3458800640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c102640 msgr2=0x7f366c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c102640 0x7f366c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 -- 172.21.15.92:0/3458800640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c19a8d0 con 0x7f366c104070 2024-09-21T01:56:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3670f17640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1036d0 0x7f366c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.982+0000 7f3671f19640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 0x7f366c19a0b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f366400eaa0 tx=0x7f366400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.983+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366400cdb0 con 0x7f366c104070 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.983+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3664005020 con 0x7f366c104070 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.983+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c19abc0 con 0x7f366c104070 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.983+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36640116d0 con 0x7f366c104070 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.983+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a12c0 con 0x7f366c104070 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.984+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c109820 con 0x7f366c104070 2024-09-21T01:56:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.986+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f36640051d0 con 0x7f366c104070 2024-09-21T01:56:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.987+0000 7f36627fc640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f36580b6500 0x7f36580b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.987+0000 7f3671718640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f36580b6500 0x7f36580b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.987+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f36640e5c40 con 0x7f366c104070 2024-09-21T01:56:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.987+0000 7f3671718640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f36580b6500 0x7f36580b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f365c005fd0 tx=0x7f365c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:30.989+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3664016030 con 0x7f366c104070 2024-09-21T01:56:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.000+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f20a01a2600 con 0x7f20a0104080 2024-09-21T01:56:31.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.004+0000 7f2095ffb640 1 -- 172.21.15.92:0/2859415779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v32)=0 v32) ==== 115+0+0 (secure 0 0 0) 0x7f20980b6ad0 con 0x7f20a0104080 2024-09-21T01:56:31.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20880b6500 msgr2=0x7f20880b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:31.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20880b6500 0x7f20880b8a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f209c009bd0 tx=0x7f209c005f90 comp rx=0 tx=0).stop 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 msgr2=0x7f20a019b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 0x7f20a019b220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2098009fd0 tx=0x7f209800b260 comp rx=0 tx=0).stop 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 shutdown_connections 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20880b6500 0x7f20880b8a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a0104080 0x7f20a019b220 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a01036e0 0x7f20a019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 --2- 172.21.15.92:0/2859415779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a0102650 0x7f20a019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 >> 172.21.15.92:0/2859415779 conn(0x7f20a00fdfe0 msgr2=0x7f20a00ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 shutdown_connections 2024-09-21T01:56:31.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.007+0000 7f20a6e9d640 1 -- 172.21.15.92:0/2859415779 wait complete. 2024-09-21T01:56:31.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:30 smithi092 ceph-mon[31190]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:31.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:30 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1104116477' entity='client.admin' 2024-09-21T01:56:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.181+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f366c02e640 con 0x7f366c104070 2024-09-21T01:56:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.181+0000 7f36627fc640 1 -- 172.21.15.92:0/3458800640 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v32)=0 v32) ==== 106+0+0 (secure 0 0 0) 0x7f36640ae220 con 0x7f366c104070 2024-09-21T01:56:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.183+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f36580b6500 msgr2=0x7f36580b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.183+0000 7f367397c640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f36580b6500 0x7f36580b8a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f365c005fd0 tx=0x7f365c005850 comp rx=0 tx=0).stop 2024-09-21T01:56:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.183+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 msgr2=0x7f366c19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.183+0000 7f367397c640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 0x7f366c19a0b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f366400eaa0 tx=0x7f366400ef60 comp rx=0 tx=0).stop 2024-09-21T01:56:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 shutdown_connections 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f36580b6500 0x7f36580b8a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c104070 0x7f366c19a0b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1036d0 0x7f366c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 --2- 172.21.15.92:0/3458800640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c102640 0x7f366c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 >> 172.21.15.92:0/3458800640 conn(0x7f366c0fdfb0 msgr2=0x7f366c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 shutdown_connections 2024-09-21T01:56:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:31.184+0000 7f367397c640 1 -- 172.21.15.92:0/3458800640 wait complete. 2024-09-21T01:56:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:30 smithi136 ceph-mon[34754]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:30 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1104116477' entity='client.admin' 2024-09-21T01:56:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:30 smithi099 ceph-mon[34404]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:30 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1104116477' entity='client.admin' 2024-09-21T01:56:32.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:32 smithi092 ceph-mon[31190]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:32.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:32 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2859415779' entity='client.admin' 2024-09-21T01:56:32.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:32 smithi136 ceph-mon[34754]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:32.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:32 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2859415779' entity='client.admin' 2024-09-21T01:56:32.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:32 smithi099 ceph-mon[34404]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:32.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:32 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2859415779' entity='client.admin' 2024-09-21T01:56:32.721 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.721 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.721 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.722 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi092') 2024-09-21T01:56:32.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 -- 172.21.15.92:0/2564471272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030105180 msgr2=0x7f703010bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:32.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 --2- 172.21.15.92:0/2564471272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030105180 0x7f703010bb20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7028009ca0 tx=0x7f702802a950 comp rx=0 tx=0).stop 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 -- 172.21.15.92:0/2564471272 shutdown_connections 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 --2- 172.21.15.92:0/2564471272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030105180 0x7f703010bb20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 --2- 172.21.15.92:0/2564471272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70301047e0 0x7f7030104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 --2- 172.21.15.92:0/2564471272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7030103750 0x7f7030103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 -- 172.21.15.92:0/2564471272 >> 172.21.15.92:0/2564471272 conn(0x7f70300ff0c0 msgr2=0x7f7030101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 -- 172.21.15.92:0/2564471272 shutdown_connections 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.954+0000 7f703520b640 1 -- 172.21.15.92:0/2564471272 wait complete. 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.955+0000 7f703520b640 1 Processor -- start 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 -- start start 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 0x7f7030076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70301047e0 0x7f7030075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7030105180 0x7f7030075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:32.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f703010f3a0 con 0x7f7030103750 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f703010f220 con 0x7f70301047e0 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f703520b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f703010f520 con 0x7f7030105180 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 0x7f7030076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 0x7f7030076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52892/0 (socket says 172.21.15.92:52892) 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 -- 172.21.15.92:0/2589510441 learned_addr learned my addr 172.21.15.92:0/2589510441 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702f59e640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7030105180 0x7f7030075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702e59c640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70301047e0 0x7f7030075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 -- 172.21.15.92:0/2589510441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7030105180 msgr2=0x7f7030075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7030105180 0x7f7030075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 -- 172.21.15.92:0/2589510441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70301047e0 msgr2=0x7f7030075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70301047e0 0x7f7030075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 -- 172.21.15.92:0/2589510441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7030076350 con 0x7f7030103750 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702f59e640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7030105180 0x7f7030075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.956+0000 7f702ed9d640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 0x7f7030076e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f702000d950 tx=0x7f702000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:32.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.957+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020017070 con 0x7f7030103750 2024-09-21T01:56:32.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.957+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7020009c30 con 0x7f7030103750 2024-09-21T01:56:32.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.957+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70301a4020 con 0x7f7030103750 2024-09-21T01:56:32.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.957+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702000f620 con 0x7f7030103750 2024-09-21T01:56:32.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.957+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301a4480 con 0x7f7030103750 2024-09-21T01:56:32.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.958+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703010a910 con 0x7f7030103750 2024-09-21T01:56:32.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.959+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f7020010460 con 0x7f7030103750 2024-09-21T01:56:32.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.960+0000 7f7017fff640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70180b6480 0x7f70180b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:32.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.960+0000 7f702e59c640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70180b6480 0x7f70180b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:32.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.960+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f70200e5260 con 0x7f7030103750 2024-09-21T01:56:32.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.961+0000 7f702e59c640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70180b6480 0x7f70180b8990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7024005fd0 tx=0x7f702400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:32.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:32.963+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70200ad7c0 con 0x7f7030103750 2024-09-21T01:56:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.141+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f703002e640 con 0x7f7030103750 2024-09-21T01:56:33.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.146+0000 7f7017fff640 1 -- 172.21.15.92:0/2589510441 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v33)=0 v33) ==== 125+0+0 (secure 0 0 0) 0x7f7020009de0 con 0x7f7030103750 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70180b6480 msgr2=0x7f70180b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70180b6480 0x7f70180b8990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7024005fd0 tx=0x7f702400b390 comp rx=0 tx=0).stop 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 msgr2=0x7f7030076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 0x7f7030076e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f702000d950 tx=0x7f702000de10 comp rx=0 tx=0).stop 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 shutdown_connections 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70180b6480 0x7f70180b8990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7030105180 0x7f7030075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70301047e0 0x7f7030075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 --2- 172.21.15.92:0/2589510441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7030103750 0x7f7030076e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.148+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 >> 172.21.15.92:0/2589510441 conn(0x7f70300ff0c0 msgr2=0x7f7030107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.149+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 shutdown_connections 2024-09-21T01:56:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:33.149+0000 7f703520b640 1 -- 172.21.15.92:0/2589510441 wait complete. 2024-09-21T01:56:33.885 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi092') 2024-09-21T01:56:33.886 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '49513f9c-77bc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi092') 2024-09-21T01:56:33.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T01:56:34.123 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:34 smithi136 ceph-mon[34754]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:34 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2589510441' entity='client.admin' 2024-09-21T01:56:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:34 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:34 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:34 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:34.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:34 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:34.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:34 smithi092 ceph-mon[31190]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:34.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:34 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2589510441' entity='client.admin' 2024-09-21T01:56:34.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:34 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:34.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:34 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:34.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:34 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:34.463 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:34 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:34.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 -- 172.21.15.92:0/3879260676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2401036d0 msgr2=0x7fa240103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 --2- 172.21.15.92:0/3879260676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2401036d0 0x7fa240103ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa228009900 tx=0x7fa22802a480 comp rx=0 tx=0).stop 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 -- 172.21.15.92:0/3879260676 shutdown_connections 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 --2- 172.21.15.92:0/3879260676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa240104070 0x7fa24010aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 --2- 172.21.15.92:0/3879260676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2401036d0 0x7fa240103ae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 --2- 172.21.15.92:0/3879260676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa240102640 0x7fa240102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 -- 172.21.15.92:0/3879260676 >> 172.21.15.92:0/3879260676 conn(0x7fa2400fdfb0 msgr2=0x7fa240100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 -- 172.21.15.92:0/3879260676 shutdown_connections 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.549+0000 7fa246f54640 1 -- 172.21.15.92:0/3879260676 wait complete. 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 Processor -- start 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 -- start start 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 0x7fa240198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2401036d0 0x7fa240199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa240104070 0x7fa24019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:34.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e270 con 0x7fa240102640 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e0f0 con 0x7fa240104070 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa246f54640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010e3f0 con 0x7fa2401036d0 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 0x7fa240198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 0x7fa240198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55842/0 (socket says 172.21.15.92:55842) 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 -- 172.21.15.92:0/4177029884 learned_addr learned my addr 172.21.15.92:0/4177029884 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa23ffff640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2401036d0 0x7fa240199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa2454f1640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa240104070 0x7fa24019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 -- 172.21.15.92:0/4177029884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2401036d0 msgr2=0x7fa240199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2401036d0 0x7fa240199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 -- 172.21.15.92:0/4177029884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa240104070 msgr2=0x7fa24019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa240104070 0x7fa24019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa244cf0640 1 -- 172.21.15.92:0/4177029884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24019a8d0 con 0x7fa240102640 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.550+0000 7fa23ffff640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2401036d0 0x7fa240199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:34.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.551+0000 7fa244cf0640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 0x7fa240198ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d950 tx=0x7fa23000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.551+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230017070 con 0x7fa240102640 2024-09-21T01:56:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.551+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa230009c30 con 0x7fa240102640 2024-09-21T01:56:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.551+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24019abc0 con 0x7fa240102640 2024-09-21T01:56:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.551+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23000f620 con 0x7fa240102640 2024-09-21T01:56:34.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.551+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2401a12c0 con 0x7fa240102640 2024-09-21T01:56:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.552+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa240109820 con 0x7fa240102640 2024-09-21T01:56:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.553+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fa230010460 con 0x7fa240102640 2024-09-21T01:56:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.554+0000 7fa23dffb640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa22c0b6480 0x7fa22c0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.554+0000 7fa23ffff640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa22c0b6480 0x7fa22c0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.554+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa2300e5320 con 0x7fa240102640 2024-09-21T01:56:34.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.555+0000 7fa23ffff640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa22c0b6480 0x7fa22c0b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa228031440 tx=0x7fa228005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:34.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.558+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2300ad880 con 0x7fa240102640 2024-09-21T01:56:34.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:34 smithi099 ceph-mon[34404]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:34.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:34 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2589510441' entity='client.admin' 2024-09-21T01:56:34.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:34 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:34.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:34 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:34.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:34 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:56:34.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:34 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:34.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.736+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa24002e640 con 0x7fa240102640 2024-09-21T01:56:34.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.737+0000 7fa23dffb640 1 -- 172.21.15.92:0/4177029884 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+21518 (secure 0 0 0) 0x7fa230009de0 con 0x7fa240102640 2024-09-21T01:56:34.738 INFO:teuthology.orchestra.run.smithi092.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T01:56:34.738 INFO:teuthology.orchestra.run.smithi092.stdout:global dev auth_debug true 2024-09-21T01:56:34.738 INFO:teuthology.orchestra.run.smithi092.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global basic log_to_file true 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global basic log_to_journald false 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global basic log_to_stderr false 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_allow_pool_delete true 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T01:56:34.739 INFO:teuthology.orchestra.run.smithi092.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:global dev ms_die_on_bug true 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:global dev ms_die_on_old_message true 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:global dev ms_die_on_skipped_message false 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced debug_mon 20/20 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced debug_ms 1/1 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced debug_paxos 20/20 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T01:56:34.740 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced client_mount_timeout 30 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced debug_client 20/20 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced debug_mgr 20/20 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced debug_ms 1/1 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T01:56:34.741 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev bdev_debug_aio true 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced bdev_enable_discard true 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced bluestore_allocator bitmap * 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced debug_bluefs 1/20 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced debug_bluestore 1/20 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced debug_ms 1/1 2024-09-21T01:56:34.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced debug_osd 20/20 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced debug_rocksdb 4/10 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_op_order true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_shutdown true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T01:56:34.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced debug_mds 20/20 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced debug_ms 2/2 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced mds_bal_split_size 100 2024-09-21T01:56:34.744 INFO:teuthology.orchestra.run.smithi092.stdout:mds dev mds_debug_frag true 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds dev mds_debug_scatterstat true 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds dev mds_verify_scatter true 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client advanced client_mount_timeout 600 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client advanced debug_client 20/20 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client advanced debug_ms 1/1 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T01:56:34.745 INFO:teuthology.orchestra.run.smithi092.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.739+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa22c0b6480 msgr2=0x7fa22c0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.739+0000 7fa246f54640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa22c0b6480 0x7fa22c0b8990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa228031440 tx=0x7fa228005d50 comp rx=0 tx=0).stop 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.739+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 msgr2=0x7fa240198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.739+0000 7fa246f54640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 0x7fa240198ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d950 tx=0x7fa23000de10 comp rx=0 tx=0).stop 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 shutdown_connections 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa22c0b6480 0x7fa22c0b8990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa240104070 0x7fa24019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2401036d0 0x7fa240199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 --2- 172.21.15.92:0/4177029884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa240102640 0x7fa240198ea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:34.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 >> 172.21.15.92:0/4177029884 conn(0x7fa2400fdfb0 msgr2=0x7fa2401069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:34.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 shutdown_connections 2024-09-21T01:56:34.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:34.740+0000 7fa246f54640 1 -- 172.21.15.92:0/4177029884 wait complete. 2024-09-21T01:56:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:35 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4177029884' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:56:35.342 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T01:56:35.342 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T01:56:35.342 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:56:35.358 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:56:35.358 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-21 01:55:25.178711483 +0000 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-21 01:50:38.886324559 +0000 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-21 01:50:38.886324559 +0000 2024-09-21T01:56:35.414 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-21 01:50:38.886324559 +0000 2024-09-21T01:56:35.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:56:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-21T01:56:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-21T01:56:35.479 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000217445 s, 2.4 MB/s 2024-09-21T01:56:35.480 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:56:35.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:35 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4177029884' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:56:35.536 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-21T01:56:35.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:35 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4177029884' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-21 01:55:25.206710658 +0000 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-21 01:50:39.202315054 +0000 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-21 01:50:39.202315054 +0000 2024-09-21T01:56:35.592 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-21 01:50:39.202315054 +0000 2024-09-21T01:56:35.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:56:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-21T01:56:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-21T01:56:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000177915 s, 2.9 MB/s 2024-09-21T01:56:35.657 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:56:35.713 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-21 01:55:25.234709832 +0000 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-21 01:50:39.547304678 +0000 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-21 01:50:39.547304678 +0000 2024-09-21T01:56:35.776 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-21 01:50:39.547304678 +0000 2024-09-21T01:56:35.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:56:35.841 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-21T01:56:35.842 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-21T01:56:35.842 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000205232 s, 2.5 MB/s 2024-09-21T01:56:35.843 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:56:35.901 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-21T01:56:35.956 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:56:35.956 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:35.956 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T01:56:35.956 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:35.957 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:35.957 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-21 01:55:25.263708978 +0000 2024-09-21T01:56:35.957 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-21 01:50:39.886294482 +0000 2024-09-21T01:56:35.957 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-21 01:50:39.886294482 +0000 2024-09-21T01:56:35.957 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-21 01:50:39.886294482 +0000 2024-09-21T01:56:35.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:56:36.020 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-21T01:56:36.020 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-21T01:56:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000213765 s, 2.4 MB/s 2024-09-21T01:56:36.022 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:56:36.078 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-21T01:56:36.078 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:56:36.098 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:56:36.099 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-21T01:56:36.156 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 01:56:12.075534941 +0000 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 01:50:55.938568483 +0000 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 01:50:55.938568483 +0000 2024-09-21T01:56:36.157 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-21 01:50:55.938568483 +0000 2024-09-21T01:56:36.157 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:56:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T01:56:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T01:56:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000353564 s, 1.4 MB/s 2024-09-21T01:56:36.227 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:56:36.285 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-21T01:56:36.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:36 smithi092 ceph-mon[31190]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 01:56:12.075534941 +0000 2024-09-21T01:56:36.342 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 01:50:56.244561262 +0000 2024-09-21T01:56:36.343 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 01:50:56.244561262 +0000 2024-09-21T01:56:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-21 01:50:56.244561262 +0000 2024-09-21T01:56:36.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:56:36.405 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:36 smithi099 ceph-mon[34404]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:36.408 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T01:56:36.408 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T01:56:36.408 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000199718 s, 2.6 MB/s 2024-09-21T01:56:36.409 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:56:36.466 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-21T01:56:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:56:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:36.522 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T01:56:36.522 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:36.523 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:36.523 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 01:56:12.076534919 +0000 2024-09-21T01:56:36.523 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 01:50:56.561553781 +0000 2024-09-21T01:56:36.523 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 01:50:56.561553781 +0000 2024-09-21T01:56:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-21 01:50:56.561553781 +0000 2024-09-21T01:56:36.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:56:36.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:36 smithi136 ceph-mon[34754]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:36.586 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T01:56:36.586 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T01:56:36.586 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000230463 s, 2.2 MB/s 2024-09-21T01:56:36.587 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:56:36.644 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-21T01:56:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:56:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:36.700 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T01:56:36.700 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:36.701 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:36.701 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-21 01:56:12.076534919 +0000 2024-09-21T01:56:36.701 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-21 01:50:56.884546159 +0000 2024-09-21T01:56:36.701 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-21 01:50:56.884546159 +0000 2024-09-21T01:56:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-21 01:50:56.884546159 +0000 2024-09-21T01:56:36.701 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:56:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-21T01:56:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-21T01:56:36.764 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000194002 s, 2.6 MB/s 2024-09-21T01:56:36.765 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:56:36.822 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-21T01:56:36.822 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:56:36.841 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:56:36.842 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-21 01:56:09.675650947 +0000 2024-09-21T01:56:36.900 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-21 01:50:55.855576695 +0000 2024-09-21T01:56:36.901 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-21 01:50:55.855576695 +0000 2024-09-21T01:56:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-21 01:50:55.855576695 +0000 2024-09-21T01:56:36.901 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:56:36.970 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-21T01:56:36.970 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-21T01:56:36.970 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000242353 s, 2.1 MB/s 2024-09-21T01:56:36.971 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:56:37.029 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-21 01:56:09.675650947 +0000 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-21 01:50:56.181569188 +0000 2024-09-21T01:56:37.086 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-21 01:50:56.181569188 +0000 2024-09-21T01:56:37.087 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-21 01:50:56.181569188 +0000 2024-09-21T01:56:37.087 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:56:37.151 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-21T01:56:37.151 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-21T01:56:37.151 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000213996 s, 2.4 MB/s 2024-09-21T01:56:37.153 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:56:37.209 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-09-21T01:56:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:56:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:37.265 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T01:56:37.266 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:37.266 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:37.266 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-21 01:56:09.675650947 +0000 2024-09-21T01:56:37.266 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-21 01:50:56.492562027 +0000 2024-09-21T01:56:37.266 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-21 01:50:56.492562027 +0000 2024-09-21T01:56:37.266 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-21 01:50:56.492562027 +0000 2024-09-21T01:56:37.266 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:56:37.329 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-21T01:56:37.329 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-21T01:56:37.329 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000205404 s, 2.5 MB/s 2024-09-21T01:56:37.330 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:56:37.386 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-21 01:56:09.676650924 +0000 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-21 01:50:56.828554290 +0000 2024-09-21T01:56:37.441 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-21 01:50:56.828554290 +0000 2024-09-21T01:56:37.442 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-09-21 01:50:56.828554290 +0000 2024-09-21T01:56:37.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:56:37.504 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-21T01:56:37.505 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-21T01:56:37.505 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000243564 s, 2.1 MB/s 2024-09-21T01:56:37.506 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:56:37.563 INFO:tasks.cephadm:Deploying osd.0 on smithi092 with /dev/vg_nvme/lv_4... 2024-09-21T01:56:37.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:56:37.804 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:38.244 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:38 smithi092 ceph-mon[31190]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:38.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:38 smithi136 ceph-mon[34754]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:38.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:38 smithi099 ceph-mon[34404]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:40.212 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:40 smithi092 ceph-mon[31190]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:56:40.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-09-21T01:56:40.482 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:56:40.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:40 smithi136 ceph-mon[34754]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:40 smithi099 ceph-mon[34404]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 -- 172.21.15.92:0/2229488342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 msgr2=0x7f7e1c103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 --2- 172.21.15.92:0/2229488342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c103b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009900 tx=0x7f7e0402a480 comp rx=0 tx=0).stop 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 -- 172.21.15.92:0/2229488342 shutdown_connections 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 --2- 172.21.15.92:0/2229488342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e1c104090 0x7f7e1c10aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 --2- 172.21.15.92:0/2229488342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c103b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 --2- 172.21.15.92:0/2229488342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 0x7f7e1c102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 -- 172.21.15.92:0/2229488342 >> 172.21.15.92:0/2229488342 conn(0x7f7e1c0fdfd0 msgr2=0x7f7e1c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 -- 172.21.15.92:0/2229488342 shutdown_connections 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.940+0000 7f7e20c36640 1 -- 172.21.15.92:0/2229488342 wait complete. 2024-09-21T01:56:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.941+0000 7f7e20c36640 1 Processor -- start 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.941+0000 7f7e20c36640 1 -- start start 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e20c36640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 0x7f7e1c106250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e20c36640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e20c36640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e1c104090 0x7f7e1c105060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e20c36640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c10e350 con 0x7f7e1c1036f0 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e20c36640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c10e1d0 con 0x7f7e1c104090 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e20c36640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c10e4d0 con 0x7f7e1c102660 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55864/0 (socket says 172.21.15.92:55864) 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 -- 172.21.15.92:0/261549109 learned_addr learned my addr 172.21.15.92:0/261549109 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e1ad9d640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e1c104090 0x7f7e1c105060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e1a59c640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 0x7f7e1c106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 -- 172.21.15.92:0/261549109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 msgr2=0x7f7e1c106250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 0x7f7e1c106250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 -- 172.21.15.92:0/261549109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e1c104090 msgr2=0x7f7e1c105060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e1c104090 0x7f7e1c105060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 -- 172.21.15.92:0/261549109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c105760 con 0x7f7e1c1036f0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e1a59c640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 0x7f7e1c106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.942+0000 7f7e13fff640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c104b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e0402aa70 tx=0x7f7e04003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.943+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0403f070 con 0x7f7e1c1036f0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.943+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0403b460 con 0x7f7e1c1036f0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.943+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c1a4180 con 0x7f7e1c1036f0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.943+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04035030 con 0x7f7e1c1036f0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.943+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c1a45a0 con 0x7f7e1c1036f0 2024-09-21T01:56:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.944+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1c1a4800 con 0x7f7e1c1036f0 2024-09-21T01:56:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.946+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f7e04042070 con 0x7f7e1c1036f0 2024-09-21T01:56:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.947+0000 7f7e137fe640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e080b6480 0x7f7e080b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:56:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.947+0000 7f7e1a59c640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e080b6480 0x7f7e080b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:56:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.947+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e041063f0 con 0x7f7e1c1036f0 2024-09-21T01:56:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.947+0000 7f7e1a59c640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e080b6480 0x7f7e080b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009930 tx=0x7f7e0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:56:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:40.951+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7e040ce950 con 0x7f7e1c1036f0 2024-09-21T01:56:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:56:41.139+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7e1c100020 con 0x7f7e080b6480 2024-09-21T01:56:41.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:56:41.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:56:41.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:41.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:56:41.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:56:41.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:41.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:56:41.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:56:41.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:42.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:42 smithi092 ceph-mon[31190]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:42.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:42 smithi092 ceph-mon[31190]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:56:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:42 smithi136 ceph-mon[34754]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:42 smithi136 ceph-mon[34754]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:56:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:42 smithi099 ceph-mon[34404]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:42 smithi099 ceph-mon[34404]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:56:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:44 smithi136 ceph-mon[34754]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1748856597' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e256750d-9725-4215-90dc-41a4dc35dfb9"} : dispatch 2024-09-21T01:56:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1748856597' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e256750d-9725-4215-90dc-41a4dc35dfb9"}]': finished 2024-09-21T01:56:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:44 smithi136 ceph-mon[34754]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T01:56:44.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:44 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:56:44.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1155195651' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:56:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:44 smithi092 ceph-mon[31190]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1748856597' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e256750d-9725-4215-90dc-41a4dc35dfb9"} : dispatch 2024-09-21T01:56:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1748856597' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e256750d-9725-4215-90dc-41a4dc35dfb9"}]': finished 2024-09-21T01:56:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:44 smithi092 ceph-mon[31190]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T01:56:44.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:44 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:56:44.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1155195651' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:56:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:44 smithi099 ceph-mon[34404]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1748856597' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e256750d-9725-4215-90dc-41a4dc35dfb9"} : dispatch 2024-09-21T01:56:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1748856597' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e256750d-9725-4215-90dc-41a4dc35dfb9"}]': finished 2024-09-21T01:56:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:44 smithi099 ceph-mon[34404]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T01:56:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:44 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:56:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1155195651' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:56:46.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:46 smithi136 ceph-mon[34754]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:46.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:46 smithi092 ceph-mon[31190]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:46.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:46 smithi099 ceph-mon[34404]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:48 smithi136 ceph-mon[34754]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:48.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:48 smithi092 ceph-mon[31190]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:48.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:48 smithi099 ceph-mon[34404]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:50.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:50 smithi092 ceph-mon[31190]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:50.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:50 smithi136 ceph-mon[34754]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:50.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:50 smithi099 ceph-mon[34404]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:52.484 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:52 smithi092 ceph-mon[31190]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:52.485 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:56:52.485 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:52.485 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:52 smithi092 ceph-mon[31190]: Deploying daemon osd.0 on smithi092 2024-09-21T01:56:52.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:52 smithi136 ceph-mon[34754]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:52.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:56:52.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:52.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:52 smithi136 ceph-mon[34754]: Deploying daemon osd.0 on smithi092 2024-09-21T01:56:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:52 smithi099 ceph-mon[34404]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:56:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:56:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:52 smithi099 ceph-mon[34404]: Deploying daemon osd.0 on smithi092 2024-09-21T01:56:54.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:54 smithi136 ceph-mon[34754]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:54 smithi092 ceph-mon[31190]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:54.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:54 smithi099 ceph-mon[34404]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:56.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:56 smithi136 ceph-mon[34754]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:56.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:56 smithi092 ceph-mon[31190]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:56.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:56 smithi099 ceph-mon[34404]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:58.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:58 smithi136 ceph-mon[34754]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:58.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:58 smithi092 ceph-mon[31190]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:58.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:58 smithi099 ceph-mon[34404]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:56:59.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:59.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:59.442 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:56:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:59.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:59.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:59.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:56:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:59.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:56:59.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:56:59.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:56:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:00.711 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:00 smithi092 ceph-mon[31190]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:00 smithi136 ceph-mon[34754]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:00 smithi099 ceph-mon[34404]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:02.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.006+0000 7f7e137fe640 1 -- 172.21.15.92:0/261549109 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7e1c100020 con 0x7f7e080b6480 2024-09-21T01:57:02.008 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 0 on host 'smithi092' 2024-09-21T01:57:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e080b6480 msgr2=0x7f7e080b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e080b6480 0x7f7e080b8990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009930 tx=0x7f7e0c008040 comp rx=0 tx=0).stop 2024-09-21T01:57:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 msgr2=0x7f7e1c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c104b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e0402aa70 tx=0x7f7e04003c80 comp rx=0 tx=0).stop 2024-09-21T01:57:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 shutdown_connections 2024-09-21T01:57:02.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e080b6480 0x7f7e080b8990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e1c104090 0x7f7e1c105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c1036f0 0x7f7e1c104b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 --2- 172.21.15.92:0/261549109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c102660 0x7f7e1c106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 >> 172.21.15.92:0/261549109 conn(0x7f7e1c0fdfd0 msgr2=0x7f7e1c0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 shutdown_connections 2024-09-21T01:57:02.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:02.009+0000 7f7e20c36640 1 -- 172.21.15.92:0/261549109 wait complete. 2024-09-21T01:57:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:02.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:02.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:02.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:02.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:02.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:02.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:02.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:02.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:02.746 DEBUG:teuthology.orchestra.run.smithi092:osd.0> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T01:57:02.748 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2024-09-21T01:57:02.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:57:03.000 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:57:04.046 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:57:03 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0[46912]: 2024-09-21T01:57:03.923+0000 7fda25026640 -1 osd.0 0 log_to_monitors true 2024-09-21T01:57:04.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:04 smithi092 ceph-mon[31190]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:04.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:04 smithi092 ceph-mon[31190]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:57:04.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:04 smithi099 ceph-mon[34404]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:04.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:04 smithi099 ceph-mon[34404]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:57:04.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:04 smithi136 ceph-mon[34754]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:04.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:04 smithi136 ceph-mon[34754]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:57:05.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:05 smithi092 ceph-mon[31190]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:57:05.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:05 smithi092 ceph-mon[31190]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:57:05.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:05 smithi092 ceph-mon[31190]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:57:05.392 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:05 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:05.392 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:57:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0[46912]: 2024-09-21T01:57:05.121+0000 7fda20fa8640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T01:57:05.392 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:57:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0[46912]: 2024-09-21T01:57:05.124+0000 7fda1cda6640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:57:05.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:05 smithi099 ceph-mon[34404]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:57:05.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:05 smithi099 ceph-mon[34404]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:57:05.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:05 smithi099 ceph-mon[34404]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:57:05.403 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:05 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:05.405 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:57:05.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-09-21T01:57:05.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:05 smithi136 ceph-mon[34754]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:57:05.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:05 smithi136 ceph-mon[34754]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:57:05.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:05 smithi136 ceph-mon[34754]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:57:05.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:05 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:05.681 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:57:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2801827195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6241051b0 msgr2=0x7fa62410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2801827195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6241051b0 0x7fa62410bb50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009ca0 tx=0x7fa61c02a950 comp rx=0 tx=0).stop 2024-09-21T01:57:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2801827195 shutdown_connections 2024-09-21T01:57:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2801827195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6241051b0 0x7fa62410bb50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:06.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2801827195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624104810 0x7fa624104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2801827195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa624103780 0x7fa624103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2801827195 >> 172.21.15.99:0/2801827195 conn(0x7fa6240ff0f0 msgr2=0x7fa6241015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2801827195 shutdown_connections 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.128+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2801827195 wait complete. 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 Processor -- start 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 -- start start 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa624103780 0x7fa624199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 0x7fa62419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6241051b0 0x7fa62419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa62410f400 con 0x7fa6241051b0 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa62410f280 con 0x7fa624104810 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62bc5d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa62410f580 con 0x7fa624103780 2024-09-21T01:57:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa6291f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 0x7fa62419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa6291f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 0x7fa62419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44416/0 (socket says 172.21.15.99:44416) 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa6291f8640 1 -- 172.21.15.99:0/2460941580 learned_addr learned my addr 172.21.15.99:0/2460941580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa6299f9640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa624103780 0x7fa624199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.129+0000 7fa62a1fa640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6241051b0 0x7fa62419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa6291f8640 1 -- 172.21.15.99:0/2460941580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa624103780 msgr2=0x7fa624199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa6291f8640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa624103780 0x7fa624199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa6291f8640 1 -- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6241051b0 msgr2=0x7fa62419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa6291f8640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6241051b0 0x7fa62419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa6291f8640 1 -- 172.21.15.99:0/2460941580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62419b9a0 con 0x7fa624104810 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa62a1fa640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6241051b0 0x7fa62419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa6291f8640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 0x7fa62419a5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa61400ea60 tx=0x7fa61400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400cde0 con 0x7fa624104810 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62419bc90 con 0x7fa624104810 2024-09-21T01:57:06.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6140035f0 con 0x7fa624104810 2024-09-21T01:57:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614010690 con 0x7fa624104810 2024-09-21T01:57:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.130+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6241a2390 con 0x7fa624104810 2024-09-21T01:57:06.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.132+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62410a940 con 0x7fa624104810 2024-09-21T01:57:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.133+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fa6140037a0 con 0x7fa624104810 2024-09-21T01:57:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.133+0000 7fa61affd640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6100b6480 0x7fa6100b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.134+0000 7fa6299f9640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6100b6480 0x7fa6100b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.134+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7fa6140e5b50 con 0x7fa624104810 2024-09-21T01:57:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.134+0000 7fa6299f9640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6100b6480 0x7fa6100b8990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa62000bbe0 tx=0x7fa62000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:06.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.138+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa6140ade50 con 0x7fa624104810 2024-09-21T01:57:06.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:06.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:57:06.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:57:06.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:06.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi092 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:06.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:06.382+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa624063800 con 0x7fa6100b6480 2024-09-21T01:57:06.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:06.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:57:06.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:57:06.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi092 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:06.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:57:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:57:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:57:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi092 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:06.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] boot 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:07.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:07.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:57:07.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:57:07.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] boot 2024-09-21T01:57:07.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T01:57:07.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:07.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:07.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:07.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:07.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:07.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:57:07.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:57:07.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: osd.0 [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] boot 2024-09-21T01:57:07.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T01:57:07.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:57:07.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:07.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:07.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:07.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:08.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:08 smithi099 ceph-mon[34404]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:08.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:08 smithi099 ceph-mon[34404]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:57:08.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:08 smithi136 ceph-mon[34754]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:08.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:08 smithi136 ceph-mon[34754]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:57:08.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:08 smithi092 ceph-mon[31190]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:08.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:08 smithi092 ceph-mon[31190]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:57:09.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:09 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/1082077222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"} : dispatch 2024-09-21T01:57:09.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:09 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"} : dispatch 2024-09-21T01:57:09.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:09 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"}]': finished 2024-09-21T01:57:09.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:09 smithi099 ceph-mon[34404]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T01:57:09.292 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:09 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:09.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:09 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/1082077222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"} : dispatch 2024-09-21T01:57:09.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:09 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"} : dispatch 2024-09-21T01:57:09.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:09 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"}]': finished 2024-09-21T01:57:09.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:09 smithi136 ceph-mon[34754]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T01:57:09.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:09 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:09.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:09 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/1082077222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"} : dispatch 2024-09-21T01:57:09.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:09 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"} : dispatch 2024-09-21T01:57:09.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:09 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b3ce9c-9425-4d9c-b63f-4a3d3181f253"}]': finished 2024-09-21T01:57:09.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:09 smithi092 ceph-mon[31190]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T01:57:09.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:09 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:10.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:10 smithi136 ceph-mon[34754]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:10.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:10 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/523206010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:57:10.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:10 smithi092 ceph-mon[31190]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:10.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:10 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/523206010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:57:10.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:10 smithi099 ceph-mon[34404]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:10.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:10 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/523206010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:57:12.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:12 smithi136 ceph-mon[34754]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:12.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:12 smithi092 ceph-mon[31190]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:12.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:12 smithi099 ceph-mon[34404]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:14.462 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:14 smithi099 ceph-mon[34404]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:14 smithi136 ceph-mon[34754]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:14.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:14 smithi092 ceph-mon[31190]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:16.443 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:16 smithi099 ceph-mon[34404]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:16 smithi136 ceph-mon[34754]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:16.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:16 smithi092 ceph-mon[31190]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:18.446 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:18 smithi099 ceph-mon[34404]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:18.446 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:57:18.446 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:18.447 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:18 smithi099 ceph-mon[34404]: Deploying daemon osd.1 on smithi099 2024-09-21T01:57:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:18 smithi136 ceph-mon[34754]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:57:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:18 smithi136 ceph-mon[34754]: Deploying daemon osd.1 on smithi099 2024-09-21T01:57:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:18 smithi092 ceph-mon[31190]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:57:18.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:18.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:18 smithi092 ceph-mon[31190]: Deploying daemon osd.1 on smithi099 2024-09-21T01:57:20.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:20 smithi099 ceph-mon[34404]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:20.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:20 smithi136 ceph-mon[34754]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:20 smithi092 ceph-mon[31190]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:22.404 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:22 smithi092 ceph-mon[31190]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:22.505 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:22 smithi099 ceph-mon[34404]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:22.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:22 smithi136 ceph-mon[34754]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:24.471 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:24 smithi099 ceph-mon[34404]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:24.472 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:24.472 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:24.472 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:24 smithi136 ceph-mon[34754]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:24 smithi092 ceph-mon[31190]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:26 smithi099 ceph-mon[34404]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:26.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:26 smithi136 ceph-mon[34754]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:26.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:26 smithi092 ceph-mon[31190]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:27.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:27.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:27.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.608+0000 7fa61affd640 1 -- 172.21.15.99:0/2460941580 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa624063800 con 0x7fa6100b6480 2024-09-21T01:57:27.609 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2024-09-21T01:57:27.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6100b6480 msgr2=0x7fa6100b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:27.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6100b6480 0x7fa6100b8990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa62000bbe0 tx=0x7fa62000b390 comp rx=0 tx=0).stop 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 msgr2=0x7fa62419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 0x7fa62419a5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa61400ea60 tx=0x7fa61400ef20 comp rx=0 tx=0).stop 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 shutdown_connections 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6100b6480 0x7fa6100b8990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6241051b0 0x7fa62419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa624104810 0x7fa62419a5e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 --2- 172.21.15.99:0/2460941580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa624103780 0x7fa624199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 >> 172.21.15.99:0/2460941580 conn(0x7fa6240ff0f0 msgr2=0x7fa624107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:27.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 shutdown_connections 2024-09-21T01:57:27.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:57:27.610+0000 7fa62bc5d640 1 -- 172.21.15.99:0/2460941580 wait complete. 2024-09-21T01:57:27.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:27.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:27.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:27.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:27.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:27.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:28.252 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T01:57:28.254 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-09-21T01:57:28.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:57:28.504 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:57:28.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:28 smithi136 ceph-mon[34754]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:28.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:28 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:28.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:28 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:28.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:28 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:28.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:28 smithi092 ceph-mon[31190]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:28.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:28 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:28.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:28 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:28.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:28 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:28 smithi099 ceph-mon[34404]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:28 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:28 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:28 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:30.081 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:57:29 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1[38361]: 2024-09-21T01:57:29.741+0000 7fc0592af640 -1 osd.1 0 log_to_monitors true 2024-09-21T01:57:30.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:30 smithi099 ceph-mon[34404]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:30.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:30 smithi099 ceph-mon[34404]: from='osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:57:30.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:30 smithi099 ceph-mon[34404]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:57:30.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:30 smithi136 ceph-mon[34754]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:30.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:30 smithi136 ceph-mon[34754]: from='osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:57:30.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:30 smithi136 ceph-mon[34754]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:57:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:30 smithi092 ceph-mon[31190]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:30 smithi092 ceph-mon[31190]: from='osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:57:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:30 smithi092 ceph-mon[31190]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:57:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:57:30.859 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-09-21T01:57:31.109 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:57:31.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 -- 172.21.15.136:0/1116643952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c104880 msgr2=0x7f211c104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:31.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 --2- 172.21.15.136:0/1116643952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c104880 0x7f211c104c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2110009900 tx=0x7f211002a480 comp rx=0 tx=0).stop 2024-09-21T01:57:31.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 -- 172.21.15.136:0/1116643952 shutdown_connections 2024-09-21T01:57:31.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 --2- 172.21.15.136:0/1116643952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 0x7f211c10bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:31.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 --2- 172.21.15.136:0/1116643952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c104880 0x7f211c104c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:31.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 --2- 172.21.15.136:0/1116643952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f211c1037f0 0x7f211c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 -- 172.21.15.136:0/1116643952 >> 172.21.15.136:0/1116643952 conn(0x7f211c0ff180 msgr2=0x7f211c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 -- 172.21.15.136:0/1116643952 shutdown_connections 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.516+0000 7f2121c74640 1 -- 172.21.15.136:0/1116643952 wait complete. 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 Processor -- start 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 -- start start 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 0x7f211c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f211c104880 0x7f211c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 0x7f211c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10f3b0 con 0x7f211c104880 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10f230 con 0x7f211c105220 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f2121c74640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10f530 con 0x7f211c1037f0 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f211b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 0x7f211c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f211b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 0x7f211c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:55998/0 (socket says 172.21.15.136:55998) 2024-09-21T01:57:31.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.517+0000 7f211b7fe640 1 -- 172.21.15.136:0/3327785129 learned_addr learned my addr 172.21.15.136:0/3327785129 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211affd640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f211c104880 0x7f211c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211bfff640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 0x7f211c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211b7fe640 1 -- 172.21.15.136:0/3327785129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 msgr2=0x7f211c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211b7fe640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 0x7f211c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211b7fe640 1 -- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f211c104880 msgr2=0x7f211c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211b7fe640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f211c104880 0x7f211c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211b7fe640 1 -- 172.21.15.136:0/3327785129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c19b9a0 con 0x7f211c1037f0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211bfff640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 0x7f211c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f211b7fe640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 0x7f211c19a000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f210c00c920 tx=0x7f210c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c007c70 con 0x7f211c1037f0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c19bc90 con 0x7f211c1037f0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f210c007e10 con 0x7f211c1037f0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c018820 con 0x7f211c1037f0 2024-09-21T01:57:31.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.518+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c1a2420 con 0x7f211c1037f0 2024-09-21T01:57:31.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.520+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c103c60 con 0x7f211c1037f0 2024-09-21T01:57:31.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.521+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7f210c0189c0 con 0x7f211c1037f0 2024-09-21T01:57:31.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.521+0000 7f2118ff9640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21040b6480 0x7f21040b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:31.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.521+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7f210c0e5b60 con 0x7f211c1037f0 2024-09-21T01:57:31.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.522+0000 7f211affd640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21040b6480 0x7f21040b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:31.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.522+0000 7f211affd640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21040b6480 0x7f21040b8990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f211002a9a0 tx=0x7f2110005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:31.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.526+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f210c014030 con 0x7f211c1037f0 2024-09-21T01:57:31.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:31.719+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f211c063800 con 0x7f21040b6480 2024-09-21T01:57:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:57:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:57:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:57:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:31.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:31 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:57:31.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:57:31.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:57:31.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:31.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:31 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:57:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:57:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:57:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:31 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:32.722 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:32.722 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi099 2024-09-21T01:57:32.722 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-21T01:57:32.722 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='osd.1 ' entity='osd.1' 2024-09-21T01:57:32.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi099 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='osd.1 ' entity='osd.1' 2024-09-21T01:57:32.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.831 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1[38361]: 2024-09-21T01:57:32.452+0000 7fc055a46640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T01:57:32.831 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1[38361]: 2024-09-21T01:57:32.589+0000 7fc04d027640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi099 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:32.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='osd.1 ' entity='osd.1' 2024-09-21T01:57:32.835 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:33.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:33 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:57:33.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:33 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:57:33.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:33 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:57:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:33 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:57:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:33.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:33 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:57:33.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:33 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:57:33.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] boot 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/1684582690' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"} : dispatch 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"} : dispatch 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"}]': finished 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:34.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:34 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/2302419458' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:57:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] boot 2024-09-21T01:57:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T01:57:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/1684582690' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"} : dispatch 2024-09-21T01:57:34.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"} : dispatch 2024-09-21T01:57:34.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"}]': finished 2024-09-21T01:57:34.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T01:57:34.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:34.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:34 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/2302419458' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:57:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:57:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: osd.1 [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] boot 2024-09-21T01:57:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T01:57:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:57:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/1684582690' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"} : dispatch 2024-09-21T01:57:34.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"} : dispatch 2024-09-21T01:57:34.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d560a67b-26ab-47d8-9a94-c9f293238171"}]': finished 2024-09-21T01:57:34.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: osdmap e14: 3 total, 2 up, 3 in 2024-09-21T01:57:34.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:34.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:34 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/2302419458' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:57:36.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:36 smithi136 ceph-mon[34754]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:36.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:36 smithi092 ceph-mon[31190]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:36.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:36 smithi099 ceph-mon[34404]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:38.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:38 smithi092 ceph-mon[31190]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:38.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:38 smithi099 ceph-mon[34404]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:39.019 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:38 smithi136 ceph-mon[34754]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:40.708 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:40 smithi136 ceph-mon[34754]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:40.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:40 smithi092 ceph-mon[31190]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:40 smithi099 ceph-mon[34404]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:42 smithi136 ceph-mon[34754]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:57:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:42 smithi092 ceph-mon[31190]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:42.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:57:42.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:42 smithi099 ceph-mon[34404]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:57:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:43.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:43 smithi099 ceph-mon[34404]: Deploying daemon osd.2 on smithi136 2024-09-21T01:57:43.905 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:43 smithi136 ceph-mon[34754]: Deploying daemon osd.2 on smithi136 2024-09-21T01:57:44.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:43 smithi092 ceph-mon[31190]: Deploying daemon osd.2 on smithi136 2024-09-21T01:57:44.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:44 smithi099 ceph-mon[34404]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:44.905 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:44 smithi136 ceph-mon[34754]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:45.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:44 smithi092 ceph-mon[31190]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:46 smithi099 ceph-mon[34404]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:47.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:46 smithi136 ceph-mon[34754]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:46 smithi092 ceph-mon[31190]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:48.691 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:48 smithi136 ceph-mon[34754]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:48 smithi099 ceph-mon[34404]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:49.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:48 smithi092 ceph-mon[31190]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:49.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:49.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:49.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:50.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:50.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:50.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:50 smithi099 ceph-mon[34404]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:50 smithi136 ceph-mon[34754]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:50 smithi092 ceph-mon[31190]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:52 smithi092 ceph-mon[31190]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:52.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:52.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:52.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:52 smithi099 ceph-mon[34404]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:52 smithi136 ceph-mon[34754]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:52.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.584 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.584 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:52.584 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:52.584 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:52.625 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-09-21T01:57:52.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.624+0000 7f2118ff9640 1 -- 172.21.15.136:0/3327785129 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f211c063800 con 0x7f21040b6480 2024-09-21T01:57:52.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21040b6480 msgr2=0x7f21040b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21040b6480 0x7f21040b8990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f211002a9a0 tx=0x7f2110005fd0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 msgr2=0x7f211c19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 0x7f211c19a000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f210c00c920 tx=0x7f210c00cde0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 shutdown_connections 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21040b6480 0x7f21040b8990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f211c105220 0x7f211c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f211c104880 0x7f211c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 --2- 172.21.15.136:0/3327785129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f211c1037f0 0x7f211c19a000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 >> 172.21.15.136:0/3327785129 conn(0x7f211c0ff180 msgr2=0x7f211c107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.626+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 shutdown_connections 2024-09-21T01:57:52.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:57:52.627+0000 7f2121c74640 1 -- 172.21.15.136:0/3327785129 wait complete. 2024-09-21T01:57:53.280 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T01:57:53.282 INFO:tasks.cephadm:Deploying osd.3 on smithi092 with /dev/vg_nvme/lv_3... 2024-09-21T01:57:53.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:57:53.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:53 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:53.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:53 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:53.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:53 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:53.538 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:53.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:53 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:53.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:53 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:53.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:53 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:53.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:53 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:57:53.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:53 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:53.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:53 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:54.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:54 smithi136 ceph-mon[34754]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:54.765 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:57:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2[39147]: 2024-09-21T01:57:54.671+0000 7f4ca065b640 -1 osd.2 0 log_to_monitors true 2024-09-21T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:54 smithi092 ceph-mon[31190]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:55.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:54 smithi099 ceph-mon[34404]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:55.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:55 smithi092 ceph-mon[31190]: from='osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:57:55.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:55 smithi092 ceph-mon[31190]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:57:55.995 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:57:56.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-09-21T01:57:56.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:55 smithi136 ceph-mon[34754]: from='osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:57:56.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:55 smithi136 ceph-mon[34754]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:57:56.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:55 smithi099 ceph-mon[34404]: from='osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:57:56.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:55 smithi099 ceph-mon[34404]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:57:56.267 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:57:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 -- 172.21.15.92:0/1318874904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 msgr2=0x7fd2ec101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/1318874904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec101f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009900 tx=0x7fd2dc02a410 comp rx=0 tx=0).stop 2024-09-21T01:57:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 -- 172.21.15.92:0/1318874904 shutdown_connections 2024-09-21T01:57:56.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/1318874904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2ec109540 0x7fd2ec10b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/1318874904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ec106be0 0x7fd2ec109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/1318874904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec101f50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 -- 172.21.15.92:0/1318874904 >> 172.21.15.92:0/1318874904 conn(0x7fd2ec0fd770 msgr2=0x7fd2ec0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 -- 172.21.15.92:0/1318874904 shutdown_connections 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 -- 172.21.15.92:0/1318874904 wait complete. 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 Processor -- start 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.724+0000 7fd2f28e1640 1 -- start start 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f28e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f28e1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ec106be0 0x7fd2ec19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f28e1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2ec109540 0x7fd2ec19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f28e1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec10f3c0 con 0x7fd2ec101b40 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f28e1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec10f240 con 0x7fd2ec109540 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f28e1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec10f540 con 0x7fd2ec106be0 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec19c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39422/0 (socket says 172.21.15.92:39422) 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 -- 172.21.15.92:0/506549049 learned_addr learned my addr 172.21.15.92:0/506549049 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2eb7fe640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ec106be0 0x7fd2ec19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2f0e7e640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2ec109540 0x7fd2ec19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 -- 172.21.15.92:0/506549049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ec106be0 msgr2=0x7fd2ec19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ec106be0 0x7fd2ec19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 -- 172.21.15.92:0/506549049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2ec109540 msgr2=0x7fd2ec19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2ec109540 0x7fd2ec19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.725+0000 7fd2ebfff640 1 -- 172.21.15.92:0/506549049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec19dc40 con 0x7fd2ec101b40 2024-09-21T01:57:56.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.726+0000 7fd2ebfff640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec19c210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc006010 tx=0x7fd2dc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.726+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc03f030 con 0x7fd2ec101b40 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.726+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2dc003cb0 con 0x7fd2ec101b40 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.726+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc035070 con 0x7fd2ec101b40 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.726+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec19ded0 con 0x7fd2ec101b40 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.726+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec1a4460 con 0x7fd2ec101b40 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.727+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec1a46c0 con 0x7fd2ec101b40 2024-09-21T01:57:56.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.729+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 154456+0+0 (secure 0 0 0) 0x7fd2dc042030 con 0x7fd2ec101b40 2024-09-21T01:57:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.730+0000 7fd2e97fa640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd2d40b6480 0x7fd2d40b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:57:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.730+0000 7fd2eb7fe640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd2d40b6480 0x7fd2d40b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:57:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.730+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7fd2dc1069f0 con 0x7fd2ec101b40 2024-09-21T01:57:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.731+0000 7fd2eb7fe640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd2d40b6480 0x7fd2d40b8990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00058c0 tx=0x7fd2e000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:57:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.734+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2dc0ce9d0 con 0x7fd2ec101b40 2024-09-21T01:57:56.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:56.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:57:56.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi136 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:56.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:56.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:56 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:56.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:57:56.923+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd2ec103a90 con 0x7fd2d40b6480 2024-09-21T01:57:57.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2[39147]: 2024-09-21T01:57:56.640+0000 7f4c9c5dd640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T01:57:57.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2[39147]: 2024-09-21T01:57:56.645+0000 7f4c983db640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi136 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:57.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:57.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:56 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi136 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:57:57.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi136 to 17088M 2024-09-21T01:57:57.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:57.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:57:57.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:56 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:57.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:58.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T01:57:58.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:57:58.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:58.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:58.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:58.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:58.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:58.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T01:57:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:57:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:57:58.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:57:58.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:57:58.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:57:58.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:58 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:57:58.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:58 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:57:58.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:58 smithi092 ceph-mon[31190]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:58 smithi092 ceph-mon[31190]: osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] boot 2024-09-21T01:57:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:58 smithi092 ceph-mon[31190]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T01:57:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:59.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:58 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:57:59.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:58 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:57:59.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:58 smithi136 ceph-mon[34754]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:59.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:58 smithi136 ceph-mon[34754]: osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] boot 2024-09-21T01:57:59.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:58 smithi136 ceph-mon[34754]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T01:57:59.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:59.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:58 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:57:59.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:58 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:57:59.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:58 smithi099 ceph-mon[34404]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:57:59.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:58 smithi099 ceph-mon[34404]: osd.2 [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] boot 2024-09-21T01:57:59.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:58 smithi099 ceph-mon[34404]: osdmap e17: 3 total, 3 up, 3 in 2024-09-21T01:57:59.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2580314392' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200ba4ba-2dd5-4dbc-a805-d0b25c4816b9"} : dispatch 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2580314392' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "200ba4ba-2dd5-4dbc-a805-d0b25c4816b9"}]': finished 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: osdmap e19: 4 total, 3 up, 4 in 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:57:59.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:57:59 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:58:00.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T01:58:00.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:58:00.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2580314392' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200ba4ba-2dd5-4dbc-a805-d0b25c4816b9"} : dispatch 2024-09-21T01:58:00.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:58:00.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2580314392' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "200ba4ba-2dd5-4dbc-a805-d0b25c4816b9"}]': finished 2024-09-21T01:58:00.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: osdmap e19: 4 total, 3 up, 4 in 2024-09-21T01:58:00.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:00.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:57:59 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:58:00.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T01:58:00.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:58:00.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2580314392' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200ba4ba-2dd5-4dbc-a805-d0b25c4816b9"} : dispatch 2024-09-21T01:58:00.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:58:00.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2580314392' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "200ba4ba-2dd5-4dbc-a805-d0b25c4816b9"}]': finished 2024-09-21T01:58:00.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: osdmap e19: 4 total, 3 up, 4 in 2024-09-21T01:58:00.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:00.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:57:59 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:58:00.834 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:58:00.834 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:00.835 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51581]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:00.835 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51581]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:00.835 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:58:00.835 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:00.835 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51584]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:00.835 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51584]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:00.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 ceph-mon[31190]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:00.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2040482941' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:00.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:58:00.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 ceph-mon[31190]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T01:58:00.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:00.835 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:00.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:58:00.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:00.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51587]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:00.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51587]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:00.836 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41965]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41965]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41965]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41965]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41968]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:00.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41968]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:00.916 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41968]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 ceph-mon[34404]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2040482941' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 ceph-mon[34404]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41971]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41971]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41971]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41971]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:00.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41974]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:58:01.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43063]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:58:01.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:01.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43063]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:01.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43063]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:01.034 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:58:01.034 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:01.034 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43066]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:01.034 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43066]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 ceph-mon[34754]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2040482941' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 ceph-mon[34754]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:01.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43069]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43069]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43069]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43069]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43072]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43072]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:01.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:00 smithi136 sudo[43072]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51590]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:00 smithi092 sudo[51590]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:58:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41974]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:58:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:00 smithi099 sudo[41974]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:58:01.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:01.810+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd2dc0ca540 con 0x7fd2ec101b40 2024-09-21T01:58:02.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:02.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T01:58:02.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T01:58:02.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:58:02.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T01:58:02.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:02 smithi099 ceph-mon[34404]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:02 smithi099 ceph-mon[34404]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T01:58:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:02 smithi136 ceph-mon[34754]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:02 smithi136 ceph-mon[34754]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T01:58:03.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:02 smithi092 ceph-mon[31190]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:03.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:02 smithi092 ceph-mon[31190]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-21T01:58:05.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:04 smithi136 ceph-mon[34754]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:05.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:04 smithi092 ceph-mon[31190]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:05.136 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:04 smithi099 ceph-mon[34404]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:06.944 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:06 smithi092 ceph-mon[31190]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:06 smithi099 ceph-mon[34404]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:06 smithi136 ceph-mon[34754]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:58:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:08.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:58:08.125 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:08.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:58:08.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:09.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:08 smithi092 ceph-mon[31190]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:09.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:08 smithi092 ceph-mon[31190]: Deploying daemon osd.3 on smithi092 2024-09-21T01:58:09.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:08 smithi099 ceph-mon[34404]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:09.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:08 smithi099 ceph-mon[34404]: Deploying daemon osd.3 on smithi092 2024-09-21T01:58:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:08 smithi136 ceph-mon[34754]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:08 smithi136 ceph-mon[34754]: Deploying daemon osd.3 on smithi092 2024-09-21T01:58:10.840 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:10 smithi092 ceph-mon[31190]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:10 smithi099 ceph-mon[34404]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:10 smithi136 ceph-mon[34754]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:12.977 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:12 smithi092 ceph-mon[31190]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:12 smithi099 ceph-mon[34404]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:12 smithi136 ceph-mon[34754]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:13.962 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:13 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:13.962 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:13 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:13.962 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:13 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:14.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:13 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:14.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:13 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:14.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:13 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:14.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:13 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:14.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:13 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:14.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:13 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:15.105 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:14 smithi092 ceph-mon[31190]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:14 smithi136 ceph-mon[34754]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:14 smithi099 ceph-mon[34404]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:17.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:16 smithi092 ceph-mon[31190]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:17.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:17.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:17.137 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:16 smithi136 ceph-mon[34754]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:17.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:17.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:16 smithi099 ceph-mon[34404]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.475+0000 7fd2e97fa640 1 -- 172.21.15.92:0/506549049 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2ec103a90 con 0x7fd2d40b6480 2024-09-21T01:58:17.476 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 3 on host 'smithi092' 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd2d40b6480 msgr2=0x7fd2d40b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd2d40b6480 0x7fd2d40b8990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00058c0 tx=0x7fd2e000a250 comp rx=0 tx=0).stop 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 msgr2=0x7fd2ec19c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec19c210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc006010 tx=0x7fd2dc0043b0 comp rx=0 tx=0).stop 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 shutdown_connections 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd2d40b6480 0x7fd2d40b8990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2ec109540 0x7fd2ec19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ec106be0 0x7fd2ec19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:17.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 --2- 172.21.15.92:0/506549049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ec101b40 0x7fd2ec19c210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:17.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 >> 172.21.15.92:0/506549049 conn(0x7fd2ec0fd770 msgr2=0x7fd2ec103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:17.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 shutdown_connections 2024-09-21T01:58:17.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:58:17.477+0000 7fd2f28e1640 1 -- 172.21.15.92:0/506549049 wait complete. 2024-09-21T01:58:18.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:18.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:18.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:18.111 DEBUG:teuthology.orchestra.run.smithi092:osd.3> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T01:58:18.113 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2024-09-21T01:58:18.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:58:18.141 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:18.141 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:18.141 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:18.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:18.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:18.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:18.369 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:58:19.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:18 smithi099 ceph-mon[34404]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:19.256 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:18 smithi092 ceph-mon[31190]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:19.256 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 01:58:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T01:58:18.872+0000 7f25a3cc3640 -1 osd.3 0 log_to_monitors true 2024-09-21T01:58:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:18 smithi136 ceph-mon[34754]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:20.008 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:19 smithi099 ceph-mon[34404]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:58:20.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:19 smithi092 ceph-mon[31190]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:58:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:19 smithi136 ceph-mon[34754]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:58:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:58:20.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-09-21T01:58:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:58:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:20 smithi099 ceph-mon[34404]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:20 smithi099 ceph-mon[34404]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:58:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:20 smithi099 ceph-mon[34404]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T01:58:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:20 smithi099 ceph-mon[34404]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:58:21.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:20 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:20 smithi136 ceph-mon[34754]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:20 smithi136 ceph-mon[34754]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:58:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:20 smithi136 ceph-mon[34754]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T01:58:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:20 smithi136 ceph-mon[34754]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:58:21.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:20 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:20 smithi092 ceph-mon[31190]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:20 smithi092 ceph-mon[31190]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:58:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:20 smithi092 ceph-mon[31190]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T01:58:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:20 smithi092 ceph-mon[31190]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:58:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:20 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:21.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 -- 172.21.15.99:0/2057609416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490102670 msgr2=0x7fe490102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:21.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 --2- 172.21.15.99:0/2057609416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490102670 0x7fe490102ae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe480009860 tx=0x7fe48002a3d0 comp rx=0 tx=0).stop 2024-09-21T01:58:21.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 -- 172.21.15.99:0/2057609416 shutdown_connections 2024-09-21T01:58:21.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 --2- 172.21.15.99:0/2057609416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4901040a0 0x7fe49010aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 --2- 172.21.15.99:0/2057609416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe490103700 0x7fe490103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 --2- 172.21.15.99:0/2057609416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490102670 0x7fe490102ae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 -- 172.21.15.99:0/2057609416 >> 172.21.15.99:0/2057609416 conn(0x7fe4900fdfe0 msgr2=0x7fe490100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 -- 172.21.15.99:0/2057609416 shutdown_connections 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 -- 172.21.15.99:0/2057609416 wait complete. 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.506+0000 7fe4967d4640 1 Processor -- start 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 -- start start 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe490102670 0x7fe490199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 0x7fe49019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4901040a0 0x7fe49019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe49010e2e0 con 0x7fe490102670 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe49010e160 con 0x7fe490103700 2024-09-21T01:58:21.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe4967d4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe49010e460 con 0x7fe4901040a0 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 0x7fe49019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe490102670 0x7fe490199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 0x7fe49019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45700/0 (socket says 172.21.15.99:45700) 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48f7fe640 1 -- 172.21.15.99:0/524154172 learned_addr learned my addr 172.21.15.99:0/524154172 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe494d71640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4901040a0 0x7fe49019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48ffff640 1 -- 172.21.15.99:0/524154172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4901040a0 msgr2=0x7fe49019b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48ffff640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4901040a0 0x7fe49019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48ffff640 1 -- 172.21.15.99:0/524154172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 msgr2=0x7fe49019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48ffff640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 0x7fe49019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48ffff640 1 -- 172.21.15.99:0/524154172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49019b9f0 con 0x7fe490102670 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.507+0000 7fe48f7fe640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 0x7fe49019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.508+0000 7fe48ffff640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe490102670 0x7fe490199fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe480000c00 tx=0x7fe480005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.508+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480005640 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.508+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49019bc80 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.508+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe480031400 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.508+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4800049b0 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.508+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4901a2210 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.509+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4901a2470 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.511+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe480039040 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.511+0000 7fe48d7fa640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4780b66b0 0x7fe4780b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.512+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3422+0+0 (secure 0 0 0) 0x7fe4801073d0 con 0x7fe490102670 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.512+0000 7fe48f7fe640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4780b66b0 0x7fe4780b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.512+0000 7fe48f7fe640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4780b66b0 0x7fe4780b8bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe484005fd0 tx=0x7fe48400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.515+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe480040030 con 0x7fe490102670 2024-09-21T01:58:21.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:21.708+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe49006c270 con 0x7fe4780b66b0 2024-09-21T01:58:22.054 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T01:58:21.761+0000 7f259fc45640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T01:58:22.054 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T01:58:21.821+0000 7f2597a3b640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi092 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-21T01:58:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:22.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:21 smithi092 ceph-mon[31190]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi092 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:21 smithi099 ceph-mon[34404]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi092 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:22.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:21 smithi136 ceph-mon[34754]: from='osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265]' entity='osd.3' 2024-09-21T01:58:23.146 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] boot 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T01:58:23.147 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:23.172 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:58:23.172 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:58:23.172 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:23.172 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:58:23.172 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:23.172 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] boot 2024-09-21T01:58:23.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T01:58:23.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:58:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:58:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:58:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: from='client.14499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:58:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: osd.3 [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] boot 2024-09-21T01:58:23.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T01:58:23.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:58:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:58:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/109826210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"} : dispatch 2024-09-21T01:58:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"} : dispatch 2024-09-21T01:58:24.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"}]': finished 2024-09-21T01:58:24.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T01:58:24.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:24.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:24.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:58:24.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/109826210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"} : dispatch 2024-09-21T01:58:24.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"} : dispatch 2024-09-21T01:58:24.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"}]': finished 2024-09-21T01:58:24.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T01:58:24.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:58:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/109826210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"} : dispatch 2024-09-21T01:58:24.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"} : dispatch 2024-09-21T01:58:24.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39775100-fc5d-4e58-b5d8-e81dbc327b31"}]': finished 2024-09-21T01:58:24.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T01:58:24.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:25.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/3983541864' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:25.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/3983541864' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:25.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/3983541864' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:26.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:26 smithi136 ceph-mon[34754]: pgmap v114: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:26.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:26 smithi136 ceph-mon[34754]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T01:58:26.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:26 smithi092 ceph-mon[31190]: pgmap v114: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:26.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:26 smithi092 ceph-mon[31190]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T01:58:26.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:26 smithi099 ceph-mon[34404]: pgmap v114: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:26.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:26 smithi099 ceph-mon[34404]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T01:58:28.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:28 smithi136 ceph-mon[34754]: pgmap v115: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:28.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:28 smithi092 ceph-mon[31190]: pgmap v115: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:28.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:28 smithi099 ceph-mon[34404]: pgmap v115: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:30.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:30 smithi136 ceph-mon[34754]: pgmap v116: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:30.542 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:30 smithi099 ceph-mon[34404]: pgmap v116: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:30.557 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:30 smithi092 ceph-mon[31190]: pgmap v116: 1 pgs: 1 activating+degraded; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:58:31.479 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:31 smithi099 ceph-mon[34404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T01:58:31.479 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:31 smithi099 ceph-mon[34404]: Cluster is now healthy 2024-09-21T01:58:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:31 smithi136 ceph-mon[34754]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T01:58:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:31 smithi136 ceph-mon[34754]: Cluster is now healthy 2024-09-21T01:58:31.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:31 smithi092 ceph-mon[31190]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T01:58:31.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:31 smithi092 ceph-mon[31190]: Cluster is now healthy 2024-09-21T01:58:32.492 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:32 smithi099 ceph-mon[34404]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:58:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:32 smithi136 ceph-mon[34754]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:58:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:32 smithi092 ceph-mon[31190]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:58:33.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:58:33.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:33.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:33 smithi099 ceph-mon[34404]: Deploying daemon osd.4 on smithi099 2024-09-21T01:58:33.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:58:33.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:33.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:33 smithi136 ceph-mon[34754]: Deploying daemon osd.4 on smithi099 2024-09-21T01:58:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:58:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:33 smithi092 ceph-mon[31190]: Deploying daemon osd.4 on smithi099 2024-09-21T01:58:34.444 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:34 smithi099 ceph-mon[34404]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:58:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:34 smithi136 ceph-mon[34754]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:58:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:34 smithi092 ceph-mon[31190]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:58:36.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:36 smithi136 ceph-mon[34754]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T01:58:36.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:36 smithi092 ceph-mon[31190]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T01:58:36.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:36 smithi099 ceph-mon[34404]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T01:58:38.490 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:38 smithi099 ceph-mon[34404]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:38.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:38 smithi136 ceph-mon[34754]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:38.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:38 smithi092 ceph-mon[31190]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:39.471 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:39.471 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:39.471 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:39.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:39.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:39.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:39.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:39.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:39.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:41.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:40 smithi136 ceph-mon[34754]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:41.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:40 smithi099 ceph-mon[34404]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:41.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:40 smithi092 ceph-mon[31190]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:42.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.164+0000 7fe48d7fa640 1 -- 172.21.15.99:0/524154172 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe49006c270 con 0x7fe4780b66b0 2024-09-21T01:58:42.165 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4780b66b0 msgr2=0x7fe4780b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4780b66b0 0x7fe4780b8bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe484005fd0 tx=0x7fe48400b4a0 comp rx=0 tx=0).stop 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe490102670 msgr2=0x7fe490199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe490102670 0x7fe490199fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe480000c00 tx=0x7fe480005040 comp rx=0 tx=0).stop 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 shutdown_connections 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4780b66b0 0x7fe4780b8bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4901040a0 0x7fe49019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe490103700 0x7fe49019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 --2- 172.21.15.99:0/524154172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe490102670 0x7fe490199fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:42.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 >> 172.21.15.99:0/524154172 conn(0x7fe4900fdfe0 msgr2=0x7fe49006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:42.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 shutdown_connections 2024-09-21T01:58:42.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:58:42.166+0000 7fe4967d4640 1 -- 172.21.15.99:0/524154172 wait complete. 2024-09-21T01:58:42.764 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:42.764 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.764 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:42.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:42.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:42.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:42.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.791 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T01:58:42.793 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-09-21T01:58:42.794 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:58:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:58:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:58:42.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:42.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:43.049 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:58:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:44 smithi099 ceph-mon[34404]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:44.713 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:44 smithi136 ceph-mon[34754]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:44.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:44 smithi092 ceph-mon[31190]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:45.081 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 01:58:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T01:58:44.827+0000 7f6467512640 -1 osd.4 0 log_to_monitors true 2024-09-21T01:58:45.353 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T01:58:45.379 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-09-21T01:58:45.631 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:58:45.636 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:45 smithi099 ceph-mon[34404]: from='osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:58:45.636 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:45 smithi099 ceph-mon[34404]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:58:45.757 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:45 smithi136 ceph-mon[34754]: from='osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:58:45.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:45 smithi136 ceph-mon[34754]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:58:45.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:45 smithi092 ceph-mon[31190]: from='osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:58:45.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:45 smithi092 ceph-mon[31190]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:58:46.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 -- 172.21.15.136:0/3274328804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c102640 msgr2=0x7fb51c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:46.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 --2- 172.21.15.136:0/3274328804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c102640 0x7fb51c102ab0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb510009900 tx=0x7fb51002a480 comp rx=0 tx=0).stop 2024-09-21T01:58:46.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 -- 172.21.15.136:0/3274328804 shutdown_connections 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 --2- 172.21.15.136:0/3274328804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c104070 0x7fb51c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 --2- 172.21.15.136:0/3274328804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c1036d0 0x7fb51c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 --2- 172.21.15.136:0/3274328804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c102640 0x7fb51c102ab0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 -- 172.21.15.136:0/3274328804 >> 172.21.15.136:0/3274328804 conn(0x7fb51c0fdfb0 msgr2=0x7fb51c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 -- 172.21.15.136:0/3274328804 shutdown_connections 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.091+0000 7fb522d3d640 1 -- 172.21.15.136:0/3274328804 wait complete. 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 Processor -- start 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 -- start start 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c102640 0x7fb51c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c1036d0 0x7fb51c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 0x7fb51c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c10e330 con 0x7fb51c102640 2024-09-21T01:58:46.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c10e1b0 con 0x7fb51c1036d0 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb522d3d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c10e4b0 con 0x7fb51c104070 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb5212da640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 0x7fb51c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb520ad9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c102640 0x7fb51c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb5212da640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 0x7fb51c19b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:37106/0 (socket says 172.21.15.136:37106) 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb5212da640 1 -- 172.21.15.136:0/1352185741 learned_addr learned my addr 172.21.15.136:0/1352185741 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.092+0000 7fb51bfff640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c1036d0 0x7fb51c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb5212da640 1 -- 172.21.15.136:0/1352185741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c1036d0 msgr2=0x7fb51c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb5212da640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c1036d0 0x7fb51c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb5212da640 1 -- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c102640 msgr2=0x7fb51c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb5212da640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c102640 0x7fb51c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb5212da640 1 -- 172.21.15.136:0/1352185741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c19ba30 con 0x7fb51c104070 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb5212da640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 0x7fb51c19b210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb51400eaa0 tx=0x7fb51400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:46.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514004d20 con 0x7fb51c104070 2024-09-21T01:58:46.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c19bd20 con 0x7fb51c104070 2024-09-21T01:58:46.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb514005020 con 0x7fb51c104070 2024-09-21T01:58:46.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5140116b0 con 0x7fb51c104070 2024-09-21T01:58:46.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.093+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c1a2420 con 0x7fb51c104070 2024-09-21T01:58:46.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.095+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51c102ab0 con 0x7fb51c104070 2024-09-21T01:58:46.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.096+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fb5140051d0 con 0x7fb51c104070 2024-09-21T01:58:46.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.096+0000 7fb519ffb640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb50c0b6510 0x7fb50c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:58:46.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.097+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3805+0+0 (secure 0 0 0) 0x7fb5140e62c0 con 0x7fb51c104070 2024-09-21T01:58:46.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.097+0000 7fb520ad9640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb50c0b6510 0x7fb50c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:58:46.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.097+0000 7fb520ad9640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb50c0b6510 0x7fb50c0b8a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb510004770 tx=0x7fb5100314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:58:46.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.101+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb514016030 con 0x7fb51c104070 2024-09-21T01:58:46.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:58:46.312+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb51c1a25f0 con 0x7fb50c0b6510 2024-09-21T01:58:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:58:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T01:58:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi099 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:58:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:58:46.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:46.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi099 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:46.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:46.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:58:46.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:58:46.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:58:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T01:58:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi099 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:58:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:58:46.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:58:46.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:47 smithi136 ceph-mon[34754]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:58:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:47 smithi136 ceph-mon[34754]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T01:58:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:47 smithi136 ceph-mon[34754]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:58:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:47 smithi136 ceph-mon[34754]: from='osd.4 ' entity='osd.4' 2024-09-21T01:58:47.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:47 smithi092 ceph-mon[31190]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:58:47.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:47 smithi092 ceph-mon[31190]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T01:58:47.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:47 smithi092 ceph-mon[31190]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:58:47.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:47.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:47.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:47 smithi092 ceph-mon[31190]: from='osd.4 ' entity='osd.4' 2024-09-21T01:58:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-mon[34404]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:58:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-mon[34404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T01:58:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-mon[34404]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:58:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-mon[34404]: from='osd.4 ' entity='osd.4' 2024-09-21T01:58:47.832 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T01:58:47.458+0000 7f6463494640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T01:58:47.832 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 01:58:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T01:58:47.638+0000 7f645b28a640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:58:48.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:48 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:58:48.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:48 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:58:48.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:48 smithi136 ceph-mon[34754]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:48.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:48 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:48.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:48 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:48 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:58:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:48 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:58:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:48 smithi092 ceph-mon[31190]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:48 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:48 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:48 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:58:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:48 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:58:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:48 smithi099 ceph-mon[34404]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:48 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:48 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:49.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] boot 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/2954261962' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"} : dispatch 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"} : dispatch 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"}]': finished 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: osdmap e31: 6 total, 5 up, 6 in 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:58:49.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:49 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/4203268502' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:49.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] boot 2024-09-21T01:58:49.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T01:58:49.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:49.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/2954261962' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"} : dispatch 2024-09-21T01:58:49.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"} : dispatch 2024-09-21T01:58:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"}]': finished 2024-09-21T01:58:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: osdmap e31: 6 total, 5 up, 6 in 2024-09-21T01:58:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:58:49.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:49 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/4203268502' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: osd.4 [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] boot 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/2954261962' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"} : dispatch 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"} : dispatch 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa5283a3-4b38-49cb-9edc-622f2c227ae1"}]': finished 2024-09-21T01:58:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: osdmap e31: 6 total, 5 up, 6 in 2024-09-21T01:58:49.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:58:49.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:49 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/4203268502' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:58:51.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:50 smithi136 ceph-mon[34754]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:51.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:50 smithi136 ceph-mon[34754]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T01:58:51.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:58:51.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:50 smithi092 ceph-mon[31190]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:51.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:50 smithi092 ceph-mon[31190]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T01:58:51.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:58:51.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:50 smithi099 ceph-mon[34404]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:58:51.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:50 smithi099 ceph-mon[34404]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T01:58:51.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:58:53.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:52 smithi136 ceph-mon[34754]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:53.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:52 smithi092 ceph-mon[31190]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:53.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:52 smithi099 ceph-mon[34404]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:55.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:54 smithi136 ceph-mon[34754]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:55.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:54 smithi092 ceph-mon[31190]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:55.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:54 smithi099 ceph-mon[34404]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:57.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:56 smithi136 ceph-mon[34754]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:57.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:56 smithi092 ceph-mon[31190]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:57.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:56 smithi099 ceph-mon[34404]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:57.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:58:57.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:57 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:58.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:58:58.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:57 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:58:58.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:57 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:58:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:58 smithi136 ceph-mon[34754]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:58:58 smithi136 ceph-mon[34754]: Deploying daemon osd.5 on smithi136 2024-09-21T01:58:59.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:58 smithi092 ceph-mon[31190]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:59.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:58:58 smithi092 ceph-mon[31190]: Deploying daemon osd.5 on smithi136 2024-09-21T01:58:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:58 smithi099 ceph-mon[34404]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:58:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:58:58 smithi099 ceph-mon[34404]: Deploying daemon osd.5 on smithi136 2024-09-21T01:59:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:00 smithi136 ceph-mon[34754]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:01.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:00 smithi092 ceph-mon[31190]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:01.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:00 smithi099 ceph-mon[34404]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:03.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:02 smithi136 ceph-mon[34754]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:03.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:02 smithi092 ceph-mon[31190]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:02 smithi099 ceph-mon[34404]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:03.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:03.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:03.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:04.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:04.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:04.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:04.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:04.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:04.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:04 smithi136 ceph-mon[34754]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:05.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:04 smithi092 ceph-mon[31190]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:05.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:04 smithi099 ceph-mon[34404]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:06 smithi136 ceph-mon[34754]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:06.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:06.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:06.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:06 smithi092 ceph-mon[31190]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:07.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:07.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:06 smithi099 ceph-mon[34404]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.320+0000 7fb519ffb640 1 -- 172.21.15.136:0/1352185741 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb51c1a25f0 con 0x7fb50c0b6510 2024-09-21T01:59:07.321 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-09-21T01:59:07.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.322+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb50c0b6510 msgr2=0x7fb50c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:07.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.322+0000 7fb522d3d640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb50c0b6510 0x7fb50c0b8a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb510004770 tx=0x7fb5100314a0 comp rx=0 tx=0).stop 2024-09-21T01:59:07.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.322+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 msgr2=0x7fb51c19b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:07.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.322+0000 7fb522d3d640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 0x7fb51c19b210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb51400eaa0 tx=0x7fb51400eea0 comp rx=0 tx=0).stop 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 shutdown_connections 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb50c0b6510 0x7fb50c0b8a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb51c104070 0x7fb51c19b210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c1036d0 0x7fb51c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 --2- 172.21.15.136:0/1352185741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c102640 0x7fb51c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 >> 172.21.15.136:0/1352185741 conn(0x7fb51c0fdfb0 msgr2=0x7fb51c06ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 shutdown_connections 2024-09-21T01:59:07.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T01:59:07.323+0000 7fb522d3d640 1 -- 172.21.15.136:0/1352185741 wait complete. 2024-09-21T01:59:07.964 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:07.965 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.965 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:07 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.966 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T01:59:07.968 INFO:tasks.cephadm:Deploying osd.6 on smithi092 with /dev/vg_nvme/lv_2... 2024-09-21T01:59:07.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:59:07.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:07.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:07.998 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:07 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:07 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:08.226 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:09.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:08 smithi136 ceph-mon[34754]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:09.030 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:08 smithi092 ceph-mon[31190]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:09.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:08 smithi099 ceph-mon[34404]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:10.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:09 smithi136 ceph-mon[34754]: from='osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:59:10.006 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:09 smithi136 ceph-mon[34754]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:59:10.006 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 01:59:09 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T01:59:09.624+0000 7eff9b4a8640 -1 osd.5 0 log_to_monitors true 2024-09-21T01:59:10.011 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:09 smithi092 ceph-mon[31190]: from='osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:59:10.012 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:09 smithi092 ceph-mon[31190]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:59:10.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:09 smithi099 ceph-mon[34404]: from='osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:59:10.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:09 smithi099 ceph-mon[34404]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:59:10.767 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T01:59:10.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-09-21T01:59:11.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:11.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:59:11.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T01:59:11.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:11.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:11.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:11.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:11.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:11.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:11.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:11.086 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 -- 172.21.15.92:0/260345220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124105220 msgr2=0x7f512410bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 --2- 172.21.15.92:0/260345220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124105220 0x7f512410bbc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f511c009ca0 tx=0x7f511c02a950 comp rx=0 tx=0).stop 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 -- 172.21.15.92:0/260345220 shutdown_connections 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 --2- 172.21.15.92:0/260345220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124105220 0x7f512410bbc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 --2- 172.21.15.92:0/260345220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5124104880 0x7f5124104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 --2- 172.21.15.92:0/260345220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51241037f0 0x7f5124103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 -- 172.21.15.92:0/260345220 >> 172.21.15.92:0/260345220 conn(0x7f51240ff180 msgr2=0x7f5124101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 -- 172.21.15.92:0/260345220 shutdown_connections 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.576+0000 7f512b595640 1 -- 172.21.15.92:0/260345220 wait complete. 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 Processor -- start 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 -- start start 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51241037f0 0x7f5124076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 0x7f5124075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5124105220 0x7f5124075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f512410f3d0 con 0x7f5124104880 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f512410f250 con 0x7f51241037f0 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f512b595640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f512410f550 con 0x7f5124105220 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 0x7f5124075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 0x7f5124075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48702/0 (socket says 172.21.15.92:48702) 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 -- 172.21.15.92:0/1230982398 learned_addr learned my addr 172.21.15.92:0/1230982398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5129331640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51241037f0 0x7f5124076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5129b32640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5124105220 0x7f5124075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 -- 172.21.15.92:0/1230982398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5124105220 msgr2=0x7f5124075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5124105220 0x7f5124075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 -- 172.21.15.92:0/1230982398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51241037f0 msgr2=0x7f5124076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51241037f0 0x7f5124076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.577+0000 7f5128b30640 1 -- 172.21.15.92:0/1230982398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5124076350 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.578+0000 7f5128b30640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 0x7f5124075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f511400d950 tx=0x7f511400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.578+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114017030 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.578+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5114004020 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.578+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140047a0 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.578+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51241a4090 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.578+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a4540 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.579+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512410a9d0 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.581+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f5114009a30 con 0x7f5124104880 2024-09-21T01:59:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.581+0000 7f511a7fc640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f51100b6590 0x7f51100b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.581+0000 7f5129331640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f51100b6590 0x7f51100b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.581+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4286+0+0 (secure 0 0 0) 0x7f51140e6340 con 0x7f5124104880 2024-09-21T01:59:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.582+0000 7f5129331640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f51100b6590 0x7f51100b8aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5120009b60 tx=0x7f5120005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.585+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51140adc10 con 0x7f5124104880 2024-09-21T01:59:11.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:11.772+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5124063800 con 0x7f51100b6590 2024-09-21T01:59:11.775 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T01:59:11.496+0000 7eff9742a640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T01:59:11.776 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T01:59:11.591+0000 7eff8f220640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:59:12.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi136 2024-09-21T01:59:12.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-21T01:59:12.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T01:59:12.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:59:12.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:59:12.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: from='osd.5 ' entity='osd.5' 2024-09-21T01:59:12.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi136 2024-09-21T01:59:12.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-21T01:59:12.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T01:59:12.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:59:12.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:59:12.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: from='osd.5 ' entity='osd.5' 2024-09-21T01:59:12.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi136 2024-09-21T01:59:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi136 to 8544M 2024-09-21T01:59:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T01:59:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:59:12.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:59:12.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:12.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: from='osd.5 ' entity='osd.5' 2024-09-21T01:59:12.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:59:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:59:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:59:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:59:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:59:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] boot 2024-09-21T01:59:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T01:59:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:13.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:59:13.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] boot 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T01:59:13.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:59:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:59:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:59:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:59:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:59:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:59:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: osd.5 [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] boot 2024-09-21T01:59:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T01:59:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:59:14.512 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:14 smithi092 ceph-mon[31190]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:14.513 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:14 smithi092 ceph-mon[31190]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T01:59:14.513 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:14 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/157683927' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d1d32dc-5414-40bf-824a-230ac53e834f"} : dispatch 2024-09-21T01:59:14.513 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:14 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/157683927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d1d32dc-5414-40bf-824a-230ac53e834f"}]': finished 2024-09-21T01:59:14.513 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:14 smithi092 ceph-mon[31190]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T01:59:14.513 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:14 smithi136 ceph-mon[34754]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:14 smithi136 ceph-mon[34754]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T01:59:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:14 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/157683927' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d1d32dc-5414-40bf-824a-230ac53e834f"} : dispatch 2024-09-21T01:59:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:14 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/157683927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d1d32dc-5414-40bf-824a-230ac53e834f"}]': finished 2024-09-21T01:59:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:14 smithi136 ceph-mon[34754]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T01:59:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:14 smithi099 ceph-mon[34404]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:14 smithi099 ceph-mon[34404]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T01:59:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:14 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/157683927' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d1d32dc-5414-40bf-824a-230ac53e834f"} : dispatch 2024-09-21T01:59:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:14 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/157683927' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d1d32dc-5414-40bf-824a-230ac53e834f"}]': finished 2024-09-21T01:59:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:14 smithi099 ceph-mon[34404]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T01:59:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:15.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:15 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/731706717' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:59:15.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:15 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/731706717' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:59:15.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:15 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/731706717' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:59:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:16 smithi099 ceph-mon[34404]: pgmap v150: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:16 smithi099 ceph-mon[34404]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T01:59:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:16 smithi136 ceph-mon[34754]: pgmap v150: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:16 smithi136 ceph-mon[34754]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T01:59:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:16 smithi092 ceph-mon[31190]: pgmap v150: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:16 smithi092 ceph-mon[31190]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T01:59:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:18 smithi099 ceph-mon[34404]: pgmap v151: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:18 smithi136 ceph-mon[34754]: pgmap v151: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:18 smithi092 ceph-mon[31190]: pgmap v151: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:20.516 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:20 smithi092 ceph-mon[31190]: pgmap v152: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:20.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:20 smithi099 ceph-mon[34404]: pgmap v152: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:20 smithi136 ceph-mon[34754]: pgmap v152: 1 pgs: 1 activating+degraded; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T01:59:21.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:21 smithi092 ceph-mon[31190]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T01:59:21.454 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:21 smithi092 ceph-mon[31190]: Cluster is now healthy 2024-09-21T01:59:21.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:21 smithi136 ceph-mon[34754]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T01:59:21.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:21 smithi136 ceph-mon[34754]: Cluster is now healthy 2024-09-21T01:59:21.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:21 smithi099 ceph-mon[34404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T01:59:21.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:21 smithi099 ceph-mon[34404]: Cluster is now healthy 2024-09-21T01:59:22.685 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:22 smithi136 ceph-mon[34754]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:59:22.733 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:22 smithi099 ceph-mon[34404]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:59:22.753 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:22 smithi092 ceph-mon[31190]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:59:23.507 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:59:23.507 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:23.507 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:23 smithi092 ceph-mon[31190]: Deploying daemon osd.6 on smithi092 2024-09-21T01:59:23.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:59:23.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:23.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:23 smithi136 ceph-mon[34754]: Deploying daemon osd.6 on smithi092 2024-09-21T01:59:23.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:59:23.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:23.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:23 smithi099 ceph-mon[34404]: Deploying daemon osd.6 on smithi092 2024-09-21T01:59:24.508 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:24 smithi092 ceph-mon[31190]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:59:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:24 smithi136 ceph-mon[34754]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:59:24.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:24 smithi099 ceph-mon[34404]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:59:26.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:26 smithi092 ceph-mon[31190]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T01:59:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:26 smithi136 ceph-mon[34754]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T01:59:26.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:26 smithi099 ceph-mon[34404]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T01:59:28.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:28 smithi092 ceph-mon[31190]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:28.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:28 smithi136 ceph-mon[34754]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:28 smithi099 ceph-mon[34404]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:29.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:29.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:29.661 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:29.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:29.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:29.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:29.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:29.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:29.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:30.435 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:30 smithi092 ceph-mon[31190]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:30.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:30 smithi136 ceph-mon[34754]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:30.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:30 smithi099 ceph-mon[34404]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:32.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.178+0000 7f511a7fc640 1 -- 172.21.15.92:0/1230982398 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5124063800 con 0x7f51100b6590 2024-09-21T01:59:32.179 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 6 on host 'smithi092' 2024-09-21T01:59:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f51100b6590 msgr2=0x7f51100b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f51100b6590 0x7f51100b8aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5120009b60 tx=0x7f5120005f90 comp rx=0 tx=0).stop 2024-09-21T01:59:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 msgr2=0x7f5124075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 0x7f5124075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f511400d950 tx=0x7f511400de10 comp rx=0 tx=0).stop 2024-09-21T01:59:32.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 shutdown_connections 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f51100b6590 0x7f51100b8aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5124105220 0x7f5124075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5124104880 0x7f5124075710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 --2- 172.21.15.92:0/1230982398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51241037f0 0x7f5124076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.181+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 >> 172.21.15.92:0/1230982398 conn(0x7f51240ff180 msgr2=0x7f5124107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.182+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 shutdown_connections 2024-09-21T01:59:32.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T01:59:32.182+0000 7f512b595640 1 -- 172.21.15.92:0/1230982398 wait complete. 2024-09-21T01:59:32.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:32 smithi092 ceph-mon[31190]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:32.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:32.474 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:32 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:32 smithi136 ceph-mon[34754]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:32 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:32.801 DEBUG:teuthology.orchestra.run.smithi092:osd.6> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T01:59:32.803 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2024-09-21T01:59:32.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:59:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:32 smithi099 ceph-mon[34404]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:59:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:32 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:33.059 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:59:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:34.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:34.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:34.309 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:34.309 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:35.163 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:34 smithi099 ceph-mon[34404]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:34 smithi136 ceph-mon[34754]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:34 smithi092 ceph-mon[31190]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:35.305 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 01:59:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T01:59:35.177+0000 7fd147022640 -1 osd.6 0 log_to_monitors true 2024-09-21T01:59:35.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T01:59:35.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-09-21T01:59:36.159 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:59:36.279 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:35 smithi099 ceph-mon[34404]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:59:36.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:35 smithi136 ceph-mon[34754]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:59:36.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:35 smithi092 ceph-mon[31190]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 -- 172.21.15.99:0/2115071531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 msgr2=0x7f092010aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 --2- 172.21.15.99:0/2115071531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092010aa10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0918009900 tx=0x7f091802a480 comp rx=0 tx=0).stop 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 -- 172.21.15.99:0/2115071531 shutdown_connections 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 --2- 172.21.15.99:0/2115071531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092010aa10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 --2- 172.21.15.99:0/2115071531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 0x7f0920103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 --2- 172.21.15.99:0/2115071531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0920102640 0x7f0920102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 -- 172.21.15.99:0/2115071531 >> 172.21.15.99:0/2115071531 conn(0x7f09200fdfb0 msgr2=0x7f0920100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 -- 172.21.15.99:0/2115071531 shutdown_connections 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.641+0000 7f0927e89640 1 -- 172.21.15.99:0/2115071531 wait complete. 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 Processor -- start 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 -- start start 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0920102640 0x7f0920199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 0x7f092019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f092010f430 con 0x7f09201036d0 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f092010f2b0 con 0x7f0920104070 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0927e89640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f092010f5b0 con 0x7f0920102640 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0926426640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0926426640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33168/0 (socket says 172.21.15.99:33168) 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0926426640 1 -- 172.21.15.99:0/3520923189 learned_addr learned my addr 172.21.15.99:0/3520923189 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0925424640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 0x7f092019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.642+0000 7f0925c25640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0920102640 0x7f0920199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:36.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0926426640 1 -- 172.21.15.99:0/3520923189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0920102640 msgr2=0x7f0920199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0926426640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0920102640 0x7f0920199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0926426640 1 -- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 msgr2=0x7f092019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0926426640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 0x7f092019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0926426640 1 -- 172.21.15.99:0/3520923189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092019b9f0 con 0x7f0920104070 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0925424640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 0x7f092019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0926426640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092019b1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f091802a9a0 tx=0x7f0918003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091803f070 con 0x7f0920104070 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0918031c50 con 0x7f0920104070 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092019bc80 con 0x7f0920104070 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918035030 con 0x7f0920104070 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.643+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09201a23e0 con 0x7f0920104070 2024-09-21T01:59:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.644+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0920109820 con 0x7f0920104070 2024-09-21T01:59:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.646+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0918042070 con 0x7f0920104070 2024-09-21T01:59:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.646+0000 7f091effd640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09080b6590 0x7f09080b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:59:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.646+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4669+0+0 (secure 0 0 0) 0x7f09181074a0 con 0x7f0920104070 2024-09-21T01:59:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.647+0000 7f0925c25640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09080b6590 0x7f09080b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:59:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.647+0000 7f0925c25640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09080b6590 0x7f09080b8aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0910009f20 tx=0x7f09100094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:59:36.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.649+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09180ceb70 con 0x7f0920104070 2024-09-21T01:59:36.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:36.837+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0920063800 con 0x7f09080b6590 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.212 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:59:37.213 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:37.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:59:37.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:36 smithi092 ceph-mon[31190]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:37.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:59:37.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:38.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi092 2024-09-21T01:59:38.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-21T01:59:38.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:59:38.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:59:38.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:59:38.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:38.168 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' 2024-09-21T01:59:38.168 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi092 2024-09-21T01:59:38.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-21T01:59:38.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:59:38.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:59:38.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:59:38.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:38.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' 2024-09-21T01:59:38.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.304 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T01:59:37.936+0000 7fd142fa4640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T01:59:38.304 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 01:59:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T01:59:37.986+0000 7fd13ad9a640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:59:38.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi092 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: from='osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183]' entity='osd.6' 2024-09-21T01:59:38.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:39 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T01:59:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:39 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T01:59:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:39 smithi099 ceph-mon[34404]: osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] boot 2024-09-21T01:59:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:39 smithi099 ceph-mon[34404]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T01:59:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:39 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T01:59:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:39 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T01:59:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:39 smithi136 ceph-mon[34754]: osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] boot 2024-09-21T01:59:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:39 smithi136 ceph-mon[34754]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T01:59:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:39 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T01:59:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:39 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T01:59:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:39 smithi092 ceph-mon[31190]: osd.6 [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] boot 2024-09-21T01:59:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:39 smithi092 ceph-mon[31190]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T01:59:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:59:40.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:40.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/375528545' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"} : dispatch 2024-09-21T01:59:40.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"} : dispatch 2024-09-21T01:59:40.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"}]': finished 2024-09-21T01:59:40.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: osdmap e42: 8 total, 7 up, 8 in 2024-09-21T01:59:40.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:59:40.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:40 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/1008863002' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/375528545' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"} : dispatch 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"} : dispatch 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"}]': finished 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: osdmap e42: 8 total, 7 up, 8 in 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:59:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:40 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/1008863002' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:59:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:59:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/375528545' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"} : dispatch 2024-09-21T01:59:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"} : dispatch 2024-09-21T01:59:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f02b02a-3345-49b2-aa55-bd16a66a1c7a"}]': finished 2024-09-21T01:59:40.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: osdmap e42: 8 total, 7 up, 8 in 2024-09-21T01:59:40.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:59:40.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:40 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/1008863002' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:59:41.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:41 smithi136 ceph-mon[34754]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T01:59:41.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:59:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:41 smithi092 ceph-mon[31190]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T01:59:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:59:41.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:41 smithi099 ceph-mon[34404]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T01:59:41.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:59:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:42 smithi136 ceph-mon[34754]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:42.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:42 smithi092 ceph-mon[31190]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:42 smithi099 ceph-mon[34404]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:44 smithi136 ceph-mon[34754]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:44 smithi092 ceph-mon[31190]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:44 smithi099 ceph-mon[34404]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:46.531 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:46 smithi099 ceph-mon[34404]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:46.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:46 smithi136 ceph-mon[34754]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:46.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:46 smithi092 ceph-mon[31190]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:48.490 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:48 smithi099 ceph-mon[34404]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:48.491 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:48 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:59:48.491 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:48 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:48.491 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:48 smithi099 ceph-mon[34404]: Deploying daemon osd.7 on smithi099 2024-09-21T01:59:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:48 smithi136 ceph-mon[34754]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:48 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:59:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:48 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:48.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:48 smithi136 ceph-mon[34754]: Deploying daemon osd.7 on smithi099 2024-09-21T01:59:48.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:48 smithi092 ceph-mon[31190]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:48.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:48 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:59:48.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:48 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:59:48.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:48 smithi092 ceph-mon[31190]: Deploying daemon osd.7 on smithi099 2024-09-21T01:59:50.500 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:50 smithi099 ceph-mon[34404]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:50.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:50 smithi136 ceph-mon[34754]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:50.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:50 smithi092 ceph-mon[31190]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:52.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:52 smithi136 ceph-mon[34754]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:52 smithi092 ceph-mon[31190]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:52 smithi099 ceph-mon[34404]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:54.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:54 smithi136 ceph-mon[34754]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:54.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:54.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:54.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:54 smithi092 ceph-mon[31190]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:54.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:54 smithi099 ceph-mon[34404]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:54.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:59:54.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:54.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:56.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:56 smithi099 ceph-mon[34404]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:56.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:56 smithi136 ceph-mon[34754]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:56.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:56 smithi092 ceph-mon[31190]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:57.113 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2024-09-21T01:59:57.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.112+0000 7f091effd640 1 -- 172.21.15.99:0/3520923189 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0920063800 con 0x7f09080b6590 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09080b6590 msgr2=0x7f09080b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09080b6590 0x7f09080b8aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0910009f20 tx=0x7f09100094f0 comp rx=0 tx=0).stop 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 msgr2=0x7f092019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092019b1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f091802a9a0 tx=0x7f0918003b90 comp rx=0 tx=0).stop 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 shutdown_connections 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09080b6590 0x7f09080b8aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:57.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0920104070 0x7f092019b1d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:57.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09201036d0 0x7f092019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:57.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 --2- 172.21.15.99:0/3520923189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0920102640 0x7f0920199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:59:57.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 >> 172.21.15.99:0/3520923189 conn(0x7f09200fdfb0 msgr2=0x7f09201069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:59:57.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 shutdown_connections 2024-09-21T01:59:57.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T01:59:57.114+0000 7f0927e89640 1 -- 172.21.15.99:0/3520923189 wait complete. 2024-09-21T01:59:57.779 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T01:59:57.781 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-09-21T01:59:57.781 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:59:58.035 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:59:58.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:58 smithi099 ceph-mon[34404]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:58.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 01:59:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:58 smithi136 ceph-mon[34754]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:58.358 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.358 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.358 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.358 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 01:59:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:58 smithi092 ceph-mon[31190]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:59:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T01:59:58.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 01:59:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:00.081 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 01:59:59 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T01:59:59.708+0000 7f4cacdbf640 -1 osd.7 0 log_to_monitors true 2024-09-21T02:00:00.374 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:00 smithi136 ceph-mon[34754]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:00.374 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:00 smithi136 ceph-mon[34754]: from='osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:00:00.374 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:00 smithi136 ceph-mon[34754]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:00:00.374 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:00 smithi136 ceph-mon[34754]: overall HEALTH_OK 2024-09-21T02:00:00.376 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T02:00:00.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-09-21T02:00:00.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:00 smithi099 ceph-mon[34404]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:00.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:00 smithi099 ceph-mon[34404]: from='osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:00:00.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:00 smithi099 ceph-mon[34404]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:00:00.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:00 smithi099 ceph-mon[34404]: overall HEALTH_OK 2024-09-21T02:00:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:00 smithi092 ceph-mon[31190]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:00 smithi092 ceph-mon[31190]: from='osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:00:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:00 smithi092 ceph-mon[31190]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:00:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:00 smithi092 ceph-mon[31190]: overall HEALTH_OK 2024-09-21T02:00:00.667 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:00:01.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 -- 172.21.15.136:0/3196331485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 msgr2=0x7f8764102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:01.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 --2- 172.21.15.136:0/3196331485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f8764102ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8760009900 tx=0x7f876002a480 comp rx=0 tx=0).stop 2024-09-21T02:00:01.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 -- 172.21.15.136:0/3196331485 shutdown_connections 2024-09-21T02:00:01.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 --2- 172.21.15.136:0/3196331485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87641040a0 0x7f876410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:01.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 --2- 172.21.15.136:0/3196331485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8764103700 0x7f8764103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:01.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 --2- 172.21.15.136:0/3196331485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f8764102ae0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:01.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 -- 172.21.15.136:0/3196331485 >> 172.21.15.136:0/3196331485 conn(0x7f87640fdfe0 msgr2=0x7f8764100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:01.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 -- 172.21.15.136:0/3196331485 shutdown_connections 2024-09-21T02:00:01.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.105+0000 7f876b49e640 1 -- 172.21.15.136:0/3196331485 wait complete. 2024-09-21T02:00:01.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 Processor -- start 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 -- start start 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f876419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8764103700 0x7f876419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87641040a0 0x7f876419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f876410e2b0 con 0x7f8764103700 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f876410e130 con 0x7f87641040a0 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.106+0000 7f876b49e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f876410e430 con 0x7f8764102670 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f876419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f876419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:57896/0 (socket says 172.21.15.136:57896) 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 -- 172.21.15.136:0/2740869376 learned_addr learned my addr 172.21.15.136:0/2740869376 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f8769a3b640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87641040a0 0x7f876419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f8768a39640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8764103700 0x7f876419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 -- 172.21.15.136:0/2740869376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87641040a0 msgr2=0x7f876419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:01.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87641040a0 0x7f876419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 -- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8764103700 msgr2=0x7f876419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8764103700 0x7f876419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 -- 172.21.15.136:0/2740869376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876419ba40 con 0x7f8764102670 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876923a640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f876419a010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8760006010 tx=0x7f87600042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876003f030 con 0x7f8764102670 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8760031910 con 0x7f8764102670 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876419bcd0 con 0x7f8764102670 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760035070 con 0x7f8764102670 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.107+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a2430 con 0x7f8764102670 2024-09-21T02:00:01.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.108+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764102ae0 con 0x7f8764102670 2024-09-21T02:00:01.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.110+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8760042030 con 0x7f8764102670 2024-09-21T02:00:01.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.111+0000 7f875a7fc640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f87500b6590 0x7f87500b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:01.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.111+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5101+0+0 (secure 0 0 0) 0x7f8760107270 con 0x7f8764102670 2024-09-21T02:00:01.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.111+0000 7f8768a39640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f87500b6590 0x7f87500b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:01.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.112+0000 7f8768a39640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f87500b6590 0x7f87500b8aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8754009b60 tx=0x7f8754005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:01.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.115+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87600ce730 con 0x7f8764102670 2024-09-21T02:00:01.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:01.311+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f87641a2600 con 0x7f87500b6590 2024-09-21T02:00:01.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi099 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-21T02:00:01.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:01.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi099 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:01.556 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:00:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T02:00:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi099 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:01.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:01.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:01.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:01.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:01.583 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T02:00:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:00:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:02 smithi136 ceph-mon[34754]: from='osd.7 ' entity='osd.7' 2024-09-21T02:00:02.331 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:00:02.034+0000 7f4ca8d41640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T02:00:02.331 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:00:02.185+0000 7f4ca0b37640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:00:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T02:00:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:00:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:02 smithi099 ceph-mon[34404]: from='osd.7 ' entity='osd.7' 2024-09-21T02:00:02.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:02.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T02:00:02.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:02.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:02 smithi092 ceph-mon[31190]: from='osd.7 ' entity='osd.7' 2024-09-21T02:00:03.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:03 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T02:00:03.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:03 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T02:00:03.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:03.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:03 smithi136 ceph-mon[34754]: osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] boot 2024-09-21T02:00:03.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:03 smithi136 ceph-mon[34754]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T02:00:03.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:03 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T02:00:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:03 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T02:00:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:03 smithi099 ceph-mon[34404]: osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] boot 2024-09-21T02:00:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:03 smithi099 ceph-mon[34404]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T02:00:03.393 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:03.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:03 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T02:00:03.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:03 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T02:00:03.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:03.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:03 smithi092 ceph-mon[31190]: osd.7 [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] boot 2024-09-21T02:00:03.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:03 smithi092 ceph-mon[31190]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T02:00:03.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:00:04.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:00:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/2937304484' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"} : dispatch 2024-09-21T02:00:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"} : dispatch 2024-09-21T02:00:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"}]': finished 2024-09-21T02:00:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T02:00:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:04.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:04.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:00:04.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/2937304484' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"} : dispatch 2024-09-21T02:00:04.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"} : dispatch 2024-09-21T02:00:04.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"}]': finished 2024-09-21T02:00:04.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T02:00:04.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:00:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:00:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/2937304484' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"} : dispatch 2024-09-21T02:00:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"} : dispatch 2024-09-21T02:00:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "813f34cb-ed23-4d4e-b578-0f9e906c8d34"}]': finished 2024-09-21T02:00:04.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T02:00:04.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:05.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/881488150' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:05.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:05 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/881488150' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:05.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:05 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/881488150' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:06 smithi092 ceph-mon[31190]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:06.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:06 smithi099 ceph-mon[34404]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:06 smithi136 ceph-mon[34754]: pgmap v186: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:08.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:08 smithi092 ceph-mon[31190]: pgmap v187: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:08.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:08 smithi099 ceph-mon[34404]: pgmap v187: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:08.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:08 smithi136 ceph-mon[34754]: pgmap v187: 1 pgs: 1 remapped; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:10.489 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:10 smithi136 ceph-mon[34754]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T02:00:10.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:10 smithi092 ceph-mon[31190]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T02:00:10.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:10 smithi099 ceph-mon[34404]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T02:00:12.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:12 smithi099 ceph-mon[34404]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T02:00:12.728 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:12 smithi136 ceph-mon[34754]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T02:00:12.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:12 smithi092 ceph-mon[31190]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T02:00:13.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:13 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:00:13.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:13 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:13.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:13 smithi099 ceph-mon[34404]: Deploying daemon osd.8 on smithi136 2024-09-21T02:00:13.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:13 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:00:13.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:13 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:13.648 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:13 smithi136 ceph-mon[34754]: Deploying daemon osd.8 on smithi136 2024-09-21T02:00:13.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:13 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:00:13.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:13 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:13.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:13 smithi092 ceph-mon[31190]: Deploying daemon osd.8 on smithi136 2024-09-21T02:00:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:14 smithi136 ceph-mon[34754]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-21T02:00:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:14 smithi099 ceph-mon[34404]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-21T02:00:14.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:14 smithi092 ceph-mon[31190]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-21T02:00:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:16 smithi099 ceph-mon[34404]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-21T02:00:16.684 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:16 smithi136 ceph-mon[34754]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-21T02:00:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:16 smithi092 ceph-mon[31190]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-21T02:00:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:18 smithi099 ceph-mon[34404]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:00:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:18.660 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:18 smithi136 ceph-mon[34754]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:00:18.661 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:18.661 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:18.661 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:18 smithi092 ceph-mon[31190]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:00:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:20.447 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:20 smithi136 ceph-mon[34754]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:00:20.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:20 smithi099 ceph-mon[34404]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:00:20.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:20 smithi092 ceph-mon[31190]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:00:21.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:21.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:21.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:21.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:21.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:21.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.741+0000 7f875a7fc640 1 -- 172.21.15.136:0/2740869376 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87641a2600 con 0x7f87500b6590 2024-09-21T02:00:21.743 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-09-21T02:00:21.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f87500b6590 msgr2=0x7f87500b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f87500b6590 0x7f87500b8aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8754009b60 tx=0x7f8754005f90 comp rx=0 tx=0).stop 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 msgr2=0x7f876419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f876419a010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8760006010 tx=0x7f87600042f0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 shutdown_connections 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f87500b6590 0x7f87500b8aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87641040a0 0x7f876419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8764103700 0x7f876419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 --2- 172.21.15.136:0/2740869376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764102670 0x7f876419a010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 >> 172.21.15.136:0/2740869376 conn(0x7f87640fdfe0 msgr2=0x7f87640ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 shutdown_connections 2024-09-21T02:00:21.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:00:21.744+0000 7f876b49e640 1 -- 172.21.15.136:0/2740869376 wait complete. 2024-09-21T02:00:22.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:22.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:22.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:22.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.374 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T02:00:22.376 INFO:tasks.cephadm:Deploying osd.9 on smithi092 with /dev/vg_nvme/lv_1... 2024-09-21T02:00:22.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:00:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:22.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:22 smithi092 ceph-mon[31190]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:22.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:22.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:22.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:23.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:22 smithi136 ceph-mon[34754]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:23.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:23.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:23.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:23.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:22 smithi099 ceph-mon[34404]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:23.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:23.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:23.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:24.016 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:00:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:00:23.974+0000 7ff5dc76d640 -1 osd.8 0 log_to_monitors true 2024-09-21T02:00:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:24 smithi136 ceph-mon[34754]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:24 smithi136 ceph-mon[34754]: from='osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:00:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:24 smithi136 ceph-mon[34754]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:00:24.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:24 smithi092 ceph-mon[31190]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:24.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:24 smithi092 ceph-mon[31190]: from='osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:00:24.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:24 smithi092 ceph-mon[31190]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:00:25.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:24 smithi099 ceph-mon[34404]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:25.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:24 smithi099 ceph-mon[34404]: from='osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:00:25.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:24 smithi099 ceph-mon[34404]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:00:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:00:25.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-09-21T02:00:25.619 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:25.994 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:25.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:25.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:25.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:25.995 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:00:26.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T02:00:26.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:26.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:26.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:00:26.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T02:00:26.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:26.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:00:26.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:26.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 -- 172.21.15.92:0/3902244027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 msgr2=0x7f1f0c10b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 --2- 172.21.15.92:0/3902244027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 0x7f1f0c10b9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009ca0 tx=0x7f1f0402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 -- 172.21.15.92:0/3902244027 shutdown_connections 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 --2- 172.21.15.92:0/3902244027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 0x7f1f0c10b9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 --2- 172.21.15.92:0/3902244027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f0c1025d0 0x7f1f0c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 --2- 172.21.15.92:0/3902244027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 -- 172.21.15.92:0/3902244027 >> 172.21.15.92:0/3902244027 conn(0x7f1f0c0fd770 msgr2=0x7f1f0c0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 -- 172.21.15.92:0/3902244027 shutdown_connections 2024-09-21T02:00:26.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 -- 172.21.15.92:0/3902244027 wait complete. 2024-09-21T02:00:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 Processor -- start 2024-09-21T02:00:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.087+0000 7f1f11766640 1 -- start start 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f11766640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f11766640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f0c1025d0 0x7f1f0c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f11766640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 0x7f1f0c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f11766640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c10f3d0 con 0x7f1f0c1094b0 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f11766640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c10f250 con 0x7f1f0c1025d0 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f11766640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c10f550 con 0x7f1f0c101ba0 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 0x7f1f0c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f0c1025d0 0x7f1f0c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:57722/0 (socket says 172.21.15.92:57722) 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 -- 172.21.15.92:0/3392425944 learned_addr learned my addr 172.21.15.92:0/3392425944 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 -- 172.21.15.92:0/3392425944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f0c1025d0 msgr2=0x7f1f0c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f0c1025d0 0x7f1f0c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 -- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 msgr2=0x7f1f0c19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 0x7f1f0c19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 -- 172.21.15.92:0/3392425944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c19b9c0 con 0x7f1f0c101ba0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.088+0000 7f1f0affd640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c199f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e9f0 tx=0x7f1efc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.089+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00cd80 con 0x7f1f0c101ba0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.089+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1efc0054d0 con 0x7f1f0c101ba0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.089+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c19bcb0 con 0x7f1f0c101ba0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.089+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc010690 con 0x7f1f0c101ba0 2024-09-21T02:00:26.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.089+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c1a23b0 con 0x7f1f0c101ba0 2024-09-21T02:00:26.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.090+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c101fb0 con 0x7f1f0c101ba0 2024-09-21T02:00:26.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.092+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f1efc004e30 con 0x7f1f0c101ba0 2024-09-21T02:00:26.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.092+0000 7f1f08ff9640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1ef80b6590 0x7f1ef80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:26.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.093+0000 7f1f03fff640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1ef80b6590 0x7f1ef80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:26.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.093+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5549+0+0 (secure 0 0 0) 0x7f1efc0e6e00 con 0x7f1f0c101ba0 2024-09-21T02:00:26.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.093+0000 7f1f03fff640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1ef80b6590 0x7f1ef80b8aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40058c0 tx=0x7f1ef400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:26.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.096+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1efc0ae0c0 con 0x7f1f0c101ba0 2024-09-21T02:00:26.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:26.293+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1f0c063800 con 0x7f1ef80b6590 2024-09-21T02:00:26.672 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:00:26.403+0000 7ff5d86ef640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T02:00:26.672 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:00:26.469+0000 7ff5d04e5640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:00:26.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:26.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi136 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:26.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:26 smithi092 ceph-mon[31190]: from='osd.8 ' entity='osd.8' 2024-09-21T02:00:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi136 2024-09-21T02:00:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-21T02:00:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T02:00:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:27.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:26 smithi136 ceph-mon[34754]: from='osd.8 ' entity='osd.8' 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi136 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi136 to 5696M 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:00:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:27.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:27.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:27.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:26 smithi099 ceph-mon[34404]: from='osd.8 ' entity='osd.8' 2024-09-21T02:00:27.932 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T02:00:27.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T02:00:27.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:27.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:27.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] boot 2024-09-21T02:00:27.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T02:00:27.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:28.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T02:00:28.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T02:00:28.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:28.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:28.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] boot 2024-09-21T02:00:28.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T02:00:28.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: from='client.24466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: osd.8 [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] boot 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T02:00:28.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:00:28.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:28 smithi092 ceph-mon[31190]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:28.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:28 smithi092 ceph-mon[31190]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:00:29.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:28 smithi136 ceph-mon[34754]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:29.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:28 smithi136 ceph-mon[34754]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:00:29.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:28 smithi099 ceph-mon[34404]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:00:29.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:28 smithi099 ceph-mon[34404]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:00:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:29 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3868093749' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "574cf4c9-c764-453d-9661-a05a8f62b827"} : dispatch 2024-09-21T02:00:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:29 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3868093749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574cf4c9-c764-453d-9661-a05a8f62b827"}]': finished 2024-09-21T02:00:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:29 smithi136 ceph-mon[34754]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T02:00:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:29 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/482691218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:30.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:29 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3868093749' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "574cf4c9-c764-453d-9661-a05a8f62b827"} : dispatch 2024-09-21T02:00:30.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:29 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3868093749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574cf4c9-c764-453d-9661-a05a8f62b827"}]': finished 2024-09-21T02:00:30.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:29 smithi092 ceph-mon[31190]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T02:00:30.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:30.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:29 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/482691218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:29 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3868093749' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "574cf4c9-c764-453d-9661-a05a8f62b827"} : dispatch 2024-09-21T02:00:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:29 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3868093749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574cf4c9-c764-453d-9661-a05a8f62b827"}]': finished 2024-09-21T02:00:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:29 smithi099 ceph-mon[34404]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T02:00:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:29 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/482691218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:31.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:30 smithi136 ceph-mon[34754]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:31.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:30 smithi092 ceph-mon[31190]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:31.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:30 smithi099 ceph-mon[34404]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:32 smithi136 ceph-mon[34754]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:33.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:32 smithi092 ceph-mon[31190]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:33.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:32 smithi099 ceph-mon[34404]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:35.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:34 smithi136 ceph-mon[34754]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:35.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:34 smithi092 ceph-mon[31190]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:35.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:34 smithi099 ceph-mon[34404]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:36 smithi136 ceph-mon[34754]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-21T02:00:37.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:36 smithi092 ceph-mon[31190]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-21T02:00:37.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:36 smithi099 ceph-mon[34404]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-21T02:00:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:00:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:00:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:38.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:00:38.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:39.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:38 smithi092 ceph-mon[31190]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T02:00:39.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:38 smithi092 ceph-mon[31190]: Deploying daemon osd.9 on smithi092 2024-09-21T02:00:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:38 smithi099 ceph-mon[34404]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T02:00:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:38 smithi099 ceph-mon[34404]: Deploying daemon osd.9 on smithi092 2024-09-21T02:00:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:38 smithi136 ceph-mon[34754]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T02:00:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:38 smithi136 ceph-mon[34754]: Deploying daemon osd.9 on smithi092 2024-09-21T02:00:41.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:40 smithi092 ceph-mon[31190]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T02:00:41.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:40 smithi099 ceph-mon[34404]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T02:00:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:40 smithi136 ceph-mon[34754]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T02:00:42.981 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:42 smithi092 ceph-mon[31190]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:43.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:42 smithi099 ceph-mon[34404]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:42 smithi136 ceph-mon[34754]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:44.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:44.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:44.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:44.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:44.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:44.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:45.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:44 smithi092 ceph-mon[31190]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:45.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:44 smithi099 ceph-mon[34404]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:44 smithi136 ceph-mon[34754]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:47.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:46 smithi092 ceph-mon[31190]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:47.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:47.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:47.025 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.069+0000 7f1f08ff9640 1 -- 172.21.15.92:0/3392425944 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1f0c063800 con 0x7f1ef80b6590 2024-09-21T02:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 9 on host 'smithi092' 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1ef80b6590 msgr2=0x7f1ef80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1ef80b6590 0x7f1ef80b8aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40058c0 tx=0x7f1ef400a2d0 comp rx=0 tx=0).stop 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 msgr2=0x7f1f0c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c199f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e9f0 tx=0x7f1efc00edf0 comp rx=0 tx=0).stop 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 shutdown_connections 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1ef80b6590 0x7f1ef80b8aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f0c1094b0 0x7f1f0c19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f0c1025d0 0x7f1f0c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 --2- 172.21.15.92:0/3392425944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f0c101ba0 0x7f1f0c199f90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 >> 172.21.15.92:0/3392425944 conn(0x7f1f0c0fd770 msgr2=0x7f1f0c1077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 shutdown_connections 2024-09-21T02:00:47.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:00:47.073+0000 7f1f11766640 1 -- 172.21.15.92:0/3392425944 wait complete. 2024-09-21T02:00:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:46 smithi099 ceph-mon[34404]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:46 smithi136 ceph-mon[34754]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:00:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:47.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:47.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.679 DEBUG:teuthology.orchestra.run.smithi092:osd.9> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T02:00:47.682 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2024-09-21T02:00:47.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:00:47.912 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:47.912 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.912 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:47.938 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:00:48.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:48.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:48.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:48.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:00:48.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:48.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:49.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:48 smithi099 ceph-mon[34404]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:49.121 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:48 smithi092 ceph-mon[31190]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:48 smithi136 ceph-mon[34754]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:49.805 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:00:49 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:00:49.408+0000 7f1bb48c2640 -1 osd.9 0 log_to_monitors true 2024-09-21T02:00:50.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:49 smithi099 ceph-mon[34404]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:00:50.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:49 smithi136 ceph-mon[34754]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:00:50.291 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:49 smithi092 ceph-mon[31190]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:00:50.771 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-21T02:00:50.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-09-21T02:00:51.057 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:51.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:50 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:00:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T02:00:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T02:00:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:50 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:00:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T02:00:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-09-21T02:00:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:00:51.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:50 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:00:51.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 -- 172.21.15.99:0/1263245980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 msgr2=0x7f8d04102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:51.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 --2- 172.21.15.99:0/1263245980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d04102ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d0002a480 comp rx=0 tx=0).stop 2024-09-21T02:00:51.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 -- 172.21.15.99:0/1263245980 shutdown_connections 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 --2- 172.21.15.99:0/1263245980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d04104070 0x7f8d0410aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 --2- 172.21.15.99:0/1263245980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d041036d0 0x7f8d04103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 --2- 172.21.15.99:0/1263245980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d04102ab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 -- 172.21.15.99:0/1263245980 >> 172.21.15.99:0/1263245980 conn(0x7f8d040fdfb0 msgr2=0x7f8d04100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 -- 172.21.15.99:0/1263245980 shutdown_connections 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.506+0000 7f8d0c664640 1 -- 172.21.15.99:0/1263245980 wait complete. 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 Processor -- start 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 -- start start 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d0406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d041036d0 0x7f8d0419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d04104070 0x7f8d041a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410e280 con 0x7f8d041036d0 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410e100 con 0x7f8d04102640 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0c664640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410e400 con 0x7f8d04104070 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0a400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d0406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0a400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d0406b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40432/0 (socket says 172.21.15.99:40432) 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d0a400640 1 -- 172.21.15.99:0/4094416619 learned_addr learned my addr 172.21.15.99:0/4094416619 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.507+0000 7f8d09bff640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d041036d0 0x7f8d0419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0ac01640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d04104070 0x7f8d041a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0a400640 1 -- 172.21.15.99:0/4094416619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d04104070 msgr2=0x7f8d041a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0a400640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d04104070 0x7f8d041a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0a400640 1 -- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d041036d0 msgr2=0x7f8d0419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0a400640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d041036d0 0x7f8d0419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:00:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0a400640 1 -- 172.21.15.99:0/4094416619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d041a1a90 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0ac01640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d04104070 0x7f8d041a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0a400640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d0406b6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d00004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0003f030 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d00003cb0 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d041a1d20 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00035070 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.508+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041a21a0 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.509+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d04109820 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.511+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8d00042030 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.511+0000 7f8cfb7fe640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8cf00b6590 0x7f8cf00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.511+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5981+0+0 (secure 0 0 0) 0x7f8d00107020 con 0x7f8d04102640 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.511+0000 7f8d09bff640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8cf00b6590 0x7f8cf00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:00:51.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.512+0000 7f8d09bff640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8cf00b6590 0x7f8cf00b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4005fd0 tx=0x7f8cf400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:00:51.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.516+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d000ce1d0 con 0x7f8d04102640 2024-09-21T02:00:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:00:51.705+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8d04063800 con 0x7f8cf00b6590 2024-09-21T02:00:52.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:00:51.738+0000 7f1bb1059640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T02:00:52.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:00:51.835+0000 7f1ba863a640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi092 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' 2024-09-21T02:00:52.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:51 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi092 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:52.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:52.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' 2024-09-21T02:00:52.186 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:51 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.221 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi092 2024-09-21T02:00:52.221 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-21T02:00:52.221 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979]' entity='osd.9' 2024-09-21T02:00:52.222 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:51 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] boot 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T02:00:53.206 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:53.220 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T02:00:53.220 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T02:00:53.220 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:53.220 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:53.221 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] boot 2024-09-21T02:00:53.221 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T02:00:53.221 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T02:00:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T02:00:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:00:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:00:53.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: osd.9 [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] boot 2024-09-21T02:00:53.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T02:00:53.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/3411772359' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"} : dispatch 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"} : dispatch 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"}]': finished 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:00:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:54 smithi136 ceph-mon[34754]: from='client.? 172.21.15.99:0/869998345' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T02:00:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/3411772359' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"} : dispatch 2024-09-21T02:00:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"} : dispatch 2024-09-21T02:00:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"}]': finished 2024-09-21T02:00:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T02:00:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:00:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:54 smithi092 ceph-mon[31190]: from='client.? 172.21.15.99:0/869998345' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/3411772359' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"} : dispatch 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"} : dispatch 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "769d723d-f839-4eca-8b28-0336cdb68063"}]': finished 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:00:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.99:0/869998345' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:00:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:56 smithi136 ceph-mon[34754]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:57.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:56 smithi092 ceph-mon[31190]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:56 smithi099 ceph-mon[34404]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:00:58 smithi136 ceph-mon[34754]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:59.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:00:58 smithi092 ceph-mon[31190]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:00:59.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:00:58 smithi099 ceph-mon[34404]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:01.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:00 smithi099 ceph-mon[34404]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:01.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:00 smithi136 ceph-mon[34754]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:00 smithi092 ceph-mon[31190]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:02 smithi136 ceph-mon[34754]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:03.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:02 smithi092 ceph-mon[31190]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:03.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:02 smithi099 ceph-mon[34404]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:04.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:01:04.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:03 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:04.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:03 smithi099 ceph-mon[34404]: Deploying daemon osd.10 on smithi099 2024-09-21T02:01:04.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:01:04.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:03 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:04.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:03 smithi136 ceph-mon[34754]: Deploying daemon osd.10 on smithi099 2024-09-21T02:01:04.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:01:04.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:03 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:04.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:03 smithi092 ceph-mon[31190]: Deploying daemon osd.10 on smithi099 2024-09-21T02:01:05.240 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:04 smithi099 ceph-mon[34404]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:04 smithi136 ceph-mon[34754]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:05.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:04 smithi092 ceph-mon[31190]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:07.219 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:06 smithi099 ceph-mon[34404]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:06 smithi136 ceph-mon[34754]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:07.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:06 smithi092 ceph-mon[31190]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:09.216 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:08 smithi099 ceph-mon[34404]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:09.216 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:09.216 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:09.216 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:08 smithi136 ceph-mon[34754]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:08 smithi092 ceph-mon[31190]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:10 smithi099 ceph-mon[34404]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:10 smithi136 ceph-mon[34754]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:10 smithi092 ceph-mon[31190]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:12.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.116+0000 7f8cfb7fe640 1 -- 172.21.15.99:0/4094416619 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8d04063800 con 0x7f8cf00b6590 2024-09-21T02:01:12.117 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2024-09-21T02:01:12.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8cf00b6590 msgr2=0x7f8cf00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:12.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8cf00b6590 0x7f8cf00b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4005fd0 tx=0x7f8cf400b390 comp rx=0 tx=0).stop 2024-09-21T02:01:12.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 msgr2=0x7f8d0406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:12.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d0406b6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d00004340 comp rx=0 tx=0).stop 2024-09-21T02:01:12.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 shutdown_connections 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8cf00b6590 0x7f8cf00b8aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d04104070 0x7f8d041a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d041036d0 0x7f8d0419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.118+0000 7f8d0c664640 1 --2- 172.21.15.99:0/4094416619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d04102640 0x7f8d0406b6c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.119+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 >> 172.21.15.99:0/4094416619 conn(0x7f8d040fdfb0 msgr2=0x7f8d041069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.119+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 shutdown_connections 2024-09-21T02:01:12.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-21T02:01:12.119+0000 7f8d0c664640 1 -- 172.21.15.99:0/4094416619 wait complete. 2024-09-21T02:01:12.746 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T02:01:12.748 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-09-21T02:01:12.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:01:13.009 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:01:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:12 smithi136 ceph-mon[34754]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:12 smithi099 ceph-mon[34404]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:12 smithi092 ceph-mon[31190]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:15.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:14 smithi099 ceph-mon[34404]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:15.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:14 smithi136 ceph-mon[34754]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:14 smithi092 ceph-mon[31190]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:15.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-21T02:01:15.548 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-09-21T02:01:15.581 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:01:15 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:01:15.180+0000 7f1066479640 -1 osd.10 0 log_to_monitors true 2024-09-21T02:01:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:01:16.085 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:15 smithi136 ceph-mon[34754]: from='osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:01:16.085 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:15 smithi136 ceph-mon[34754]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:01:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2378054116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a48102640 msgr2=0x7f8a48102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2378054116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a48102640 0x7f8a48102ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009900 tx=0x7f8a3802a480 comp rx=0 tx=0).stop 2024-09-21T02:01:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2378054116 shutdown_connections 2024-09-21T02:01:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2378054116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 0x7f8a4810aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2378054116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481036d0 0x7f8a48103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2378054116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a48102640 0x7f8a48102ab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2378054116 >> 172.21.15.136:0/2378054116 conn(0x7f8a480fdfb0 msgr2=0x7f8a48100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2378054116 shutdown_connections 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2378054116 wait complete. 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 Processor -- start 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.287+0000 7f8a4e7dd640 1 -- start start 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4e7dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a48102640 0x7f8a48199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4e7dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 0x7f8a4819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4e7dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 0x7f8a4819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4e7dd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810e280 con 0x7f8a48104070 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4e7dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810e100 con 0x7f8a48102640 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4e7dd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810e400 con 0x7f8a481036d0 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 0x7f8a4819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 0x7f8a4819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:43734/0 (socket says 172.21.15.136:43734) 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 -- 172.21.15.136:0/2016337712 learned_addr learned my addr 172.21.15.136:0/2016337712 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a47fff640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a48102640 0x7f8a48199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4cd7a640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 0x7f8a4819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 -- 172.21.15.136:0/2016337712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a48102640 msgr2=0x7f8a48199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a48102640 0x7f8a48199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 -- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 msgr2=0x7f8a4819b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 0x7f8a4819b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 -- 172.21.15.136:0/2016337712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4819b970 con 0x7f8a481036d0 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a47fff640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a48102640 0x7f8a48199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a4cd7a640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 0x7f8a4819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.288+0000 7f8a477fe640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 0x7f8a4819a640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9f0 tx=0x7f8a3c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.289+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00cd80 con 0x7f8a481036d0 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.289+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4819bc60 con 0x7f8a481036d0 2024-09-21T02:01:16.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.289+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0054d0 con 0x7f8a481036d0 2024-09-21T02:01:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.289+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c010690 con 0x7f8a481036d0 2024-09-21T02:01:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.289+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a481a2220 con 0x7f8a481036d0 2024-09-21T02:01:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.290+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a481a24b0 con 0x7f8a481036d0 2024-09-21T02:01:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.292+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8a3c004e30 con 0x7f8a481036d0 2024-09-21T02:01:16.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.292+0000 7f8a457fa640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8a300b6510 0x7f8a300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:16.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.292+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6398+0+0 (secure 0 0 0) 0x7f8a3c0e71d0 con 0x7f8a481036d0 2024-09-21T02:01:16.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.293+0000 7f8a47fff640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8a300b6510 0x7f8a300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:16.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.293+0000 7f8a47fff640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8a300b6510 0x7f8a300b8a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c00 tx=0x7f8a38031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:16.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.295+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a3c056710 con 0x7f8a481036d0 2024-09-21T02:01:16.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:15 smithi092 ceph-mon[31190]: from='osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:01:16.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:15 smithi092 ceph-mon[31190]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:01:16.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:15 smithi099 ceph-mon[34404]: from='osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:01:16.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:15 smithi099 ceph-mon[34404]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:01:16.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:16.489+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8a480ffff0 con 0x7f8a300b6510 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi099 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.111 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.112 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:01:17.113 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:16 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi099 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:17.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:01:17.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:01:17.307 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:16 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi099 2024-09-21T02:01:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:01:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T02:01:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:01:17.333 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:16 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:18.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:17 smithi136 ceph-mon[34754]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:01:18.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:17 smithi136 ceph-mon[34754]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T02:01:18.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:17 smithi136 ceph-mon[34754]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:01:18.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:18.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:18.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:17 smithi136 ceph-mon[34754]: from='osd.10 ' entity='osd.10' 2024-09-21T02:01:18.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:17 smithi092 ceph-mon[31190]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:01:18.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:17 smithi092 ceph-mon[31190]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T02:01:18.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:17 smithi092 ceph-mon[31190]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:01:18.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:18.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:18.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:17 smithi092 ceph-mon[31190]: from='osd.10 ' entity='osd.10' 2024-09-21T02:01:18.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:01:17.888+0000 7f1062c10640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T02:01:18.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:01:18.040+0000 7f105a1f1640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:01:18.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-mon[34404]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:01:18.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-mon[34404]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-21T02:01:18.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-mon[34404]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:01:18.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:18.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:18.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:17 smithi099 ceph-mon[34404]: from='osd.10 ' entity='osd.10' 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/3103804482' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"} : dispatch 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"} : dispatch 2024-09-21T02:01:19.234 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"}]': finished 2024-09-21T02:01:19.235 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] boot 2024-09-21T02:01:19.235 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T02:01:19.235 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:19.235 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:18 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/3103804482' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"} : dispatch 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"} : dispatch 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"}]': finished 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] boot 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:19.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:18 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/3103804482' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"} : dispatch 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"} : dispatch 2024-09-21T02:01:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bccf2012-84f1-4ce9-a21d-9016f7d5cce6"}]': finished 2024-09-21T02:01:19.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: osd.10 [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] boot 2024-09-21T02:01:19.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T02:01:19.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:01:19.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:18 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:20.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:20 smithi092 ceph-mon[31190]: from='client.? 172.21.15.136:0/3913542541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:01:20.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:20 smithi092 ceph-mon[31190]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T02:01:20.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:20 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:20 smithi099 ceph-mon[34404]: from='client.? 172.21.15.136:0/3913542541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:01:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:20 smithi099 ceph-mon[34404]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T02:01:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:20 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:20.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:20 smithi136 ceph-mon[34754]: from='client.? 172.21.15.136:0/3913542541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:01:20.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:20 smithi136 ceph-mon[34754]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T02:01:20.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:20 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:21 smithi136 ceph-mon[34754]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:21 smithi136 ceph-mon[34754]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T02:01:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:21 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:21 smithi092 ceph-mon[31190]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:21 smithi092 ceph-mon[31190]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T02:01:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:21 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:21 smithi099 ceph-mon[34404]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:21 smithi099 ceph-mon[34404]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T02:01:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:21 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:22 smithi136 ceph-mon[34754]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:23.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:22 smithi092 ceph-mon[31190]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:22 smithi099 ceph-mon[34404]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:24.957 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:24 smithi136 ceph-mon[34754]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:25.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:24 smithi092 ceph-mon[31190]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:24 smithi099 ceph-mon[34404]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:26 smithi136 ceph-mon[34754]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T02:01:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:26 smithi092 ceph-mon[31190]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T02:01:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:26 smithi099 ceph-mon[34404]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T02:01:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:01:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:01:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:01:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:29.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:28 smithi136 ceph-mon[34754]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:01:29.217 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:28 smithi136 ceph-mon[34754]: Deploying daemon osd.11 on smithi136 2024-09-21T02:01:29.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:28 smithi092 ceph-mon[31190]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:01:29.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:28 smithi092 ceph-mon[31190]: Deploying daemon osd.11 on smithi136 2024-09-21T02:01:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:28 smithi099 ceph-mon[34404]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:01:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:28 smithi099 ceph-mon[34404]: Deploying daemon osd.11 on smithi136 2024-09-21T02:01:31.207 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:30 smithi136 ceph-mon[34754]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T02:01:31.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:30 smithi092 ceph-mon[31190]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T02:01:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:30 smithi099 ceph-mon[34404]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T02:01:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:32 smithi136 ceph-mon[34754]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T02:01:33.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:32 smithi092 ceph-mon[31190]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T02:01:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:32 smithi099 ceph-mon[34404]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T02:01:34.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:34.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:34.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:33 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:34.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:34.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:34.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:33 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:33 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:34 smithi136 ceph-mon[34754]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:01:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:34 smithi092 ceph-mon[31190]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:01:35.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:34 smithi099 ceph-mon[34404]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:01:36.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.799+0000 7f8a457fa640 1 -- 172.21.15.136:0/2016337712 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8a480ffff0 con 0x7f8a300b6510 2024-09-21T02:01:36.799 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8a300b6510 msgr2=0x7f8a300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8a300b6510 0x7f8a300b8a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c00 tx=0x7f8a38031400 comp rx=0 tx=0).stop 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 msgr2=0x7f8a4819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 0x7f8a4819a640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9f0 tx=0x7f8a3c00edf0 comp rx=0 tx=0).stop 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 shutdown_connections 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8a300b6510 0x7f8a300b8a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a48104070 0x7f8a4819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:36.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a481036d0 0x7f8a4819a640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:36.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 --2- 172.21.15.136:0/2016337712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a48102640 0x7f8a48199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:36.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 >> 172.21.15.136:0/2016337712 conn(0x7f8a480fdfb0 msgr2=0x7f8a480ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:36.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 shutdown_connections 2024-09-21T02:01:36.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:36.801+0000 7f8a4e7dd640 1 -- 172.21.15.136:0/2016337712 wait complete. 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:37.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:36 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.308 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:01:37.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:37.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:37.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:37.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.314 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:36 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:01:37.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.334 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:37.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:37.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:37.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:36 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:37.475 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T02:01:37.483 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T02:01:37.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:01:37.745 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:38.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 -- 172.21.15.92:0/3537870580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 msgr2=0x7fd46c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:38.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 --2- 172.21.15.92:0/3537870580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c101f10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009900 tx=0x7fd45c02a410 comp rx=0 tx=0).stop 2024-09-21T02:01:38.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 -- 172.21.15.92:0/3537870580 shutdown_connections 2024-09-21T02:01:38.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 --2- 172.21.15.92:0/3537870580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c109500 0x7fd46c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 --2- 172.21.15.92:0/3537870580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c106ba0 0x7fd46c108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 --2- 172.21.15.92:0/3537870580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c101f10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 -- 172.21.15.92:0/3537870580 >> 172.21.15.92:0/3537870580 conn(0x7fd46c0fd770 msgr2=0x7fd46c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 -- 172.21.15.92:0/3537870580 shutdown_connections 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.227+0000 7fd4728dd640 1 -- 172.21.15.92:0/3537870580 wait complete. 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 Processor -- start 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 -- start start 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c106ba0 0x7fd46c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c109500 0x7fd46c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c10f380 con 0x7fd46c101b00 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c10f200 con 0x7fd46c109500 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd4728dd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c10f500 con 0x7fd46c106ba0 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd46bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44356/0 (socket says 172.21.15.92:44356) 2024-09-21T02:01:38.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.228+0000 7fd46bfff640 1 -- 172.21.15.92:0/2520223542 learned_addr learned my addr 172.21.15.92:0/2520223542 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd463fff640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c106ba0 0x7fd46c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd470e7a640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c109500 0x7fd46c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd46bfff640 1 -- 172.21.15.92:0/2520223542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c106ba0 msgr2=0x7fd46c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd46bfff640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c106ba0 0x7fd46c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd46bfff640 1 -- 172.21.15.92:0/2520223542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c109500 msgr2=0x7fd46c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd46bfff640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c109500 0x7fd46c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd46bfff640 1 -- 172.21.15.92:0/2520223542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c19b940 con 0x7fd46c101b00 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd46bfff640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c199fa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd45c006010 tx=0x7fd45c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c03f030 con 0x7fd46c101b00 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c19bbd0 con 0x7fd46c101b00 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c003aa0 con 0x7fd46c101b00 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c035070 con 0x7fd46c101b00 2024-09-21T02:01:38.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.229+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c1a21f0 con 0x7fd46c101b00 2024-09-21T02:01:38.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.230+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c0ff290 con 0x7fd46c101b00 2024-09-21T02:01:38.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.232+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd45c042030 con 0x7fd46c101b00 2024-09-21T02:01:38.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.232+0000 7fd469ffb640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4580b6590 0x7fd4580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:38.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.233+0000 7fd463fff640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4580b6590 0x7fd4580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:38.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.233+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6809+0+0 (secure 0 0 0) 0x7fd45c1078a0 con 0x7fd46c101b00 2024-09-21T02:01:38.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.233+0000 7fd463fff640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4580b6590 0x7fd4580b8aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd454009bd0 tx=0x7fd454005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:38.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.236+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd45c0d2af0 con 0x7fd46c101b00 2024-09-21T02:01:38.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.416+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd46c063fc0 con 0x7fd46c101b00 2024-09-21T02:01:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.416+0000 7fd469ffb640 1 -- 172.21.15.92:0/2520223542 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7fd45c0d7a30 con 0x7fd46c101b00 2024-09-21T02:01:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4580b6590 msgr2=0x7fd4580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4580b6590 0x7fd4580b8aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd454009bd0 tx=0x7fd454005f90 comp rx=0 tx=0).stop 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 msgr2=0x7fd46c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c199fa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd45c006010 tx=0x7fd45c031a50 comp rx=0 tx=0).stop 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 shutdown_connections 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4580b6590 0x7fd4580b8aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c109500 0x7fd46c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c106ba0 0x7fd46c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 --2- 172.21.15.92:0/2520223542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c101b00 0x7fd46c199fa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:38.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 >> 172.21.15.92:0/2520223542 conn(0x7fd46c0fd770 msgr2=0x7fd46c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:38.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 shutdown_connections 2024-09-21T02:01:38.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:38.419+0000 7fd4728dd640 1 -- 172.21.15.92:0/2520223542 wait complete. 2024-09-21T02:01:39.136 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726884078,"num_in_osds":12,"osd_in_since":1726884078,"num_remapped_pgs":0} 2024-09-21T02:01:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:38 smithi136 ceph-mon[34754]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:38 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2520223542' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:38 smithi092 ceph-mon[31190]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:38 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2520223542' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:38 smithi099 ceph-mon[34404]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:38 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2520223542' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:39.784 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:01:39 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:01:39.488+0000 7f9afecb3640 -1 osd.11 0 log_to_monitors true 2024-09-21T02:01:40.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:01:40.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:39 smithi136 ceph-mon[34754]: from='osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:01:40.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:39 smithi136 ceph-mon[34754]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:01:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:39 smithi092 ceph-mon[31190]: from='osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:01:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:39 smithi092 ceph-mon[31190]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:01:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:39 smithi099 ceph-mon[34404]: from='osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:01:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:39 smithi099 ceph-mon[34404]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:01:40.401 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.865+0000 7fba75c4e640 1 -- 172.21.15.92:0/856385791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba701083b0 msgr2=0x7fba7010a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.865+0000 7fba75c4e640 1 --2- 172.21.15.92:0/856385791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba701083b0 0x7fba7010a8c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ca0 tx=0x7fba6802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:01:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.865+0000 7fba75c4e640 1 -- 172.21.15.92:0/856385791 shutdown_connections 2024-09-21T02:01:40.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.865+0000 7fba75c4e640 1 --2- 172.21.15.92:0/856385791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba701083b0 0x7fba7010a8c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 --2- 172.21.15.92:0/856385791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba70105790 0x7fba70107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 --2- 172.21.15.92:0/856385791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba70102cd0 0x7fba70105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 -- 172.21.15.92:0/856385791 >> 172.21.15.92:0/856385791 conn(0x7fba700fd770 msgr2=0x7fba700ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 -- 172.21.15.92:0/856385791 shutdown_connections 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 -- 172.21.15.92:0/856385791 wait complete. 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 Processor -- start 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 -- start start 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.866+0000 7fba75c4e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba70102cd0 0x7fba70102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba75c4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 0x7fba701a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba75c4e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba701083b0 0x7fba701a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba75c4e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010e2c0 con 0x7fba70105790 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba75c4e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010e140 con 0x7fba701083b0 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba75c4e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba7010e440 con 0x7fba70102cd0 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 0x7fba701a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 0x7fba701a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44376/0 (socket says 172.21.15.92:44376) 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 -- 172.21.15.92:0/4291040523 learned_addr learned my addr 172.21.15.92:0/4291040523 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6f7fe640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba70102cd0 0x7fba70102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6ffff640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba701083b0 0x7fba701a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 -- 172.21.15.92:0/4291040523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba70102cd0 msgr2=0x7fba70102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba70102cd0 0x7fba70102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 -- 172.21.15.92:0/4291040523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba701083b0 msgr2=0x7fba701a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba701083b0 0x7fba701a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 -- 172.21.15.92:0/4291040523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701a3c60 con 0x7fba70105790 2024-09-21T02:01:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6effd640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 0x7fba701a1100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba6400d7a0 tx=0x7fba6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:40.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64017070 con 0x7fba70105790 2024-09-21T02:01:40.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba640054d0 con 0x7fba70105790 2024-09-21T02:01:40.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701a3f50 con 0x7fba70105790 2024-09-21T02:01:40.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.867+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64004c10 con 0x7fba70105790 2024-09-21T02:01:40.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.868+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701a43b0 con 0x7fba70105790 2024-09-21T02:01:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.868+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba700ff290 con 0x7fba70105790 2024-09-21T02:01:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.870+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fba64004db0 con 0x7fba70105790 2024-09-21T02:01:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.871+0000 7fba6cff9640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba580b6590 0x7fba580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.871+0000 7fba6f7fe640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba580b6590 0x7fba580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.871+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6831+0+0 (secure 0 0 0) 0x7fba640e6b10 con 0x7fba70105790 2024-09-21T02:01:40.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.871+0000 7fba6f7fe640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba580b6590 0x7fba580b8aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba600058c0 tx=0x7fba6000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:40.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:40.874+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba640ad9f0 con 0x7fba70105790 2024-09-21T02:01:41.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:41.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:41.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.056+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fba70063fc0 con 0x7fba70105790 2024-09-21T02:01:41.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.056+0000 7fba6cff9640 1 -- 172.21.15.92:0/4291040523 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7fba640b1dd0 con 0x7fba70105790 2024-09-21T02:01:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba580b6590 msgr2=0x7fba580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba580b6590 0x7fba580b8aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba600058c0 tx=0x7fba6000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 msgr2=0x7fba701a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 0x7fba701a1100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba6400d7a0 tx=0x7fba6400dc60 comp rx=0 tx=0).stop 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 shutdown_connections 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba580b6590 0x7fba580b8aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba701083b0 0x7fba701a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba70105790 0x7fba701a1100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 --2- 172.21.15.92:0/4291040523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba70102cd0 0x7fba70102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:41.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 >> 172.21.15.92:0/4291040523 conn(0x7fba700fd770 msgr2=0x7fba70103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:41.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 shutdown_connections 2024-09-21T02:01:41.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:41.058+0000 7fba75c4e640 1 -- 172.21.15.92:0/4291040523 wait complete. 2024-09-21T02:01:41.286 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:01:40.931+0000 7f9afac35640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T02:01:41.286 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:01:40.946+0000 7f9af6a33640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:01:41.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:41.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:01:41.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:01:41.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:01:41.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:41.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:01:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:01:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:41.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:41.860 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726884078,"num_in_osds":12,"osd_in_since":1726884078,"num_remapped_pgs":0} 2024-09-21T02:01:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: Detected new or changed devices on smithi136 2024-09-21T02:01:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-21T02:01:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T02:01:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:01:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:41 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4291040523' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: Detected new or changed devices on smithi136 2024-09-21T02:01:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-21T02:01:42.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T02:01:42.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:01:42.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:42.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:42.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:41 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4291040523' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: Detected new or changed devices on smithi136 2024-09-21T02:01:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi136 to 4272M 2024-09-21T02:01:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-09-21T02:01:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:01:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:41 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4291040523' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:42.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:01:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:43.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:42 smithi092 ceph-mon[31190]: purged_snaps scrub starts 2024-09-21T02:01:43.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:42 smithi092 ceph-mon[31190]: purged_snaps scrub ok 2024-09-21T02:01:43.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:42 smithi092 ceph-mon[31190]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:43.207 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:42 smithi092 ceph-mon[31190]: osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] boot 2024-09-21T02:01:43.207 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:42 smithi092 ceph-mon[31190]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T02:01:43.207 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:42 smithi136 ceph-mon[34754]: purged_snaps scrub starts 2024-09-21T02:01:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:42 smithi136 ceph-mon[34754]: purged_snaps scrub ok 2024-09-21T02:01:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:42 smithi136 ceph-mon[34754]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:42 smithi136 ceph-mon[34754]: osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] boot 2024-09-21T02:01:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:42 smithi136 ceph-mon[34754]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T02:01:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:42 smithi099 ceph-mon[34404]: purged_snaps scrub starts 2024-09-21T02:01:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:42 smithi099 ceph-mon[34404]: purged_snaps scrub ok 2024-09-21T02:01:43.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:42 smithi099 ceph-mon[34404]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:01:43.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:42 smithi099 ceph-mon[34404]: osd.11 [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] boot 2024-09-21T02:01:43.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:42 smithi099 ceph-mon[34404]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T02:01:43.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:01:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 -- 172.21.15.92:0/2960324165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 msgr2=0x7f281c10b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 --2- 172.21.15.92:0/2960324165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c10b9b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2814009ca0 tx=0x7f281402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 -- 172.21.15.92:0/2960324165 shutdown_connections 2024-09-21T02:01:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 --2- 172.21.15.92:0/2960324165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c10b9b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 --2- 172.21.15.92:0/2960324165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c106960 0x7f281c108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 --2- 172.21.15.92:0/2960324165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c06c620 0x7f281c106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 -- 172.21.15.92:0/2960324165 >> 172.21.15.92:0/2960324165 conn(0x7f281c0fdfe0 msgr2=0x7f281c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 -- 172.21.15.92:0/2960324165 shutdown_connections 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.625+0000 7f2823c73640 1 -- 172.21.15.92:0/2960324165 wait complete. 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 Processor -- start 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 -- start start 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c06c620 0x7f281c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c106960 0x7f281c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10f390 con 0x7f281c1094a0 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10f210 con 0x7f281c06c620 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2823c73640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10f510 con 0x7f281c106960 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2822210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2822210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60098/0 (socket says 172.21.15.92:60098) 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f2822210640 1 -- 172.21.15.92:0/2081767886 learned_addr learned my addr 172.21.15.92:0/2081767886 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:43.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.626+0000 7f282120e640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c106960 0x7f281c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2821a0f640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c06c620 0x7f281c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2822210640 1 -- 172.21.15.92:0/2081767886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c106960 msgr2=0x7f281c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2822210640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c106960 0x7f281c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2822210640 1 -- 172.21.15.92:0/2081767886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c06c620 msgr2=0x7f281c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2822210640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c06c620 0x7f281c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2822210640 1 -- 172.21.15.92:0/2081767886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c19b9b0 con 0x7f281c1094a0 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f282120e640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c106960 0x7f281c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2821a0f640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c06c620 0x7f281c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2822210640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c19b190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28140097b0 tx=0x7f2814003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814004240 con 0x7f281c1094a0 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281402f380 con 0x7f281c1094a0 2024-09-21T02:01:43.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c19bc40 con 0x7f281c1094a0 2024-09-21T02:01:43.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281402fc30 con 0x7f281c1094a0 2024-09-21T02:01:43.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.627+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c1a23a0 con 0x7f281c1094a0 2024-09-21T02:01:43.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.628+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c0ffb00 con 0x7f281c1094a0 2024-09-21T02:01:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.629+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f28140043e0 con 0x7f281c1094a0 2024-09-21T02:01:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.630+0000 7f2812ffd640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28080b6510 0x7f28080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.630+0000 7f2821a0f640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28080b6510 0x7f28080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.630+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f28141085f0 con 0x7f281c1094a0 2024-09-21T02:01:43.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.631+0000 7f2821a0f640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28080b6510 0x7f28080b8a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f280c005d00 tx=0x7f280c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:43.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.634+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28140cf320 con 0x7f281c1094a0 2024-09-21T02:01:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.814+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f281c063fc0 con 0x7f281c1094a0 2024-09-21T02:01:43.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.815+0000 7f2812ffd640 1 -- 172.21.15.92:0/2081767886 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f28140d3700 con 0x7f281c1094a0 2024-09-21T02:01:43.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28080b6510 msgr2=0x7f28080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28080b6510 0x7f28080b8a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f280c005d00 tx=0x7f280c005ed0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 msgr2=0x7f281c19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c19b190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28140097b0 tx=0x7f2814003c20 comp rx=0 tx=0).stop 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 shutdown_connections 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28080b6510 0x7f28080b8a20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f281c1094a0 0x7f281c19b190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f281c106960 0x7f281c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 --2- 172.21.15.92:0/2081767886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281c06c620 0x7f281c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 >> 172.21.15.92:0/2081767886 conn(0x7f281c0fdfe0 msgr2=0x7f281c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 shutdown_connections 2024-09-21T02:01:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:43.817+0000 7f2823c73640 1 -- 172.21.15.92:0/2081767886 wait complete. 2024-09-21T02:01:44.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:43 smithi092 ceph-mon[31190]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:01:44.277 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:43 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2081767886' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:43 smithi136 ceph-mon[34754]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:01:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:43 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2081767886' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:43 smithi099 ceph-mon[34404]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:01:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:43 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2081767886' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:01:44.516 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726884101,"num_in_osds":12,"osd_in_since":1726884078,"num_remapped_pgs":0} 2024-09-21T02:01:44.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:01:44.758 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:45.167 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:44 smithi092 ceph-mon[31190]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.255+0000 7f4c6a007640 1 -- 172.21.15.92:0/3139247423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c641083c0 msgr2=0x7f4c641087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.255+0000 7f4c6a007640 1 --2- 172.21.15.92:0/3139247423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c641083c0 0x7f4c641087d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a410 comp rx=0 tx=0).stop 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 -- 172.21.15.92:0/3139247423 shutdown_connections 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 --2- 172.21.15.92:0/3139247423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c64108d10 0x7f4c6410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 --2- 172.21.15.92:0/3139247423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c641083c0 0x7f4c641087d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 --2- 172.21.15.92:0/3139247423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 0x7f4c64102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 -- 172.21.15.92:0/3139247423 >> 172.21.15.92:0/3139247423 conn(0x7f4c64079740 msgr2=0x7f4c64079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 -- 172.21.15.92:0/3139247423 shutdown_connections 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 -- 172.21.15.92:0/3139247423 wait complete. 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 Processor -- start 2024-09-21T02:01:45.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 -- start start 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 0x7f4c6419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.256+0000 7f4c6a007640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c641083c0 0x7f4c6419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c6a007640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 0x7f4c6419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c6a007640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64112980 con 0x7f4c64108d10 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c6a007640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64112800 con 0x7f4c641083c0 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c6a007640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64112b00 con 0x7f4c64102a80 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 0x7f4c6419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 0x7f4c6419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60124/0 (socket says 172.21.15.92:60124) 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 -- 172.21.15.92:0/2039657507 learned_addr learned my addr 172.21.15.92:0/2039657507 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c5bfff640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c641083c0 0x7f4c6419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c637fe640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 0x7f4c6419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 -- 172.21.15.92:0/2039657507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 msgr2=0x7f4c6419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 0x7f4c6419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 -- 172.21.15.92:0/2039657507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c641083c0 msgr2=0x7f4c6419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c641083c0 0x7f4c6419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 -- 172.21.15.92:0/2039657507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c641a0330 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c637fe640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 0x7f4c6419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c63fff640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 0x7f4c6419fcc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00cce0 tx=0x7f4c5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00f450 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c641a0620 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c00faa0 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c018b70 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.257+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a0ae0 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.258+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c64103d00 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.260+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f4c5c018d10 con 0x7f4c64108d10 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.260+0000 7f4c617fa640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c500b6540 0x7f4c500b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:45.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.261+0000 7f4c637fe640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c500b6540 0x7f4c500b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:45.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.261+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f4c5c0e73b0 con 0x7f4c64108d10 2024-09-21T02:01:45.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.261+0000 7f4c637fe640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c500b6540 0x7f4c500b8a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c540058c0 tx=0x7f4c54005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:45.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.264+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c5c014030 con 0x7f4c64108d10 2024-09-21T02:01:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:44 smithi136 ceph-mon[34754]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:44 smithi099 ceph-mon[34404]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:45.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.442+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c64102e90 con 0x7f4c64108d10 2024-09-21T02:01:45.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.443+0000 7f4c617fa640 1 -- 172.21.15.92:0/2039657507 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18457 (secure 0 0 0) 0x7f4c5c0ae170 con 0x7f4c64108d10 2024-09-21T02:01:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:45.444 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:01:42.935148+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c500b6540 msgr2=0x7f4c500b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c500b6540 0x7f4c500b8a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4c540058c0 tx=0x7f4c54005d30 comp rx=0 tx=0).stop 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 msgr2=0x7f4c6419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 0x7f4c6419fcc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00cce0 tx=0x7f4c5c00a620 comp rx=0 tx=0).stop 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 shutdown_connections 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c500b6540 0x7f4c500b8a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108d10 0x7f4c6419fcc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c641083c0 0x7f4c6419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 --2- 172.21.15.92:0/2039657507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c64102a80 0x7f4c6419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 >> 172.21.15.92:0/2039657507 conn(0x7f4c64079740 msgr2=0x7f4c64101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 shutdown_connections 2024-09-21T02:01:45.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:45.446+0000 7f4c6a007640 1 -- 172.21.15.92:0/2039657507 wait complete. 2024-09-21T02:01:46.129 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:45 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2039657507' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:01:46.157 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:57:59.243065+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:01:46.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T02:01:46.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:45 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2039657507' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:01:46.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:45 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2039657507' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:01:46.399 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 -- 172.21.15.92:0/2337986963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 msgr2=0x7f4e80110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 --2- 172.21.15.92:0/2337986963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e80110300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e7802a410 comp rx=0 tx=0).stop 2024-09-21T02:01:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 -- 172.21.15.92:0/2337986963 shutdown_connections 2024-09-21T02:01:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 --2- 172.21.15.92:0/2337986963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e80110300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 --2- 172.21.15.92:0/2337986963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e80105270 0x7f4e8010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 --2- 172.21.15.92:0/2337986963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80104840 0x7f4e80104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 -- 172.21.15.92:0/2337986963 >> 172.21.15.92:0/2337986963 conn(0x7f4e80079730 msgr2=0x7f4e80079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 -- 172.21.15.92:0/2337986963 shutdown_connections 2024-09-21T02:01:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.863+0000 7f4e8854a640 1 -- 172.21.15.92:0/2337986963 wait complete. 2024-09-21T02:01:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 Processor -- start 2024-09-21T02:01:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 -- start start 2024-09-21T02:01:46.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80104840 0x7f4e8019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e80105270 0x7f4e8019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e8019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80113a70 con 0x7f4e8010ddf0 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801138f0 con 0x7f4e80105270 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e8854a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80113bf0 con 0x7f4e80104840 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e8019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e8019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60152/0 (socket says 172.21.15.92:60152) 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 -- 172.21.15.92:0/330162050 learned_addr learned my addr 172.21.15.92:0/330162050 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e85ae5640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e80105270 0x7f4e8019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e862e6640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80104840 0x7f4e8019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 -- 172.21.15.92:0/330162050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80104840 msgr2=0x7f4e8019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:46.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80104840 0x7f4e8019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 -- 172.21.15.92:0/330162050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e80105270 msgr2=0x7f4e8019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e80105270 0x7f4e8019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.864+0000 7f4e86ae7640 1 -- 172.21.15.92:0/330162050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e801a03b0 con 0x7f4e8010ddf0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.865+0000 7f4e86ae7640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e8019fc80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e78004770 tx=0x7f4e78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.865+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7803f030 con 0x7f4e8010ddf0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.865+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e78003aa0 con 0x7f4e8010ddf0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.865+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a0640 con 0x7f4e8010ddf0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.865+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78035070 con 0x7f4e8010ddf0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.865+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a6e00 con 0x7f4e8010ddf0 2024-09-21T02:01:46.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.866+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e80104c50 con 0x7f4e8010ddf0 2024-09-21T02:01:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.867+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f4e78042030 con 0x7f4e8010ddf0 2024-09-21T02:01:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.868+0000 7f4e7f7fe640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6590 0x7f4e680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.868+0000 7f4e862e6640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6590 0x7f4e680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.868+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f4e78107860 con 0x7f4e8010ddf0 2024-09-21T02:01:46.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.869+0000 7f4e862e6640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6590 0x7f4e680b8aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e8006bc00 tx=0x7f4e70009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:46.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:46.871+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e780ce590 con 0x7f4e8010ddf0 2024-09-21T02:01:47.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.051+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e801a70a0 con 0x7f4e8010ddf0 2024-09-21T02:01:47.052 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:46 smithi092 ceph-mon[31190]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:47.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.051+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/330162050 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f4e780d2970 con 0x7f4e8010ddf0 2024-09-21T02:01:47.052 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-09-21T02:01:47.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6590 msgr2=0x7f4e680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6590 0x7f4e680b8aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e8006bc00 tx=0x7f4e70009870 comp rx=0 tx=0).stop 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 msgr2=0x7f4e8019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e8019fc80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e78004770 tx=0x7f4e78004270 comp rx=0 tx=0).stop 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 shutdown_connections 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6590 0x7f4e680b8aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8010ddf0 0x7f4e8019fc80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e80105270 0x7f4e8019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 --2- 172.21.15.92:0/330162050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80104840 0x7f4e8019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 >> 172.21.15.92:0/330162050 conn(0x7f4e80079730 msgr2=0x7f4e80107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 shutdown_connections 2024-09-21T02:01:47.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:47.054+0000 7f4e8854a640 1 -- 172.21.15.92:0/330162050 wait complete. 2024-09-21T02:01:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:46 smithi136 ceph-mon[34754]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:47.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:46 smithi099 ceph-mon[34404]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:47.703 INFO:tasks.cephadm:Adding mds.a on smithi092 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.d on smithi092 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.g on smithi092 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.j on smithi092 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.b on smithi099 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.e on smithi099 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.h on smithi099 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.k on smithi099 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-09-21T02:01:47.704 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-09-21T02:01:47.704 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-09-21T02:01:47.964 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:01:48.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:47 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/330162050' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:01:48.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:47 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/330162050' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:01:48.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:47 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/330162050' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:01:48.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 -- 172.21.15.136:0/2701279962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c109470 msgr2=0x7fc80c10b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:48.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 --2- 172.21.15.136:0/2701279962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c109470 0x7fc80c10b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc804009ca0 tx=0x7fc80402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 -- 172.21.15.136:0/2701279962 shutdown_connections 2024-09-21T02:01:48.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 --2- 172.21.15.136:0/2701279962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c109470 0x7fc80c10b980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 --2- 172.21.15.136:0/2701279962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc80c1068d0 0x7fc80c108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 --2- 172.21.15.136:0/2701279962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc80c06b980 0x7fc80c106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 -- 172.21.15.136:0/2701279962 >> 172.21.15.136:0/2701279962 conn(0x7fc80c0fdfb0 msgr2=0x7fc80c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 -- 172.21.15.136:0/2701279962 shutdown_connections 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.449+0000 7fc812320640 1 -- 172.21.15.136:0/2701279962 wait complete. 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 Processor -- start 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 -- start start 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc80c06b980 0x7fc80c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 0x7fc80c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc80c109470 0x7fc80c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c10f390 con 0x7fc80c06b980 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c10f210 con 0x7fc80c109470 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.450+0000 7fc812320640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c10f510 con 0x7fc80c1068d0 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 0x7fc80c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 0x7fc80c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:51888/0 (socket says 172.21.15.136:51888) 2024-09-21T02:01:48.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 -- 172.21.15.136:0/1756435412 learned_addr learned my addr 172.21.15.136:0/1756435412 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80bfff640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc80c06b980 0x7fc80c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc8108bd640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc80c109470 0x7fc80c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 -- 172.21.15.136:0/1756435412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc80c109470 msgr2=0x7fc80c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc80c109470 0x7fc80c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 -- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc80c06b980 msgr2=0x7fc80c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc80c06b980 0x7fc80c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 -- 172.21.15.136:0/1756435412 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c19b950 con 0x7fc80c1068d0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc80b7fe640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 0x7fc80c19a590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400e9f0 tx=0x7fc7f400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400cd80 con 0x7fc80c1068d0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c19bc40 con 0x7fc80c1068d0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f40054d0 con 0x7fc80c1068d0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4010690 con 0x7fc80c1068d0 2024-09-21T02:01:48.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.451+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1a2340 con 0x7fc80c1068d0 2024-09-21T02:01:48.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.453+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c0ffad0 con 0x7fc80c1068d0 2024-09-21T02:01:48.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.455+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc7f4004e50 con 0x7fc80c1068d0 2024-09-21T02:01:48.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.456+0000 7fc8097fa640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7f80b6510 0x7fc7f80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:48.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.456+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc7f40e74f0 con 0x7fc80c1068d0 2024-09-21T02:01:48.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.456+0000 7fc80bfff640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7f80b6510 0x7fc7f80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:48.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.457+0000 7fc80bfff640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7f80b6510 0x7fc7f80b8a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009bd0 tx=0x7fc7fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:48.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.459+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7f4056710 con 0x7fc80c1068d0 2024-09-21T02:01:48.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.654+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) -- 0x7fc80c063900 con 0x7fc7f80b6510 2024-09-21T02:01:48.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.662+0000 7fc8097fa640 1 -- 172.21.15.136:0/1756435412 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc80c063900 con 0x7fc7f80b6510 2024-09-21T02:01:48.663 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7f80b6510 msgr2=0x7fc7f80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7f80b6510 0x7fc7f80b8a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009bd0 tx=0x7fc7fc005f90 comp rx=0 tx=0).stop 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 msgr2=0x7fc80c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 0x7fc80c19a590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400e9f0 tx=0x7fc7f400edf0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 shutdown_connections 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7f80b6510 0x7fc7f80b8a20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc80c109470 0x7fc80c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc80c1068d0 0x7fc80c19a590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 --2- 172.21.15.136:0/1756435412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc80c06b980 0x7fc80c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:48.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 >> 172.21.15.136:0/1756435412 conn(0x7fc80c0fdfb0 msgr2=0x7fc80c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:48.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 shutdown_connections 2024-09-21T02:01:48.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-21T02:01:48.664+0000 7fc812320640 1 -- 172.21.15.136:0/1756435412 wait complete. 2024-09-21T02:01:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:48 smithi136 ceph-mon[34754]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:48 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:48 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:48 smithi092 ceph-mon[31190]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:49.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:48 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:49.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:48 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:48 smithi099 ceph-mon[34404]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:48 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:48 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:01:49.347 DEBUG:teuthology.orchestra.run.smithi092:mds.a> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T02:01:49.350 DEBUG:teuthology.orchestra.run.smithi092:mds.d> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T02:01:49.393 DEBUG:teuthology.orchestra.run.smithi092:mds.g> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T02:01:49.437 DEBUG:teuthology.orchestra.run.smithi092:mds.j> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T02:01:49.479 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T02:01:49.483 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T02:01:49.525 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T02:01:49.568 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T02:01:49.612 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T02:01:49.615 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T02:01:49.658 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T02:01:49.701 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T02:01:49.744 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T02:01:49.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:50.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:49 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:50.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:01:50.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-21T02:01:50.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:50.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:50.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:50.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:50.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:50.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:49 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:50.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:01:50.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-21T02:01:50.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:50.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:01:50.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:50.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:50.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:50.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:49 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 -- 172.21.15.92:0/311154974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710107bd0 msgr2=0x7fc710107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 --2- 172.21.15.92:0/311154974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710107bd0 0x7fc710107fe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc700009900 tx=0x7fc70002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 -- 172.21.15.92:0/311154974 shutdown_connections 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 --2- 172.21.15.92:0/311154974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 0x7fc71010ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 --2- 172.21.15.92:0/311154974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710107bd0 0x7fc710107fe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 --2- 172.21.15.92:0/311154974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc710102290 0x7fc7101026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.426+0000 7fc717375640 1 -- 172.21.15.92:0/311154974 >> 172.21.15.92:0/311154974 conn(0x7fc710079ea0 msgr2=0x7fc71007a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.427+0000 7fc717375640 1 -- 172.21.15.92:0/311154974 shutdown_connections 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.427+0000 7fc717375640 1 -- 172.21.15.92:0/311154974 wait complete. 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.427+0000 7fc717375640 1 Processor -- start 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.427+0000 7fc717375640 1 -- start start 2024-09-21T02:01:50.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 0x7fc71019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc710107bd0 0x7fc71019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 0x7fc71019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc710112a60 con 0x7fc710102290 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101128e0 con 0x7fc710108520 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc710112be0 con 0x7fc710107bd0 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 0x7fc71019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 0x7fc71019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60164/0 (socket says 172.21.15.92:60164) 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 -- 172.21.15.92:0/3105892754 learned_addr learned my addr 172.21.15.92:0/3105892754 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc714910640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc710107bd0 0x7fc71019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715912640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 0x7fc71019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 -- 172.21.15.92:0/3105892754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc710107bd0 msgr2=0x7fc71019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc710107bd0 0x7fc71019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 -- 172.21.15.92:0/3105892754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 msgr2=0x7fc71019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 0x7fc71019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 -- 172.21.15.92:0/3105892754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7101a6620 con 0x7fc710102290 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715912640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 0x7fc71019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc715111640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 0x7fc71019b390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d770 tx=0x7fc70c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c017070 con 0x7fc710102290 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7101a6910 con 0x7fc710102290 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c0054d0 con 0x7fc710102290 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.428+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c004c30 con 0x7fc710102290 2024-09-21T02:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.429+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a6dd0 con 0x7fc710102290 2024-09-21T02:01:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.430+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7101026a0 con 0x7fc710102290 2024-09-21T02:01:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.431+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fc70c004dd0 con 0x7fc710102290 2024-09-21T02:01:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.432+0000 7fc7067fc640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6fc0b6540 0x7fc6fc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.432+0000 7fc714910640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6fc0b6540 0x7fc6fc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.432+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc70c0e6ce0 con 0x7fc710102290 2024-09-21T02:01:50.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.433+0000 7fc714910640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6fc0b6540 0x7fc6fc0b8a50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc700009230 tx=0x7fc700031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.435+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc70c0ada10 con 0x7fc710102290 2024-09-21T02:01:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.648+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc71010fb20 con 0x7fc710102290 2024-09-21T02:01:50.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.649+0000 7fc7067fc640 1 -- 172.21.15.92:0/3105892754 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fc70c0b1df0 con 0x7fc710102290 2024-09-21T02:01:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:50.650 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"btime":"2024-09-21T01:54:21:537800+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T02:01:50.650 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 1 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6fc0b6540 msgr2=0x7fc6fc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6fc0b6540 0x7fc6fc0b8a50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc700009230 tx=0x7fc700031400 comp rx=0 tx=0).stop 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 msgr2=0x7fc71019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 0x7fc71019b390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d770 tx=0x7fc70c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 shutdown_connections 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6fc0b6540 0x7fc6fc0b8a50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc710108520 0x7fc71019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc710107bd0 0x7fc71019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 --2- 172.21.15.92:0/3105892754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc710102290 0x7fc71019b390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 >> 172.21.15.92:0/3105892754 conn(0x7fc710079ea0 msgr2=0x7fc710103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 shutdown_connections 2024-09-21T02:01:50.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:50.651+0000 7fc717375640 1 -- 172.21.15.92:0/3105892754 wait complete. 2024-09-21T02:01:50.975 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:50 smithi136 systemd[1]: Starting Ceph mds.l for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:01:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:50 smithi136 ceph-mon[34754]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:50 smithi136 ceph-mon[34754]: Deploying daemon mds.l on smithi136 2024-09-21T02:01:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:50 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3105892754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:51.283 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 podman[62946]: 2024-09-21 02:01:50.9550221 +0000 UTC m=+0.016026050 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:01:51.284 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 podman[62946]: 2024-09-21 02:01:51.075876541 +0000 UTC m=+0.136880522 container create cdcff13c35e561a507584f1a750a24e1db8c8df82f0dd4a6efcd1b7808e03da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:01:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:50 smithi092 ceph-mon[31190]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:50 smithi092 ceph-mon[31190]: Deploying daemon mds.l on smithi136 2024-09-21T02:01:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:50 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3105892754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:50 smithi099 ceph-mon[34404]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:50 smithi099 ceph-mon[34404]: Deploying daemon mds.l on smithi136 2024-09-21T02:01:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:50 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3105892754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:51.783 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 podman[62946]: 2024-09-21 02:01:51.286590591 +0000 UTC m=+0.347594544 container init cdcff13c35e561a507584f1a750a24e1db8c8df82f0dd4a6efcd1b7808e03da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:01:51.783 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 podman[62946]: 2024-09-21 02:01:51.289644031 +0000 UTC m=+0.350647979 container start cdcff13c35e561a507584f1a750a24e1db8c8df82f0dd4a6efcd1b7808e03da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T02:01:51.783 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 ceph-mds[62965]: -- 172.21.15.136:0/553017653 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563cd296b860 con 0x563cd6b13a80 2024-09-21T02:01:51.784 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l[62961]: starting mds.l at 2024-09-21T02:01:51.784 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 bash[62946]: cdcff13c35e561a507584f1a750a24e1db8c8df82f0dd4a6efcd1b7808e03da5 2024-09-21T02:01:51.784 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:01:51 smithi136 systemd[1]: Started Ceph mds.l for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:01:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6832/2879720066,v1:172.21.15.136:6833/2879720066] up:boot 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: fsmap 1 up:standby 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:52.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:52 smithi099 ceph-mon[34404]: Deploying daemon mds.k on smithi099 2024-09-21T02:01:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6832/2879720066,v1:172.21.15.136:6833/2879720066] up:boot 2024-09-21T02:01:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: fsmap 1 up:standby 2024-09-21T02:01:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:01:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:52.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:52.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:52.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:52 smithi136 ceph-mon[34754]: Deploying daemon mds.k on smithi099 2024-09-21T02:01:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6832/2879720066,v1:172.21.15.136:6833/2879720066] up:boot 2024-09-21T02:01:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: fsmap 1 up:standby 2024-09-21T02:01:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:01:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:52.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:52.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:52.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:52.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:52 smithi092 ceph-mon[31190]: Deploying daemon mds.k on smithi099 2024-09-21T02:01:53.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:01:53.500 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 systemd[1]: Starting Ceph mds.k for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:01:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:53.765 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 podman[61575]: 2024-09-21 02:01:53.402312748 +0000 UTC m=+0.015520581 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:01:53.765 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 podman[61575]: 2024-09-21 02:01:53.521055954 +0000 UTC m=+0.134263772 container create 424c9ed6011cab1f5e26f3fae76d7ad05031152753846ed0c68a9ffc2afd435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:01:53.765 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 podman[61575]: 2024-09-21 02:01:53.765627287 +0000 UTC m=+0.378835105 container init 424c9ed6011cab1f5e26f3fae76d7ad05031152753846ed0c68a9ffc2afd435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T02:01:54.081 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 podman[61575]: 2024-09-21 02:01:53.768555712 +0000 UTC m=+0.381763543 container start 424c9ed6011cab1f5e26f3fae76d7ad05031152753846ed0c68a9ffc2afd435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T02:01:54.081 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 ceph-mds[61595]: -- 172.21.15.99:0/590706431 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565e1715860 con 0x5565e58bd600 2024-09-21T02:01:54.081 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k[61591]: starting mds.k at 2024-09-21T02:01:54.081 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 bash[61575]: 424c9ed6011cab1f5e26f3fae76d7ad05031152753846ed0c68a9ffc2afd435e 2024-09-21T02:01:54.081 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:01:53 smithi099 systemd[1]: Started Ceph mds.k for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 -- 172.21.15.92:0/2100713148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a70102ae0 msgr2=0x7f6a70102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 --2- 172.21.15.92:0/2100713148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a70102ae0 0x7f6a70102ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009900 tx=0x7f6a6002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 -- 172.21.15.92:0/2100713148 shutdown_connections 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 --2- 172.21.15.92:0/2100713148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a70108d20 0x7f6a7010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 --2- 172.21.15.92:0/2100713148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a701083d0 0x7f6a701087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 --2- 172.21.15.92:0/2100713148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a70102ae0 0x7f6a70102ef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 -- 172.21.15.92:0/2100713148 >> 172.21.15.92:0/2100713148 conn(0x7f6a70079790 msgr2=0x7f6a70079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 -- 172.21.15.92:0/2100713148 shutdown_connections 2024-09-21T02:01:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.137+0000 7f6a76727640 1 -- 172.21.15.92:0/2100713148 wait complete. 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 Processor -- start 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 -- start start 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a70102ae0 0x7f6a701990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 0x7f6a70199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70108d20 0x7f6a70199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701129e0 con 0x7f6a701083d0 2024-09-21T02:01:54.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112860 con 0x7f6a70102ae0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a76727640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112b60 con 0x7f6a70108d20 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a67fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 0x7f6a70199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a67fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 0x7f6a70199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45470/0 (socket says 172.21.15.92:45470) 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a67fff640 1 -- 172.21.15.92:0/1550964194 learned_addr learned my addr 172.21.15.92:0/1550964194 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.138+0000 7f6a6ffff640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a70102ae0 0x7f6a701990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a74cc4640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70108d20 0x7f6a70199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a67fff640 1 -- 172.21.15.92:0/1550964194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70108d20 msgr2=0x7f6a70199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a67fff640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70108d20 0x7f6a70199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a67fff640 1 -- 172.21.15.92:0/1550964194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a70102ae0 msgr2=0x7f6a701990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a67fff640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a70102ae0 0x7f6a701990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a67fff640 1 -- 172.21.15.92:0/1550964194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a701a87c0 con 0x7f6a701083d0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a74cc4640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70108d20 0x7f6a70199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a67fff640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 0x7f6a70199600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800e9f0 tx=0x7f6a5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800ccc0 con 0x7f6a701083d0 2024-09-21T02:01:54.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a5800ce60 con 0x7f6a701083d0 2024-09-21T02:01:54.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a701a8ab0 con 0x7f6a701083d0 2024-09-21T02:01:54.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a701a8fa0 con 0x7f6a701083d0 2024-09-21T02:01:54.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.139+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58010430 con 0x7f6a701083d0 2024-09-21T02:01:54.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.140+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a70102ef0 con 0x7f6a701083d0 2024-09-21T02:01:54.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.142+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f6a580046c0 con 0x7f6a701083d0 2024-09-21T02:01:54.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.143+0000 7f6a6dffb640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a5c0b6510 0x7f6a5c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:54.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.143+0000 7f6a6ffff640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a5c0b6510 0x7f6a5c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:54.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.143+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f6a580e7490 con 0x7f6a701083d0 2024-09-21T02:01:54.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.143+0000 7f6a6ffff640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a5c0b6510 0x7f6a5c0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a60000c00 tx=0x7f6a60031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:54.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.146+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6a58056710 con 0x7f6a701083d0 2024-09-21T02:01:54.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.371+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a7006bc90 con 0x7f6a701083d0 2024-09-21T02:01:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.371+0000 7f6a6dffb640 1 -- 172.21.15.92:0/1550964194 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f6a580ae1a0 con 0x7f6a701083d0 2024-09-21T02:01:54.372 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:54.372 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"btime":"2024-09-21T02:01:54:333922+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:01:54.372 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2024-09-21T02:01:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a5c0b6510 msgr2=0x7f6a5c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a5c0b6510 0x7f6a5c0b8a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a60000c00 tx=0x7f6a60031400 comp rx=0 tx=0).stop 2024-09-21T02:01:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 msgr2=0x7f6a70199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 0x7f6a70199600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800e9f0 tx=0x7f6a5800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 shutdown_connections 2024-09-21T02:01:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a5c0b6510 0x7f6a5c0b8a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a70108d20 0x7f6a70199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a701083d0 0x7f6a70199600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 --2- 172.21.15.92:0/1550964194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a70102ae0 0x7f6a701990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 >> 172.21.15.92:0/1550964194 conn(0x7f6a70079790 msgr2=0x7f6a701064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 shutdown_connections 2024-09-21T02:01:54.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:54.373+0000 7f6a76727640 1 -- 172.21.15.92:0/1550964194 wait complete. 2024-09-21T02:01:55.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: Deploying daemon mds.j on smithi092 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6832/3451484092,v1:172.21.15.99:6833/3451484092] up:boot 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: fsmap 2 up:standby 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:01:55.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:54 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1550964194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: Deploying daemon mds.j on smithi092 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6832/3451484092,v1:172.21.15.99:6833/3451484092] up:boot 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: fsmap 2 up:standby 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:01:55.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:54 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1550964194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:55.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: Deploying daemon mds.j on smithi092 2024-09-21T02:01:55.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6832/3451484092,v1:172.21.15.99:6833/3451484092] up:boot 2024-09-21T02:01:55.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: fsmap 2 up:standby 2024-09-21T02:01:55.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:01:55.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:54 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1550964194' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:01:56.942 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:56 smithi092 systemd[1]: Starting Ceph mds.j for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:01:57.269 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:56 smithi092 ceph-mon[31190]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:57.269 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 podman[71191]: 2024-09-21 02:01:56.90721639 +0000 UTC m=+0.016274030 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:01:57.269 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 podman[71191]: 2024-09-21 02:01:57.033389942 +0000 UTC m=+0.142447569 container create c5d7e7469ad55876169bfa297c431ff19df388a1ea186b55235b5f2e9213c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T02:01:57.269 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 podman[71191]: 2024-09-21 02:01:57.269232473 +0000 UTC m=+0.378290105 container init c5d7e7469ad55876169bfa297c431ff19df388a1ea186b55235b5f2e9213c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T02:01:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:56 smithi136 ceph-mon[34754]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:56 smithi099 ceph-mon[34404]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:57.554 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 podman[71191]: 2024-09-21 02:01:57.272066221 +0000 UTC m=+0.381123860 container start c5d7e7469ad55876169bfa297c431ff19df388a1ea186b55235b5f2e9213c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:01:57.554 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 ceph-mds[71212]: -- 172.21.15.92:0/2745005314 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1c62c3860 con 0x55d1ca46b600 2024-09-21T02:01:57.554 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j[71208]: starting mds.j at 2024-09-21T02:01:57.554 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 bash[71191]: c5d7e7469ad55876169bfa297c431ff19df388a1ea186b55235b5f2e9213c8df 2024-09-21T02:01:57.555 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:01:57 smithi092 systemd[1]: Started Ceph mds.j for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:01:57.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:01:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: Deploying daemon mds.d on smithi092 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6834/859830568,v1:172.21.15.92:6835/859830568] up:boot 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: fsmap 3 up:standby 2024-09-21T02:01:58.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:01:58 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:01:58.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: Deploying daemon mds.d on smithi092 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6834/859830568,v1:172.21.15.92:6835/859830568] up:boot 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: fsmap 3 up:standby 2024-09-21T02:01:58.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:01:58 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:01:58.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: Deploying daemon mds.d on smithi092 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6834/859830568,v1:172.21.15.92:6835/859830568] up:boot 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: fsmap 3 up:standby 2024-09-21T02:01:58.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:01:58 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:01:58.885 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:01:59.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 -- 172.21.15.92:0/2041253310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24077380 msgr2=0x7fdd240757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 --2- 172.21.15.92:0/2041253310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24077380 0x7fdd240757b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009900 tx=0x7fdd2002a410 comp rx=0 tx=0).stop 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 -- 172.21.15.92:0/2041253310 shutdown_connections 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 --2- 172.21.15.92:0/2041253310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd24114430 0x7fdd24116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 --2- 172.21.15.92:0/2041253310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd24075dd0 0x7fdd240761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 --2- 172.21.15.92:0/2041253310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24077380 0x7fdd240757b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 -- 172.21.15.92:0/2041253310 >> 172.21.15.92:0/2041253310 conn(0x7fdd2407b850 msgr2=0x7fdd2407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 -- 172.21.15.92:0/2041253310 shutdown_connections 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 -- 172.21.15.92:0/2041253310 wait complete. 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.532+0000 7fdd2b894640 1 Processor -- start 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 -- start start 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd24075dd0 0x7fdd241a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd24077380 0x7fdd241a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 0x7fdd241a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2411a150 con 0x7fdd24114430 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24119fd0 con 0x7fdd24075dd0 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd2b894640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2411a2d0 con 0x7fdd24077380 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 0x7fdd241a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 0x7fdd241a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45516/0 (socket says 172.21.15.92:45516) 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 -- 172.21.15.92:0/3163280780 learned_addr learned my addr 172.21.15.92:0/3163280780 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29630640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd24075dd0 0x7fdd241a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd28e2f640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd24077380 0x7fdd241a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 -- 172.21.15.92:0/3163280780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd24077380 msgr2=0x7fdd241a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd24077380 0x7fdd241a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 -- 172.21.15.92:0/3163280780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd24075dd0 msgr2=0x7fdd241a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd24075dd0 0x7fdd241a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 -- 172.21.15.92:0/3163280780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd241a6b10 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.533+0000 7fdd29e31640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 0x7fdd241a63e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00b970 tx=0x7fdd1c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.534+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c019070 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.534+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1c014440 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.534+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd241a6e00 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.534+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0075d0 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.534+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd241ad560 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.535+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd24076260 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.536+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fdd1c01c070 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.537+0000 7fdd1a7fc640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd100b6590 0x7fdd100b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.537+0000 7fdd29630640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd100b6590 0x7fdd100b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.537+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fdd1c0e6740 con 0x7fdd24114430 2024-09-21T02:01:59.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.538+0000 7fdd29630640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd100b6590 0x7fdd100b8aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdd20031440 tx=0x7fdd20005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:01:59.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.541+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd1c0b1850 con 0x7fdd24114430 2024-09-21T02:01:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.763+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd241a1cd0 con 0x7fdd24114430 2024-09-21T02:01:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.763+0000 7fdd1a7fc640 1 -- 172.21.15.92:0/3163280780 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fdd241a1cd0 con 0x7fdd24114430 2024-09-21T02:01:59.765 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:01:59.765 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":4,"btime":"2024-09-21T02:01:57:942956+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:01:59.765 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 4 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd100b6590 msgr2=0x7fdd100b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd100b6590 0x7fdd100b8aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdd20031440 tx=0x7fdd20005e80 comp rx=0 tx=0).stop 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 msgr2=0x7fdd241a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 0x7fdd241a63e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00b970 tx=0x7fdd1c00be30 comp rx=0 tx=0).stop 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 shutdown_connections 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd100b6590 0x7fdd100b8aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd24114430 0x7fdd241a63e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd24077380 0x7fdd241a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 --2- 172.21.15.92:0/3163280780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd24075dd0 0x7fdd241a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:01:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 >> 172.21.15.92:0/3163280780 conn(0x7fdd2407b850 msgr2=0x7fdd24113100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:01:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 shutdown_connections 2024-09-21T02:01:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:01:59.766+0000 7fdd2b894640 1 -- 172.21.15.92:0/3163280780 wait complete. 2024-09-21T02:02:00.698 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:00 smithi092 ceph-mon[31190]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:00.698 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:00 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3163280780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:00 smithi136 ceph-mon[34754]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:00 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3163280780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:00 smithi099 ceph-mon[34404]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:00 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3163280780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:01.054 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:02:00 smithi092 systemd[1]: Started Ceph mds.d for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:01.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: Deploying daemon mds.e on smithi099 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6836/67515545,v1:172.21.15.92:6837/67515545] up:boot 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: fsmap 4 up:standby 2024-09-21T02:02:01.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:01 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:02:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: Deploying daemon mds.e on smithi099 2024-09-21T02:02:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6836/67515545,v1:172.21.15.92:6837/67515545] up:boot 2024-09-21T02:02:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: fsmap 4 up:standby 2024-09-21T02:02:02.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:01 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:02:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:02.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: Deploying daemon mds.e on smithi099 2024-09-21T02:02:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6836/67515545,v1:172.21.15.92:6837/67515545] up:boot 2024-09-21T02:02:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: fsmap 4 up:standby 2024-09-21T02:02:02.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:01 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:02:02.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:02.831 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:02 smithi099 systemd[1]: Starting Ceph mds.e for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:02.943 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:02 smithi092 ceph-mon[31190]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:03.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:02 smithi099 ceph-mon[34404]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:03.091 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:02 smithi099 podman[61858]: 2024-09-21 02:02:02.869309658 +0000 UTC m=+0.015334033 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:03.091 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:02 smithi099 podman[61858]: 2024-09-21 02:02:02.974039894 +0000 UTC m=+0.120064255 container create 1a54c297d594919802f95dad6e470cf9fdd27ced5931c5cf9f849309564e6bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T02:02:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:02 smithi136 ceph-mon[34754]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 -- 172.21.15.92:0/3028452548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c541094c0 msgr2=0x7f8c541098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 --2- 172.21.15.92:0/3028452548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c541094c0 0x7f8c541098d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009900 tx=0x7f8c3c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 -- 172.21.15.92:0/3028452548 shutdown_connections 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 --2- 172.21.15.92:0/3028452548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c54109e10 0x7f8c54110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 --2- 172.21.15.92:0/3028452548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c541094c0 0x7f8c541098d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 --2- 172.21.15.92:0/3028452548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c54103b80 0x7f8c54103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 -- 172.21.15.92:0/3028452548 >> 172.21.15.92:0/3028452548 conn(0x7f8c540ff5d0 msgr2=0x7f8c54101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 -- 172.21.15.92:0/3028452548 shutdown_connections 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.314+0000 7f8c5925b640 1 -- 172.21.15.92:0/3028452548 wait complete. 2024-09-21T02:02:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 Processor -- start 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 -- start start 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c54103b80 0x7f8c5419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c541094c0 0x7f8c5419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 0x7f8c5419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113ad0 con 0x7f8c54109e10 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113950 con 0x7f8c541094c0 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5925b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113c50 con 0x7f8c54103b80 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 0x7f8c5419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 0x7f8c5419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50078/0 (socket says 172.21.15.92:50078) 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 -- 172.21.15.92:0/2136042017 learned_addr learned my addr 172.21.15.92:0/2136042017 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5259c640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c541094c0 0x7f8c5419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c52d9d640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c54103b80 0x7f8c5419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 -- 172.21.15.92:0/2136042017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c54103b80 msgr2=0x7f8c5419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c54103b80 0x7f8c5419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 -- 172.21.15.92:0/2136042017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c541094c0 msgr2=0x7f8c5419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.315+0000 7f8c5359e640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c541094c0 0x7f8c5419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c5359e640 1 -- 172.21.15.92:0/2136042017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c541a0430 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c5359e640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 0x7f8c5419fd00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009fd0 tx=0x7f8c4400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44017070 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c541a0720 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c44003e20 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44010460 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.316+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a6e80 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.317+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c54103f90 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.318+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8c44003970 con 0x7f8c54109e10 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.319+0000 7f8c4bfff640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c400b6510 0x7f8c400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.319+0000 7f8c52d9d640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c400b6510 0x7f8c400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:03.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.319+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f8c440e6b30 con 0x7f8c54109e10 2024-09-21T02:02:03.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.320+0000 7f8c52d9d640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c400b6510 0x7f8c400b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009bd0 tx=0x7f8c4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:03.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.323+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8c440b1c40 con 0x7f8c54109e10 2024-09-21T02:02:03.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.539+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c5410bc40 con 0x7f8c54109e10 2024-09-21T02:02:03.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.539+0000 7f8c4bfff640 1 -- 172.21.15.92:0/2136042017 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7f8c440b6b80 con 0x7f8c54109e10 2024-09-21T02:02:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:03.541 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"btime":"2024-09-21T02:02:01:414137+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:02:03.542 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 5 2024-09-21T02:02:03.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.542+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c400b6510 msgr2=0x7f8c400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:03.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.542+0000 7f8c5925b640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c400b6510 0x7f8c400b8a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009bd0 tx=0x7f8c4c005f90 comp rx=0 tx=0).stop 2024-09-21T02:02:03.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.542+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 msgr2=0x7f8c5419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:03.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.542+0000 7f8c5925b640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 0x7f8c5419fd00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009fd0 tx=0x7f8c4400b260 comp rx=0 tx=0).stop 2024-09-21T02:02:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 shutdown_connections 2024-09-21T02:02:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c400b6510 0x7f8c400b8a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c54109e10 0x7f8c5419fd00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c541094c0 0x7f8c5419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 --2- 172.21.15.92:0/2136042017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c54103b80 0x7f8c5419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 >> 172.21.15.92:0/2136042017 conn(0x7f8c540ff5d0 msgr2=0x7f8c541019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 shutdown_connections 2024-09-21T02:02:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:03.543+0000 7f8c5925b640 1 -- 172.21.15.92:0/2136042017 wait complete. 2024-09-21T02:02:03.582 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:03 smithi099 podman[61858]: 2024-09-21 02:02:03.193400622 +0000 UTC m=+0.339424989 container init 1a54c297d594919802f95dad6e470cf9fdd27ced5931c5cf9f849309564e6bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T02:02:03.582 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:03 smithi099 podman[61858]: 2024-09-21 02:02:03.196505655 +0000 UTC m=+0.342530028 container start 1a54c297d594919802f95dad6e470cf9fdd27ced5931c5cf9f849309564e6bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:02:03.582 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:03 smithi099 ceph-mds[61878]: -- 172.21.15.99:0/1337316212 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636cd153860 con 0x5636d12fb180 2024-09-21T02:02:03.582 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:03 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e[61873]: starting mds.e at 2024-09-21T02:02:03.582 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:03 smithi099 bash[61858]: 1a54c297d594919802f95dad6e470cf9fdd27ced5931c5cf9f849309564e6bf1 2024-09-21T02:02:03.582 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:02:03 smithi099 systemd[1]: Started Ceph mds.e for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:04.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:04.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: Deploying daemon mds.f on smithi136 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2136042017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6834/2004123702,v1:172.21.15.99:6835/2004123702] up:boot 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: fsmap 5 up:standby 2024-09-21T02:02:04.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:04 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:02:04.700 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: Deploying daemon mds.f on smithi136 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2136042017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6834/2004123702,v1:172.21.15.99:6835/2004123702] up:boot 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: fsmap 5 up:standby 2024-09-21T02:02:04.701 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:04 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:02:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: Deploying daemon mds.f on smithi136 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2136042017' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6834/2004123702,v1:172.21.15.99:6835/2004123702] up:boot 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: fsmap 5 up:standby 2024-09-21T02:02:04.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:04 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:02:05.283 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 systemd[1]: Starting Ceph mds.f for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:05.631 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 podman[63216]: 2024-09-21 02:02:05.287209354 +0000 UTC m=+0.016157595 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:05.631 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 podman[63216]: 2024-09-21 02:02:05.428034169 +0000 UTC m=+0.156982400 container create b140a19dfe02b54a180391ba0c791831036712163bd16d633741095874f8d837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T02:02:05.631 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 podman[63216]: 2024-09-21 02:02:05.630422291 +0000 UTC m=+0.359370520 container init b140a19dfe02b54a180391ba0c791831036712163bd16d633741095874f8d837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T02:02:06.033 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 podman[63216]: 2024-09-21 02:02:05.633351945 +0000 UTC m=+0.362300176 container start b140a19dfe02b54a180391ba0c791831036712163bd16d633741095874f8d837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T02:02:06.033 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 ceph-mds[63235]: -- 172.21.15.136:0/3810149050 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ec9759860 con 0x562ecd901a80 2024-09-21T02:02:06.034 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f[63231]: starting mds.f at 2024-09-21T02:02:06.034 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 bash[63216]: b140a19dfe02b54a180391ba0c791831036712163bd16d633741095874f8d837 2024-09-21T02:02:06.034 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:02:05 smithi136 systemd[1]: Started Ceph mds.f for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:06.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:06.527 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:06.939 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:06.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: Deploying daemon mds.c on smithi136 2024-09-21T02:02:06.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] up:boot 2024-09-21T02:02:06.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: fsmap 6 up:standby 2024-09-21T02:02:06.940 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:06 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:02:07.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:07.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: Deploying daemon mds.c on smithi136 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] up:boot 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: fsmap 6 up:standby 2024-09-21T02:02:07.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:06 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 -- 172.21.15.92:0/3515820049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 msgr2=0x7fe90c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 --2- 172.21.15.92:0/3515820049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c102ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009900 tx=0x7fe8fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 -- 172.21.15.92:0/3515820049 shutdown_connections 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 --2- 172.21.15.92:0/3515820049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108d70 0x7fe90c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 --2- 172.21.15.92:0/3515820049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe90c108420 0x7fe90c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 --2- 172.21.15.92:0/3515820049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c102ef0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 -- 172.21.15.92:0/3515820049 >> 172.21.15.92:0/3515820049 conn(0x7fe90c079770 msgr2=0x7fe90c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 -- 172.21.15.92:0/3515820049 shutdown_connections 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.026+0000 7fe913778640 1 -- 172.21.15.92:0/3515820049 wait complete. 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 Processor -- start 2024-09-21T02:02:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 -- start start 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108420 0x7fe90c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe90c108d70 0x7fe90c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c112a30 con 0x7fe90c102ae0 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c1128b0 con 0x7fe90c108420 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe913778640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c112bb0 con 0x7fe90c108d70 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.027+0000 7fe911514640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50096/0 (socket says 172.21.15.92:50096) 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 -- 172.21.15.92:0/748074398 learned_addr learned my addr 172.21.15.92:0/748074398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911d15640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe90c108d70 0x7fe90c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe910d13640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108420 0x7fe90c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 -- 172.21.15.92:0/748074398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe90c108d70 msgr2=0x7fe90c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe90c108d70 0x7fe90c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 -- 172.21.15.92:0/748074398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108420 msgr2=0x7fe90c19b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108420 0x7fe90c19b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 -- 172.21.15.92:0/748074398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c1a65c0 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe910d13640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108420 0x7fe90c19b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe911514640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c19b300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005940 tx=0x7fe8fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc03f030 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8fc003cb0 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c1a6850 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc035070 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.028+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1a6d80 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.029+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c103d60 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.032+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fe8fc042030 con 0x7fe90c102ae0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.033+0000 7fe9027fc640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8f80b6590 0x7fe8f80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.033+0000 7fe910d13640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8f80b6590 0x7fe8f80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.033+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fe8fc1079b0 con 0x7fe90c102ae0 2024-09-21T02:02:07.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.033+0000 7fe910d13640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8f80b6590 0x7fe8f80b8aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe90c1053d0 tx=0x7fe908005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:07.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.036+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe8fc0ce6e0 con 0x7fe90c102ae0 2024-09-21T02:02:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: Deploying daemon mds.c on smithi136 2024-09-21T02:02:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] up:boot 2024-09-21T02:02:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: fsmap 6 up:standby 2024-09-21T02:02:07.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:06 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:02:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.249+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe90c102ef0 con 0x7fe90c102ae0 2024-09-21T02:02:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.249+0000 7fe9027fc640 1 -- 172.21.15.92:0/748074398 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5557 (secure 0 0 0) 0x7fe8fc0d2ac0 con 0x7fe90c102ae0 2024-09-21T02:02:07.252 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:07.252 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"btime":"2024-09-21T02:02:06:334680+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24610,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T02:02:07.252 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2024-09-21T02:02:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8f80b6590 msgr2=0x7fe8f80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8f80b6590 0x7fe8f80b8aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe90c1053d0 tx=0x7fe908005d30 comp rx=0 tx=0).stop 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 msgr2=0x7fe90c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c19b300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005940 tx=0x7fe8fc0042f0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 shutdown_connections 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8f80b6590 0x7fe8f80b8aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe90c108d70 0x7fe90c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe90c108420 0x7fe90c19b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 --2- 172.21.15.92:0/748074398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe90c102ae0 0x7fe90c19b300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 >> 172.21.15.92:0/748074398 conn(0x7fe90c079770 msgr2=0x7fe90c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 shutdown_connections 2024-09-21T02:02:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:07.252+0000 7fe913778640 1 -- 172.21.15.92:0/748074398 wait complete. 2024-09-21T02:02:07.734 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:07 smithi136 systemd[1]: Starting Ceph mds.c for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:07.968 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:07 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/748074398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:08.018 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:07 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/748074398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:08.018 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:07 smithi136 podman[63486]: 2024-09-21 02:02:07.635983516 +0000 UTC m=+0.016038224 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:08.018 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:07 smithi136 podman[63486]: 2024-09-21 02:02:07.773560548 +0000 UTC m=+0.153615247 container create 66597b4ff0b7d9a1c5897e0fb601879fb866b45d460b4c999ce5c8c65be0df22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T02:02:08.018 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:08 smithi136 podman[63486]: 2024-09-21 02:02:08.01824274 +0000 UTC m=+0.398297437 container init 66597b4ff0b7d9a1c5897e0fb601879fb866b45d460b4c999ce5c8c65be0df22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T02:02:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:07 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/748074398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:08.283 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:08 smithi136 podman[63486]: 2024-09-21 02:02:08.021109829 +0000 UTC m=+0.401164527 container start 66597b4ff0b7d9a1c5897e0fb601879fb866b45d460b4c999ce5c8c65be0df22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T02:02:08.283 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mds[63505]: -- 172.21.15.136:0/2373646431 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563e7907860 con 0x5563ebaafa80 2024-09-21T02:02:08.283 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c[63501]: starting mds.c at 2024-09-21T02:02:08.283 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:08 smithi136 bash[63486]: 66597b4ff0b7d9a1c5897e0fb601879fb866b45d460b4c999ce5c8c65be0df22 2024-09-21T02:02:08.284 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:02:08 smithi136 systemd[1]: Started Ceph mds.c for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:08.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:08.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:08.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:08.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:08.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:08.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:08.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:08 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:09.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:08 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:09.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:09.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:09.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:08 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:09.897 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:09 smithi099 ceph-mon[34404]: Deploying daemon mds.h on smithi099 2024-09-21T02:02:09.897 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:09 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] up:boot 2024-09-21T02:02:09.897 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:09 smithi099 ceph-mon[34404]: fsmap 7 up:standby 2024-09-21T02:02:09.897 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:09 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:02:10.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:10.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:09 smithi136 ceph-mon[34754]: Deploying daemon mds.h on smithi099 2024-09-21T02:02:10.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:09 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] up:boot 2024-09-21T02:02:10.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:09 smithi136 ceph-mon[34754]: fsmap 7 up:standby 2024-09-21T02:02:10.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:09 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:02:10.292 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 systemd[1]: Starting Ceph mds.h for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:10.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:09 smithi092 ceph-mon[31190]: Deploying daemon mds.h on smithi099 2024-09-21T02:02:10.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:09 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] up:boot 2024-09-21T02:02:10.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:09 smithi092 ceph-mon[31190]: fsmap 7 up:standby 2024-09-21T02:02:10.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:09 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:02:10.575 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 podman[62128]: 2024-09-21 02:02:10.194999269 +0000 UTC m=+0.015632878 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:10.575 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 podman[62128]: 2024-09-21 02:02:10.30304753 +0000 UTC m=+0.123681126 container create 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T02:02:10.575 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 podman[62128]: 2024-09-21 02:02:10.538874859 +0000 UTC m=+0.359508456 container init 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T02:02:10.575 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 podman[62128]: 2024-09-21 02:02:10.541705916 +0000 UTC m=+0.362339522 container start 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 -- 172.21.15.92:0/2909355144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9854108d20 msgr2=0x7f985410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 --2- 172.21.15.92:0/2909355144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9854108d20 0x7f985410f550 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f984c009900 tx=0x7f984c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 -- 172.21.15.92:0/2909355144 shutdown_connections 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 --2- 172.21.15.92:0/2909355144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9854108d20 0x7f985410f550 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 --2- 172.21.15.92:0/2909355144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98541083d0 0x7f98541087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 --2- 172.21.15.92:0/2909355144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9854102ae0 0x7f9854102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 -- 172.21.15.92:0/2909355144 >> 172.21.15.92:0/2909355144 conn(0x7f98540797b0 msgr2=0x7f9854079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 -- 172.21.15.92:0/2909355144 shutdown_connections 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.651+0000 7f985b598640 1 -- 172.21.15.92:0/2909355144 wait complete. 2024-09-21T02:02:10.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.652+0000 7f985b598640 1 Processor -- start 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.652+0000 7f985b598640 1 -- start start 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9854102ae0 0x7f9854199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 0x7f98541998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9854108d20 0x7f985419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98541129e0 con 0x7f98541083d0 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9854112860 con 0x7f9854102ae0 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9854112b60 con 0x7f9854108d20 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 0x7f98541998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 0x7f98541998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50118/0 (socket says 172.21.15.92:50118) 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 -- 172.21.15.92:0/919692855 learned_addr learned my addr 172.21.15.92:0/919692855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9859334640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9854102ae0 0x7f9854199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9859b35640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9854108d20 0x7f985419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 -- 172.21.15.92:0/919692855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9854108d20 msgr2=0x7f985419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9854108d20 0x7f985419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 -- 172.21.15.92:0/919692855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9854102ae0 msgr2=0x7f9854199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9854102ae0 0x7f9854199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 -- 172.21.15.92:0/919692855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985419f280 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f9858b33640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 0x7f98541998b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f984800d750 tx=0x7f984800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848017070 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98480054d0 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985419f570 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848004c30 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.653+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541a5cd0 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.655+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9854102ef0 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.656+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f9848004dd0 con 0x7f98541083d0 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.657+0000 7f98527fc640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f983c0b6510 0x7f983c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:10.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.657+0000 7f9859334640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f983c0b6510 0x7f983c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:10.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.657+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f98480e67a0 con 0x7f98541083d0 2024-09-21T02:02:10.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.657+0000 7f9859334640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f983c0b6510 0x7f983c0b8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9844005fd0 tx=0x7f9844009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:10.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.661+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98480ad4d0 con 0x7f98541083d0 2024-09-21T02:02:10.831 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mds[62147]: -- 172.21.15.99:0/303318420 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fc8abd860 con 0x557fccc65a80 2024-09-21T02:02:10.831 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h[62143]: starting mds.h at 2024-09-21T02:02:10.831 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 bash[62128]: 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:02:10 smithi099 systemd[1]: Started Ceph mds.h for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:10.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:10 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:10.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.900+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f985410ab70 con 0x7f98541083d0 2024-09-21T02:02:10.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.900+0000 7f98527fc640 1 -- 172.21.15.92:0/919692855 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f98480b18b0 con 0x7f98541083d0 2024-09-21T02:02:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:10.902 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"btime":"2024-09-21T02:02:10:806340+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T02:02:10.902 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2024-09-21T02:02:10.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f983c0b6510 msgr2=0x7f983c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f983c0b6510 0x7f983c0b8a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9844005fd0 tx=0x7f9844009760 comp rx=0 tx=0).stop 2024-09-21T02:02:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 msgr2=0x7f98541998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:10.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 0x7f98541998b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f984800d750 tx=0x7f984800dc10 comp rx=0 tx=0).stop 2024-09-21T02:02:10.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:10.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:10.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:10.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:10.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:10.904 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:10.905 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:10 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 shutdown_connections 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f983c0b6510 0x7f983c0b8a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9854108d20 0x7f985419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98541083d0 0x7f98541998b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 --2- 172.21.15.92:0/919692855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9854102ae0 0x7f9854199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 >> 172.21.15.92:0/919692855 conn(0x7f98540797b0 msgr2=0x7f9854106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 shutdown_connections 2024-09-21T02:02:10.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:10.903+0000 7f985b598640 1 -- 172.21.15.92:0/919692855 wait complete. 2024-09-21T02:02:11.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:11.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:11.108 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:11.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:11.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:11.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:11.109 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:10 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:11 smithi099 ceph-mon[34404]: Deploying daemon mds.i on smithi136 2024-09-21T02:02:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:11 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6836/3685475120,v1:172.21.15.99:6837/3685475120] up:boot 2024-09-21T02:02:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:11 smithi099 ceph-mon[34404]: fsmap 8 up:standby 2024-09-21T02:02:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:02:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:11 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/919692855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:12.124 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:11 smithi136 ceph-mon[34754]: Deploying daemon mds.i on smithi136 2024-09-21T02:02:12.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:11 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6836/3685475120,v1:172.21.15.99:6837/3685475120] up:boot 2024-09-21T02:02:12.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:11 smithi136 ceph-mon[34754]: fsmap 8 up:standby 2024-09-21T02:02:12.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:02:12.125 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:11 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/919692855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:12.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:11 smithi092 ceph-mon[31190]: Deploying daemon mds.i on smithi136 2024-09-21T02:02:12.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:11 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6836/3685475120,v1:172.21.15.99:6837/3685475120] up:boot 2024-09-21T02:02:12.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:11 smithi092 ceph-mon[31190]: fsmap 8 up:standby 2024-09-21T02:02:12.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:02:12.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:11 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/919692855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:12.784 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:12 smithi136 systemd[1]: Starting Ceph mds.i for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:12 smithi099 ceph-mon[34404]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:13.105 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:12 smithi136 ceph-mon[34754]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:13.105 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:12 smithi136 podman[63752]: 2024-09-21 02:02:12.738740431 +0000 UTC m=+0.015726437 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:13.106 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:12 smithi136 podman[63752]: 2024-09-21 02:02:12.886143526 +0000 UTC m=+0.163129582 container create 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:02:13.106 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:13 smithi136 podman[63752]: 2024-09-21 02:02:13.105699764 +0000 UTC m=+0.382685757 container init 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:02:13.206 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:12 smithi092 ceph-mon[31190]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:13.533 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:13 smithi136 podman[63752]: 2024-09-21 02:02:13.108531544 +0000 UTC m=+0.385517538 container start 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:02:13.534 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:13 smithi136 ceph-mds[63771]: -- 172.21.15.136:0/3689437518 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e621b99860 con 0x55e625d41600 2024-09-21T02:02:13.534 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:13 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: starting mds.i at 2024-09-21T02:02:13.534 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:13 smithi136 bash[63752]: 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef 2024-09-21T02:02:13.534 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:13 smithi136 systemd[1]: Started Ceph mds.i for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:13.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: Deploying daemon mds.g on smithi092 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] up:boot 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: fsmap 9 up:standby 2024-09-21T02:02:14.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:02:14.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:14.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: Deploying daemon mds.g on smithi092 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] up:boot 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: fsmap 9 up:standby 2024-09-21T02:02:14.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: Deploying daemon mds.g on smithi092 2024-09-21T02:02:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] up:boot 2024-09-21T02:02:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: fsmap 9 up:standby 2024-09-21T02:02:14.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:02:14.752 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:15.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 -- 172.21.15.92:0/3644181476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc1094b0 msgr2=0x7f90fc1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 --2- 172.21.15.92:0/3644181476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc1094b0 0x7f90fc1098c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e402a410 comp rx=0 tx=0).stop 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 -- 172.21.15.92:0/3644181476 shutdown_connections 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 --2- 172.21.15.92:0/3644181476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90fc109e00 0x7f90fc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 --2- 172.21.15.92:0/3644181476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc1094b0 0x7f90fc1098c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 --2- 172.21.15.92:0/3644181476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90fc103bc0 0x7f90fc103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 -- 172.21.15.92:0/3644181476 >> 172.21.15.92:0/3644181476 conn(0x7f90fc07b910 msgr2=0x7f90fc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 -- 172.21.15.92:0/3644181476 shutdown_connections 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.375+0000 7f91011c1640 1 -- 172.21.15.92:0/3644181476 wait complete. 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.376+0000 7f91011c1640 1 Processor -- start 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.376+0000 7f91011c1640 1 -- start start 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f91011c1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90fc103bc0 0x7f90fc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:15.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f91011c1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90fc1094b0 0x7f90fc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f91011c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 0x7f90fc19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f91011c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc113b60 con 0x7f90fc109e00 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f91011c1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc1139e0 con 0x7f90fc103bc0 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f91011c1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc113ce0 con 0x7f90fc1094b0 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 0x7f90fc19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 0x7f90fc19fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50114/0 (socket says 172.21.15.92:50114) 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 -- 172.21.15.92:0/3863196604 learned_addr learned my addr 172.21.15.92:0/3863196604 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fad9d640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90fc103bc0 0x7f90fc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fa59c640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90fc1094b0 0x7f90fc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 -- 172.21.15.92:0/3863196604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90fc1094b0 msgr2=0x7f90fc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90fc1094b0 0x7f90fc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 -- 172.21.15.92:0/3863196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90fc103bc0 msgr2=0x7f90fc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90fc103bc0 0x7f90fc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.377+0000 7f90fb59e640 1 -- 172.21.15.92:0/3863196604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc1a0580 con 0x7f90fc109e00 2024-09-21T02:02:15.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.378+0000 7f90fb59e640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 0x7f90fc19fe50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b970 tx=0x7f90f400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:15.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.378+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4019070 con 0x7f90fc109e00 2024-09-21T02:02:15.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.378+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f4014440 con 0x7f90fc109e00 2024-09-21T02:02:15.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.378+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc1a0870 con 0x7f90fc109e00 2024-09-21T02:02:15.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.378+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40075d0 con 0x7f90fc109e00 2024-09-21T02:02:15.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.378+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc1a6fd0 con 0x7f90fc109e00 2024-09-21T02:02:15.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.379+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc104050 con 0x7f90fc109e00 2024-09-21T02:02:15.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.381+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f90f401c070 con 0x7f90fc109e00 2024-09-21T02:02:15.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.382+0000 7f90f3fff640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f90e80b6510 0x7f90e80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:15.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.382+0000 7f90fad9d640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f90e80b6510 0x7f90e80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:15.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.382+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f90f40e67b0 con 0x7f90fc109e00 2024-09-21T02:02:15.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.382+0000 7f90fad9d640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f90e80b6510 0x7f90e80b8a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009990 tx=0x7f90ec009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:15.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.385+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90f40ad4e0 con 0x7f90fc109e00 2024-09-21T02:02:15.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.619+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90fc19b740 con 0x7f90fc109e00 2024-09-21T02:02:15.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.619+0000 7f90f3fff640 1 -- 172.21.15.92:0/3863196604 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8037 (secure 0 0 0) 0x7f90fc19b740 con 0x7f90fc109e00 2024-09-21T02:02:15.620 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:15.620 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"btime":"2024-09-21T02:02:13:822932+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T02:02:15.620 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 10 2024-09-21T02:02:15.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f90e80b6510 msgr2=0x7f90e80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:15.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f90e80b6510 0x7f90e80b8a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009990 tx=0x7f90ec009110 comp rx=0 tx=0).stop 2024-09-21T02:02:15.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 msgr2=0x7f90fc19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:15.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 0x7f90fc19fe50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b970 tx=0x7f90f400bd70 comp rx=0 tx=0).stop 2024-09-21T02:02:15.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 shutdown_connections 2024-09-21T02:02:15.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f90e80b6510 0x7f90e80b8a20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90fc109e00 0x7f90fc19fe50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90fc1094b0 0x7f90fc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.621+0000 7f91011c1640 1 --2- 172.21.15.92:0/3863196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90fc103bc0 0x7f90fc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.622+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 >> 172.21.15.92:0/3863196604 conn(0x7f90fc07b910 msgr2=0x7f90fc1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.622+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 shutdown_connections 2024-09-21T02:02:15.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:15.622+0000 7f91011c1640 1 -- 172.21.15.92:0/3863196604 wait complete. 2024-09-21T02:02:16.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:16 smithi092 ceph-mon[31190]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:16.526 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:16 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3863196604' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:16.526 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:02:16 smithi092 systemd[1]: Started Ceph mds.g for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:16 smithi136 ceph-mon[34754]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:16 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3863196604' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:16 smithi099 ceph-mon[34404]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:16 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3863196604' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:17.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:17.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:17.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:17.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: Deploying daemon mds.a on smithi092 2024-09-21T02:02:17.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6838/4068035941,v1:172.21.15.92:6839/4068035941] up:boot 2024-09-21T02:02:17.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: fsmap 10 up:standby 2024-09-21T02:02:17.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:17 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: Deploying daemon mds.a on smithi092 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6838/4068035941,v1:172.21.15.92:6839/4068035941] up:boot 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: fsmap 10 up:standby 2024-09-21T02:02:17.799 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:17 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:02:17.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:17.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:17.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:17.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:17.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: Deploying daemon mds.a on smithi092 2024-09-21T02:02:17.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6838/4068035941,v1:172.21.15.92:6839/4068035941] up:boot 2024-09-21T02:02:17.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: fsmap 10 up:standby 2024-09-21T02:02:17.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:17 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:02:18.450 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 systemd[1]: Starting Ceph mds.a for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:18.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:18.720 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:18 smithi092 ceph-mon[31190]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:18.720 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 podman[72577]: 2024-09-21 02:02:18.38178051 +0000 UTC m=+0.015749340 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:18.720 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 podman[72577]: 2024-09-21 02:02:18.49303203 +0000 UTC m=+0.127000836 container create acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T02:02:18.720 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 podman[72577]: 2024-09-21 02:02:18.720371079 +0000 UTC m=+0.354339889 container init acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T02:02:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:18 smithi136 ceph-mon[34754]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:18.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:18 smithi099 ceph-mon[34404]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:18.947 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:19.054 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 podman[72577]: 2024-09-21 02:02:18.723486021 +0000 UTC m=+0.357454826 container start acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T02:02:19.054 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 ceph-mds[72614]: -- 172.21.15.92:0/3339448370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563826ef5860 con 0x56382b09d600 2024-09-21T02:02:19.054 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a[72610]: starting mds.a at 2024-09-21T02:02:19.054 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 bash[72577]: acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 2024-09-21T02:02:19.054 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:02:18 smithi092 systemd[1]: Started Ceph mds.a for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:19.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 -- 172.21.15.92:0/1299621090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 msgr2=0x7f0f90109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:19.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 --2- 172.21.15.92:0/1299621090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f90109900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009900 tx=0x7f0f8002a410 comp rx=0 tx=0).stop 2024-09-21T02:02:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 -- 172.21.15.92:0/1299621090 shutdown_connections 2024-09-21T02:02:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 --2- 172.21.15.92:0/1299621090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90109e40 0x7f0f90110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 --2- 172.21.15.92:0/1299621090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f90109900 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 --2- 172.21.15.92:0/1299621090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90103bb0 0x7f0f90103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 -- 172.21.15.92:0/1299621090 >> 172.21.15.92:0/1299621090 conn(0x7f0f900ff600 msgr2=0x7f0f90101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 -- 172.21.15.92:0/1299621090 shutdown_connections 2024-09-21T02:02:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.446+0000 7f0f97639640 1 -- 172.21.15.92:0/1299621090 wait complete. 2024-09-21T02:02:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.447+0000 7f0f97639640 1 Processor -- start 2024-09-21T02:02:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.447+0000 7f0f97639640 1 -- start start 2024-09-21T02:02:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.447+0000 7f0f97639640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90103bb0 0x7f0f9019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.447+0000 7f0f97639640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f9019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.447+0000 7f0f97639640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90109e40 0x7f0f9019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f97639640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90113b20 con 0x7f0f901094f0 2024-09-21T02:02:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f97639640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f901139a0 con 0x7f0f90109e40 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f97639640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90113ca0 con 0x7f0f90103bb0 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f9019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f9019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50168/0 (socket says 172.21.15.92:50168) 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 -- 172.21.15.92:0/1996514760 learned_addr learned my addr 172.21.15.92:0/1996514760 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f95bd6640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90109e40 0x7f0f9019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f953d5640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90103bb0 0x7f0f9019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 -- 172.21.15.92:0/1996514760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90103bb0 msgr2=0x7f0f9019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90103bb0 0x7f0f9019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 -- 172.21.15.92:0/1996514760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90109e40 msgr2=0x7f0f9019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90109e40 0x7f0f9019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 -- 172.21.15.92:0/1996514760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f901a0500 con 0x7f0f901094f0 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f953d5640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90103bb0 0x7f0f9019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f95bd6640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90109e40 0x7f0f9019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f94bd4640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f9019ab30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002a930 tx=0x7f0f80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.448+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8003f030 con 0x7f0f901094f0 2024-09-21T02:02:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.449+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f80003aa0 con 0x7f0f901094f0 2024-09-21T02:02:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.449+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f901a0790 con 0x7f0f901094f0 2024-09-21T02:02:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.449+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80035070 con 0x7f0f901094f0 2024-09-21T02:02:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.449+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f901a6f50 con 0x7f0f901094f0 2024-09-21T02:02:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.450+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f90104e30 con 0x7f0f901094f0 2024-09-21T02:02:19.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.451+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f0f80042030 con 0x7f0f901094f0 2024-09-21T02:02:19.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.452+0000 7f0f867fc640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0f7c0b6510 0x7f0f7c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:19.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.452+0000 7f0f953d5640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0f7c0b6510 0x7f0f7c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.452+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f0f80107930 con 0x7f0f901094f0 2024-09-21T02:02:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.452+0000 7f0f953d5640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0f7c0b6510 0x7f0f7c0b8a20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c005fd0 tx=0x7f0f8c00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:19.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.455+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f800ce660 con 0x7f0f901094f0 2024-09-21T02:02:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.670+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f90103fc0 con 0x7f0f901094f0 2024-09-21T02:02:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.670+0000 7f0f867fc640 1 -- 172.21.15.92:0/1996514760 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7f0f90103fc0 con 0x7f0f901094f0 2024-09-21T02:02:19.672 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:19.672 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"btime":"2024-09-21T02:02:19:450583+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14793,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T02:02:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2024-09-21T02:02:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0f7c0b6510 msgr2=0x7f0f7c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0f7c0b6510 0x7f0f7c0b8a20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c005fd0 tx=0x7f0f8c00b5f0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 msgr2=0x7f0f9019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f9019ab30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002a930 tx=0x7f0f80004270 comp rx=0 tx=0).stop 2024-09-21T02:02:19.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 shutdown_connections 2024-09-21T02:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0f7c0b6510 0x7f0f7c0b8a20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f90109e40 0x7f0f9019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f901094f0 0x7f0f9019ab30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 --2- 172.21.15.92:0/1996514760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f90103bb0 0x7f0f9019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 >> 172.21.15.92:0/1996514760 conn(0x7f0f900ff600 msgr2=0x7f0f90101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 shutdown_connections 2024-09-21T02:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:19.673+0000 7f0f97639640 1 -- 172.21.15.92:0/1996514760 wait complete. 2024-09-21T02:02:20.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: Deploying daemon mds.b on smithi099 2024-09-21T02:02:20.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] up:boot 2024-09-21T02:02:20.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: fsmap 11 up:standby 2024-09-21T02:02:20.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:02:20.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:19 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1996514760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:20.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:20.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: Deploying daemon mds.b on smithi099 2024-09-21T02:02:20.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] up:boot 2024-09-21T02:02:20.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: fsmap 11 up:standby 2024-09-21T02:02:20.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:02:20.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:19 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1996514760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:02:20.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:02:20.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:20.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: Deploying daemon mds.b on smithi099 2024-09-21T02:02:20.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] up:boot 2024-09-21T02:02:20.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: fsmap 11 up:standby 2024-09-21T02:02:20.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:02:20.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:19 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1996514760' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:20.949 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:20 smithi099 systemd[1]: Starting Ceph mds.b for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:02:21.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:20 smithi099 ceph-mon[34404]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:21.276 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 podman[62403]: 2024-09-21 02:02:20.941318885 +0000 UTC m=+0.015652733 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:02:21.276 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 podman[62403]: 2024-09-21 02:02:21.065158575 +0000 UTC m=+0.139492426 container create 4186d3a6afba9505a296c13dc3b465db0f7f6e633e5178bc6ee4e4ec6bc959e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T02:02:21.276 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 podman[62403]: 2024-09-21 02:02:21.276332602 +0000 UTC m=+0.350666450 container init 4186d3a6afba9505a296c13dc3b465db0f7f6e633e5178bc6ee4e4ec6bc959e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T02:02:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:20 smithi136 ceph-mon[34754]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:20 smithi092 ceph-mon[31190]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:21.581 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 podman[62403]: 2024-09-21 02:02:21.279133468 +0000 UTC m=+0.353467317 container start 4186d3a6afba9505a296c13dc3b465db0f7f6e633e5178bc6ee4e4ec6bc959e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:02:21.581 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 ceph-mds[62422]: -- 172.21.15.99:0/3781217020 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56254aa8f860 con 0x56254ec37a80 2024-09-21T02:02:21.581 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b[62418]: starting mds.b at 2024-09-21T02:02:21.581 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 bash[62403]: 4186d3a6afba9505a296c13dc3b465db0f7f6e633e5178bc6ee4e4ec6bc959e1 2024-09-21T02:02:21.581 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:02:21 smithi099 systemd[1]: Started Ceph mds.b for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:02:22.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] up:boot 2024-09-21T02:02:22.692 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: fsmap 12 up:standby 2024-09-21T02:02:22.692 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:22 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:02:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:22.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] up:boot 2024-09-21T02:02:22.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: fsmap 12 up:standby 2024-09-21T02:02:22.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:22 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:02:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] up:boot 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: fsmap 12 up:standby 2024-09-21T02:02:22.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:22 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:02:23.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 -- 172.21.15.92:0/1057011336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4108cf0 msgr2=0x7f84b410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1057011336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4108cf0 0x7f84b410f520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009900 tx=0x7f84ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 -- 172.21.15.92:0/1057011336 shutdown_connections 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1057011336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4108cf0 0x7f84b410f520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1057011336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b41083a0 0x7f84b41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1057011336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b4102a60 0x7f84b4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 -- 172.21.15.92:0/1057011336 >> 172.21.15.92:0/1057011336 conn(0x7f84b4079730 msgr2=0x7f84b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 -- 172.21.15.92:0/1057011336 shutdown_connections 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.047+0000 7f84b9d43640 1 -- 172.21.15.92:0/1057011336 wait complete. 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.048+0000 7f84b9d43640 1 Processor -- start 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.048+0000 7f84b9d43640 1 -- start start 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 0x7f84b419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b41083a0 0x7f84b419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b4108cf0 0x7f84b419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4113a80 con 0x7f84b4102a60 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4113900 con 0x7f84b41083a0 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4113c00 con 0x7f84b4108cf0 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 0x7f84b419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 0x7f84b419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56594/0 (socket says 172.21.15.92:56594) 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 -- 172.21.15.92:0/1220738357 learned_addr learned my addr 172.21.15.92:0/1220738357 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b2ffd640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b41083a0 0x7f84b419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b3fff640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b4108cf0 0x7f84b419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 -- 172.21.15.92:0/1220738357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b4108cf0 msgr2=0x7f84b419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b4108cf0 0x7f84b419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 -- 172.21.15.92:0/1220738357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b41083a0 msgr2=0x7f84b419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b41083a0 0x7f84b419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 -- 172.21.15.92:0/1220738357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b41a0380 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b37fe640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 0x7f84b419a410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84a400ebc0 tx=0x7f84a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4010630 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41a0670 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84a4010c80 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.049+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4018b60 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.050+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a6dd0 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.050+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b4102e70 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.052+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f84a4018d00 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.053+0000 7f84b0ff9640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f84a00b6510 0x7f84a00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.053+0000 7f84b2ffd640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f84a00b6510 0x7f84a00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.053+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f84a40e78b0 con 0x7f84b4102a60 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.053+0000 7f84b2ffd640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f84a00b6510 0x7f84a00b8a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84b4107100 tx=0x7f849c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.056+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84a40b29c0 con 0x7f84b4102a60 2024-09-21T02:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.320+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84b419b540 con 0x7f84b4102a60 2024-09-21T02:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.321+0000 7f84b0ff9640 1 -- 172.21.15.92:0/1220738357 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10515 (secure 0 0 0) 0x7f84a40b7900 con 0x7f84b4102a60 2024-09-21T02:02:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:23.322 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":13,"btime":"2024-09-21T02:02:21:947036+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14793,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24551,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24593,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24610,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T02:02:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 13 2024-09-21T02:02:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f84a00b6510 msgr2=0x7f84a00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f84a00b6510 0x7f84a00b8a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84b4107100 tx=0x7f849c005f90 comp rx=0 tx=0).stop 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 msgr2=0x7f84b419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 0x7f84b419a410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84a400ebc0 tx=0x7f84a400c6a0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 shutdown_connections 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f84a00b6510 0x7f84a00b8a20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b4108cf0 0x7f84b419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b41083a0 0x7f84b419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 --2- 172.21.15.92:0/1220738357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b4102a60 0x7f84b419a410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 >> 172.21.15.92:0/1220738357 conn(0x7f84b4079730 msgr2=0x7f84b4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 shutdown_connections 2024-09-21T02:02:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:23.324+0000 7f84b9d43640 1 -- 172.21.15.92:0/1220738357 wait complete. 2024-09-21T02:02:24.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1220738357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:23 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1220738357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:23 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.308 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1220738357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:23 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 -- 172.21.15.92:0/546425440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 msgr2=0x7fefe810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:24.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 --2- 172.21.15.92:0/546425440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe810f520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009ca0 tx=0x7fefe002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 -- 172.21.15.92:0/546425440 shutdown_connections 2024-09-21T02:02:24.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 --2- 172.21.15.92:0/546425440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe810f520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 --2- 172.21.15.92:0/546425440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe81083a0 0x7fefe81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 --2- 172.21.15.92:0/546425440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8102a60 0x7fefe8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 -- 172.21.15.92:0/546425440 >> 172.21.15.92:0/546425440 conn(0x7fefe8079730 msgr2=0x7fefe8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 -- 172.21.15.92:0/546425440 shutdown_connections 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.774+0000 7fefef173640 1 -- 172.21.15.92:0/546425440 wait complete. 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 Processor -- start 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 -- start start 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe8102a60 0x7fefe81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe81083a0 0x7fefe8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8112960 con 0x7fefe8108cf0 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fefe81127e0 con 0x7fefe81083a0 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefef173640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8112ae0 con 0x7fefe8102a60 2024-09-21T02:02:24.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefed710640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefed710640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56606/0 (socket says 172.21.15.92:56606) 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefed710640 1 -- 172.21.15.92:0/4013080408 learned_addr learned my addr 172.21.15.92:0/4013080408 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefe7fff640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe81083a0 0x7fefe8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.775+0000 7fefecf0f640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe8102a60 0x7fefe81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefed710640 1 -- 172.21.15.92:0/4013080408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe8102a60 msgr2=0x7fefe81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefed710640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe8102a60 0x7fefe81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefed710640 1 -- 172.21.15.92:0/4013080408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe81083a0 msgr2=0x7fefe8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefed710640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe81083a0 0x7fefe8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefed710640 1 -- 172.21.15.92:0/4013080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe819f260 con 0x7fefe8108cf0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefed710640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe819eb30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fefe0005cc0 tx=0x7fefe002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0003bf0 con 0x7fefe8108cf0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe0003d90 con 0x7fefe8108cf0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe819f4f0 con 0x7fefe8108cf0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe002fbc0 con 0x7fefe8108cf0 2024-09-21T02:02:24.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.776+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81a5cb0 con 0x7fefe8108cf0 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.777+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe8102e70 con 0x7fefe8108cf0 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.779+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fefe002fd60 con 0x7fefe8108cf0 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.779+0000 7fefe5ffb640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fefd40b6510 0x7fefd40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.780+0000 7fefecf0f640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fefd40b6510 0x7fefd40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.780+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fefe0108470 con 0x7fefe8108cf0 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.780+0000 7fefecf0f640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fefd40b6510 0x7fefd40b8a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefd800bbe0 tx=0x7fefd800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:24.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.783+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fefe00cf180 con 0x7fefe8108cf0 2024-09-21T02:02:24.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.981+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fefe819a420 con 0x7fefe8108cf0 2024-09-21T02:02:24.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:24 smithi092 ceph-mon[31190]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:24.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:24.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:24.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:24.982 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:24 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.981+0000 7fefe5ffb640 1 -- 172.21.15.92:0/4013080408 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fefe819a420 con 0x7fefe8108cf0 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:02:24.983 INFO:teuthology.orchestra.run.smithi092.stdout:] 2024-09-21T02:02:24.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fefd40b6510 msgr2=0x7fefd40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:24.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fefd40b6510 0x7fefd40b8a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefd800bbe0 tx=0x7fefd800b390 comp rx=0 tx=0).stop 2024-09-21T02:02:24.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 msgr2=0x7fefe819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe819eb30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fefe0005cc0 tx=0x7fefe002f380 comp rx=0 tx=0).stop 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 shutdown_connections 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fefd40b6510 0x7fefd40b8a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe8108cf0 0x7fefe819eb30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe81083a0 0x7fefe8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 --2- 172.21.15.92:0/4013080408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe8102a60 0x7fefe81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.984+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 >> 172.21.15.92:0/4013080408 conn(0x7fefe8079730 msgr2=0x7fefe81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.985+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 shutdown_connections 2024-09-21T02:02:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:24.985+0000 7fefef173640 1 -- 172.21.15.92:0/4013080408 wait complete. 2024-09-21T02:02:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:24 smithi136 ceph-mon[34754]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:24 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:24 smithi099 ceph-mon[34404]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:24 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:02:25.925 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:02:25.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:02:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4013080408' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:02:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:25 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4013080408' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:02:26.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:25 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4013080408' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:02:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:25 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 -- 172.21.15.92:0/1216056512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8105380 msgr2=0x7f71d810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 --2- 172.21.15.92:0/1216056512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8105380 0x7f71d810d8d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009900 tx=0x7f71cc02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 -- 172.21.15.92:0/1216056512 shutdown_connections 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 --2- 172.21.15.92:0/1216056512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 0x7f71d8110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 --2- 172.21.15.92:0/1216056512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8105380 0x7f71d810d8d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 --2- 172.21.15.92:0/1216056512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71d8104a30 0x7f71d8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 -- 172.21.15.92:0/1216056512 >> 172.21.15.92:0/1216056512 conn(0x7f71d8079770 msgr2=0x7f71d8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 -- 172.21.15.92:0/1216056512 shutdown_connections 2024-09-21T02:02:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.853+0000 7f71dc87b640 1 -- 172.21.15.92:0/1216056512 wait complete. 2024-09-21T02:02:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 Processor -- start 2024-09-21T02:02:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 -- start start 2024-09-21T02:02:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 0x7f71d819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71d8105380 0x7f71d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 0x7f71d819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8113af0 con 0x7f71d8104a30 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8113970 con 0x7f71d810de90 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71dc87b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8113c70 con 0x7f71d8105380 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 0x7f71d819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 0x7f71d819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56616/0 (socket says 172.21.15.92:56616) 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 -- 172.21.15.92:0/512636526 learned_addr learned my addr 172.21.15.92:0/512636526 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d5d9b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71d8105380 0x7f71d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d6d9d640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 0x7f71d819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 -- 172.21.15.92:0/512636526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71d8105380 msgr2=0x7f71d819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71d8105380 0x7f71d819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 -- 172.21.15.92:0/512636526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 msgr2=0x7f71d819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 0x7f71d819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d659c640 1 -- 172.21.15.92:0/512636526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81a0430 con 0x7f71d8104a30 2024-09-21T02:02:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.854+0000 7f71d6d9d640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 0x7f71d819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.855+0000 7f71d659c640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 0x7f71d819a580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f71c800c920 tx=0x7f71c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.855+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8007c70 con 0x7f71d8104a30 2024-09-21T02:02:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.855+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d81a0720 con 0x7f71d8104a30 2024-09-21T02:02:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.855+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c8007e10 con 0x7f71d8104a30 2024-09-21T02:02:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.855+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8018820 con 0x7f71d8104a30 2024-09-21T02:02:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.856+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d81a6f40 con 0x7f71d8104a30 2024-09-21T02:02:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.857+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d806c260 con 0x7f71d8104a30 2024-09-21T02:02:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.858+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f71c800f450 con 0x7f71d8104a30 2024-09-21T02:02:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.858+0000 7f71c77fe640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c00b6540 0x7f71c00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.858+0000 7f71d5d9b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c00b6540 0x7f71c00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.858+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f71c80e7050 con 0x7f71d8104a30 2024-09-21T02:02:26.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.859+0000 7f71d5d9b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c00b6540 0x7f71c00b8a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009230 tx=0x7f71cc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:26.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:26.863+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71c8014030 con 0x7f71d8104a30 2024-09-21T02:02:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.079+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f71d8108f90 con 0x7f71d8104a30 2024-09-21T02:02:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:26 smithi136 ceph-mon[34754]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:26 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:26 smithi092 ceph-mon[31190]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:26 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:26 smithi099 ceph-mon[34404]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:26 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.936+0000 7f71c77fe640 1 -- 172.21.15.92:0/512636526 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f71c80add80 con 0x7f71d8104a30 2024-09-21T02:02:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c00b6540 msgr2=0x7f71c00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c00b6540 0x7f71c00b8a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009230 tx=0x7f71cc031400 comp rx=0 tx=0).stop 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 msgr2=0x7f71d819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 0x7f71d819a580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f71c800c920 tx=0x7f71c800cde0 comp rx=0 tx=0).stop 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 shutdown_connections 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c00b6540 0x7f71c00b8a50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71d810de90 0x7f71d819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71d8105380 0x7f71d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:27.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 --2- 172.21.15.92:0/512636526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8104a30 0x7f71d819a580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 >> 172.21.15.92:0/512636526 conn(0x7f71d8079770 msgr2=0x7f71d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 shutdown_connections 2024-09-21T02:02:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:27.939+0000 7f71dc87b640 1 -- 172.21.15.92:0/512636526 wait complete. 2024-09-21T02:02:28.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:27 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/512636526' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:02:28.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:28.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:28.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:27 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/512636526' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:02:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:28.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:28.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:27 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/512636526' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:02:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:02:28.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:02:28.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:02:28.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:02:28.875 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:28 smithi136 ceph-mon[34754]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:28 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/512636526' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:02:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:28 smithi136 ceph-mon[34754]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:02:29.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:28 smithi092 ceph-mon[31190]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:29.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:28 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/512636526' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:02:29.285 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:28 smithi092 ceph-mon[31190]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:02:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:28 smithi099 ceph-mon[34404]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:28 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/512636526' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:02:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:28 smithi099 ceph-mon[34404]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.381+0000 7fd3a5533640 1 -- 172.21.15.92:0/1288697181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a0103b80 msgr2=0x7fd3a0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.381+0000 7fd3a5533640 1 --2- 172.21.15.92:0/1288697181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a0103b80 0x7fd3a0103f90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd390009ca0 tx=0x7fd39002a8e0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 -- 172.21.15.92:0/1288697181 shutdown_connections 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 --2- 172.21.15.92:0/1288697181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a0109e10 0x7fd3a0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 --2- 172.21.15.92:0/1288697181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a01094c0 0x7fd3a01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 --2- 172.21.15.92:0/1288697181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a0103b80 0x7fd3a0103f90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 -- 172.21.15.92:0/1288697181 >> 172.21.15.92:0/1288697181 conn(0x7fd3a00ff5d0 msgr2=0x7fd3a0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 -- 172.21.15.92:0/1288697181 shutdown_connections 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 -- 172.21.15.92:0/1288697181 wait complete. 2024-09-21T02:02:29.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 Processor -- start 2024-09-21T02:02:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.382+0000 7fd3a5533640 1 -- start start 2024-09-21T02:02:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd3a5533640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a0103b80 0x7fd3a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd3a5533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 0x7fd3a019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:29.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd3a5533640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0109e10 0x7fd3a019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd3a5533640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a0113b30 con 0x7fd3a01094c0 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd3a5533640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a01139b0 con 0x7fd3a0109e10 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd3a5533640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a0113cb0 con 0x7fd3a0103b80 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 0x7fd3a019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a0103b80 0x7fd3a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 0x7fd3a019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56626/0 (socket says 172.21.15.92:56626) 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 -- 172.21.15.92:0/226093162 learned_addr learned my addr 172.21.15.92:0/226093162 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39f7fe640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0109e10 0x7fd3a019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 -- 172.21.15.92:0/226093162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a0103b80 msgr2=0x7fd3a019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a0103b80 0x7fd3a019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 -- 172.21.15.92:0/226093162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0109e10 msgr2=0x7fd3a019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0109e10 0x7fd3a019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 -- 172.21.15.92:0/226093162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a01a0450 con 0x7fd3a01094c0 2024-09-21T02:02:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.383+0000 7fd39e7fc640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 0x7fd3a019aa80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd394006fd0 tx=0x7fd39400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.384+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394011830 con 0x7fd3a01094c0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.384+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a01a0740 con 0x7fd3a01094c0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.384+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd394016070 con 0x7fd3a01094c0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.384+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394010e00 con 0x7fd3a01094c0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.384+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a01a6ea0 con 0x7fd3a01094c0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.385+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a0103f90 con 0x7fd3a01094c0 2024-09-21T02:02:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.386+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd3940119d0 con 0x7fd3a01094c0 2024-09-21T02:02:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.387+0000 7fd387fff640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd3880b6510 0x7fd3880b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.387+0000 7fd39effd640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd3880b6510 0x7fd3880b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.387+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7fd3940e6af0 con 0x7fd3a01094c0 2024-09-21T02:02:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.387+0000 7fd39effd640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd3880b6510 0x7fd3880b8a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd390005e30 tx=0x7fd390009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.390+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd3940ad6b0 con 0x7fd3a01094c0 2024-09-21T02:02:29.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.574+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd3a019b610 con 0x7fd3a01094c0 2024-09-21T02:02:29.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.950+0000 7fd387fff640 1 -- 172.21.15.92:0/226093162 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7fd3940b1a90 con 0x7fd3a01094c0 2024-09-21T02:02:29.951 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.953+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd3880b6510 msgr2=0x7fd3880b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.953+0000 7fd3a5533640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd3880b6510 0x7fd3880b8a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd390005e30 tx=0x7fd390009360 comp rx=0 tx=0).stop 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.953+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 msgr2=0x7fd3a019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.953+0000 7fd3a5533640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 0x7fd3a019aa80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd394006fd0 tx=0x7fd39400b8e0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 shutdown_connections 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd3880b6510 0x7fd3880b8a20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0109e10 0x7fd3a019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a01094c0 0x7fd3a019aa80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 --2- 172.21.15.92:0/226093162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a0103b80 0x7fd3a019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 >> 172.21.15.92:0/226093162 conn(0x7fd3a00ff5d0 msgr2=0x7fd3a01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 shutdown_connections 2024-09-21T02:02:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:29.954+0000 7fd3a5533640 1 -- 172.21.15.92:0/226093162 wait complete. 2024-09-21T02:02:30.263 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:29 smithi092 ceph-mon[31190]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:02:30.264 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:29 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/226093162' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:02:30.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:29 smithi136 ceph-mon[34754]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:02:30.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:29 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/226093162' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:02:30.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:29 smithi099 ceph-mon[34404]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:02:30.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:29 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/226093162' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:02:30.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:02:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:31.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:30 smithi092 ceph-mon[31190]: pgmap v281: 33 pgs: 6 creating+peering, 7 active+clean, 20 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:31.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:30 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/226093162' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:02:31.225 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:30 smithi092 ceph-mon[31190]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:02:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:30 smithi136 ceph-mon[34754]: pgmap v281: 33 pgs: 6 creating+peering, 7 active+clean, 20 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:30 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/226093162' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:02:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:30 smithi136 ceph-mon[34754]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 -- 172.21.15.92:0/412548924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 msgr2=0x7f575010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 --2- 172.21.15.92:0/412548924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575010f5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5748009ca0 tx=0x7f574802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 -- 172.21.15.92:0/412548924 shutdown_connections 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 --2- 172.21.15.92:0/412548924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575010f5a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 --2- 172.21.15.92:0/412548924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750108420 0x7f5750108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 --2- 172.21.15.92:0/412548924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5750102ae0 0x7f5750102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 -- 172.21.15.92:0/412548924 >> 172.21.15.92:0/412548924 conn(0x7f5750079770 msgr2=0x7f5750079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 -- 172.21.15.92:0/412548924 shutdown_connections 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.318+0000 7f5757325640 1 -- 172.21.15.92:0/412548924 wait complete. 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 Processor -- start 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 -- start start 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750102ae0 0x7f575019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5750108420 0x7f575019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5750112a30 con 0x7f5750108d70 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57501128b0 con 0x7f5750102ae0 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f5757325640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5750112bb0 con 0x7f5750108420 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56638/0 (socket says 172.21.15.92:56638) 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 -- 172.21.15.92:0/2328261087 learned_addr learned my addr 172.21.15.92:0/2328261087 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57548c0640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5750108420 0x7f575019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57550c1640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750102ae0 0x7f575019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 -- 172.21.15.92:0/2328261087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5750108420 msgr2=0x7f575019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5750108420 0x7f575019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 -- 172.21.15.92:0/2328261087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750102ae0 msgr2=0x7f575019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750102ae0 0x7f575019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.319+0000 7f57558c2640 1 -- 172.21.15.92:0/2328261087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57501a6560 con 0x7f5750108d70 2024-09-21T02:02:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f57550c1640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750102ae0 0x7f575019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f57558c2640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575019bd20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5748005cc0 tx=0x7f574802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748003bf0 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5748003d90 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574802fbc0 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57501a67f0 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.320+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a6ce0 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.321+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5750103d60 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.322+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f574802fd60 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.323+0000 7f57467fc640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f573c0b6590 0x7f573c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.324+0000 7f57550c1640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f573c0b6590 0x7f573c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.324+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7f5748108830 con 0x7f5750108d70 2024-09-21T02:02:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.324+0000 7f57550c1640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f573c0b6590 0x7f573c0b8aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f575006c6e0 tx=0x7f574c005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:31.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.327+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f57480cf270 con 0x7f5750108d70 2024-09-21T02:02:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:30 smithi099 ceph-mon[34404]: pgmap v281: 33 pgs: 6 creating+peering, 7 active+clean, 20 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:30 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/226093162' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:02:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:30 smithi099 ceph-mon[34404]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:02:31.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.544+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f575006bf90 con 0x7f5750108d70 2024-09-21T02:02:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.968+0000 7f57467fc640 1 -- 172.21.15.92:0/2328261087 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f575006bf90 con 0x7f5750108d70 2024-09-21T02:02:31.969 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T02:02:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f573c0b6590 msgr2=0x7f573c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f573c0b6590 0x7f573c0b8aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f575006c6e0 tx=0x7f574c005e70 comp rx=0 tx=0).stop 2024-09-21T02:02:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 msgr2=0x7f575019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575019bd20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5748005cc0 tx=0x7f574802f380 comp rx=0 tx=0).stop 2024-09-21T02:02:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 shutdown_connections 2024-09-21T02:02:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f573c0b6590 0x7f573c0b8aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750108d70 0x7f575019bd20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5750108420 0x7f575019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 --2- 172.21.15.92:0/2328261087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5750102ae0 0x7f575019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.970+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 >> 172.21.15.92:0/2328261087 conn(0x7f5750079770 msgr2=0x7f5750104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.971+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 shutdown_connections 2024-09-21T02:02:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:31.971+0000 7f5757325640 1 -- 172.21.15.92:0/2328261087 wait complete. 2024-09-21T02:02:32.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:31 smithi092 ceph-mon[31190]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:02:32.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:31 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2328261087' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:02:32.278 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:31 smithi092 ceph-mon[31190]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-21T02:02:32.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:31 smithi136 ceph-mon[34754]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:02:32.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:31 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2328261087' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:02:32.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:31 smithi136 ceph-mon[34754]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-21T02:02:32.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:31 smithi099 ceph-mon[34404]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:02:32.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:31 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2328261087' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:02:32.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:31 smithi099 ceph-mon[34404]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-21T02:02:32.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:02:32.851 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:33.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:32 smithi092 ceph-mon[31190]: pgmap v284: 97 pgs: 6 creating+peering, 11 active+clean, 80 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:33.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:32 smithi092 ceph-mon[31190]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:02:33.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:32 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2328261087' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:02:33.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:32 smithi092 ceph-mon[31190]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-21T02:02:33.282 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:32 smithi092 ceph-mon[31190]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-21T02:02:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:32 smithi136 ceph-mon[34754]: pgmap v284: 97 pgs: 6 creating+peering, 11 active+clean, 80 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:32 smithi136 ceph-mon[34754]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:02:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:32 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2328261087' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:02:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:32 smithi136 ceph-mon[34754]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-21T02:02:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:32 smithi136 ceph-mon[34754]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-21T02:02:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:32 smithi099 ceph-mon[34404]: pgmap v284: 97 pgs: 6 creating+peering, 11 active+clean, 80 unknown; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:02:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:32 smithi099 ceph-mon[34404]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:02:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:32 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2328261087' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:02:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:32 smithi099 ceph-mon[34404]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-21T02:02:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:32 smithi099 ceph-mon[34404]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-21T02:02:33.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 -- 172.21.15.92:0/3471533178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae81083a0 msgr2=0x7f8ae81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:33.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 --2- 172.21.15.92:0/3471533178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae81083a0 0x7f8ae81087b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009900 tx=0x7f8ad002a410 comp rx=0 tx=0).stop 2024-09-21T02:02:33.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 -- 172.21.15.92:0/3471533178 shutdown_connections 2024-09-21T02:02:33.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 --2- 172.21.15.92:0/3471533178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ae8108cf0 0x7f8ae810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 --2- 172.21.15.92:0/3471533178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae81083a0 0x7f8ae81087b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 --2- 172.21.15.92:0/3471533178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae8102a60 0x7f8ae8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 -- 172.21.15.92:0/3471533178 >> 172.21.15.92:0/3471533178 conn(0x7f8ae8079730 msgr2=0x7f8ae8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:33.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 -- 172.21.15.92:0/3471533178 shutdown_connections 2024-09-21T02:02:33.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.370+0000 7f8aed782640 1 -- 172.21.15.92:0/3471533178 wait complete. 2024-09-21T02:02:33.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 Processor -- start 2024-09-21T02:02:33.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 -- start start 2024-09-21T02:02:33.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ae8102a60 0x7f8ae8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:33.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae81083a0 0x7f8ae81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 0x7f8ae8199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8112a20 con 0x7f8ae8108cf0 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae81128a0 con 0x7f8ae81083a0 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8aed782640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8112ba0 con 0x7f8ae8102a60 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8ae77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 0x7f8ae8199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8ae77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 0x7f8ae8199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58080/0 (socket says 172.21.15.92:58080) 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8ae77fe640 1 -- 172.21.15.92:0/2494901602 learned_addr learned my addr 172.21.15.92:0/2494901602 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.371+0000 7f8ae6ffd640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ae8102a60 0x7f8ae8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8adffff640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae81083a0 0x7f8ae81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae77fe640 1 -- 172.21.15.92:0/2494901602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ae8102a60 msgr2=0x7f8ae8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae77fe640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ae8102a60 0x7f8ae8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae77fe640 1 -- 172.21.15.92:0/2494901602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae81083a0 msgr2=0x7f8ae81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae77fe640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae81083a0 0x7f8ae81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae77fe640 1 -- 172.21.15.92:0/2494901602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae81a8910 con 0x7f8ae8108cf0 2024-09-21T02:02:33.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae77fe640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 0x7f8ae8199ce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000b970 tx=0x7f8ae000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:33.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0019070 con 0x7f8ae8108cf0 2024-09-21T02:02:33.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae0014440 con 0x7f8ae8108cf0 2024-09-21T02:02:33.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae81a8c00 con 0x7f8ae8108cf0 2024-09-21T02:02:33.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00075d0 con 0x7f8ae8108cf0 2024-09-21T02:02:33.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.372+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae81a9160 con 0x7f8ae8108cf0 2024-09-21T02:02:33.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.373+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae8102ef0 con 0x7f8ae8108cf0 2024-09-21T02:02:33.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.375+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f8ae001c070 con 0x7f8ae8108cf0 2024-09-21T02:02:33.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.375+0000 7f8ae4ff9640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8ad40b6590 0x7f8ad40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:33.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.375+0000 7f8ae6ffd640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8ad40b6590 0x7f8ad40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:33.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.375+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f8ae00e6a50 con 0x7f8ae8108cf0 2024-09-21T02:02:33.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.376+0000 7f8ae6ffd640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8ad40b6590 0x7f8ad40b8aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ae81070d0 tx=0x7f8ad8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:33.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.379+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ae00ad430 con 0x7f8ae8108cf0 2024-09-21T02:02:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.556+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ae806b270 con 0x7f8ae8108cf0 2024-09-21T02:02:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.558+0000 7f8ae4ff9640 1 -- 172.21.15.92:0/2494901602 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22268 (secure 0 0 0) 0x7f8ae806b270 con 0x7f8ae8108cf0 2024-09-21T02:02:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:33.559 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:02:31.545307+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:02:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8ad40b6590 msgr2=0x7f8ad40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8ad40b6590 0x7f8ad40b8aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ae81070d0 tx=0x7f8ad8009110 comp rx=0 tx=0).stop 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 msgr2=0x7f8ae8199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 0x7f8ae8199ce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000b970 tx=0x7f8ae000bd70 comp rx=0 tx=0).stop 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 shutdown_connections 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8ad40b6590 0x7f8ad40b8aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ae8108cf0 0x7f8ae8199ce0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae81083a0 0x7f8ae81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 --2- 172.21.15.92:0/2494901602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ae8102a60 0x7f8ae8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 >> 172.21.15.92:0/2494901602 conn(0x7f8ae8079730 msgr2=0x7f8ae8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 shutdown_connections 2024-09-21T02:02:33.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:33.560+0000 7f8aed782640 1 -- 172.21.15.92:0/2494901602 wait complete. 2024-09-21T02:02:34.211 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:33 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] up:active 2024-09-21T02:02:34.211 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:33 smithi092 ceph-mon[31190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:34.211 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:33 smithi092 ceph-mon[31190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:34.211 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:33 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2494901602' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:34.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:02:34.283 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:02:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: 2024-09-21T02:02:33.835+0000 7face8011640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:02:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:33 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] up:active 2024-09-21T02:02:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:33 smithi136 ceph-mon[34754]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:34.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:33 smithi136 ceph-mon[34754]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:34.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:33 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2494901602' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:33 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] up:active 2024-09-21T02:02:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:33 smithi099 ceph-mon[34404]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:33 smithi099 ceph-mon[34404]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:33 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2494901602' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 -- 172.21.15.92:0/4003375838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0102ae0 msgr2=0x7f34e0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 --2- 172.21.15.92:0/4003375838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0102ae0 0x7f34e0102ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009900 tx=0x7f34dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 -- 172.21.15.92:0/4003375838 shutdown_connections 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 --2- 172.21.15.92:0/4003375838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108d70 0x7f34e010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 --2- 172.21.15.92:0/4003375838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e0108420 0x7f34e0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 --2- 172.21.15.92:0/4003375838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0102ae0 0x7f34e0102ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 -- 172.21.15.92:0/4003375838 >> 172.21.15.92:0/4003375838 conn(0x7f34e0079770 msgr2=0x7f34e0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 -- 172.21.15.92:0/4003375838 shutdown_connections 2024-09-21T02:02:35.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.020+0000 7f34e727f640 1 -- 172.21.15.92:0/4003375838 wait complete. 2024-09-21T02:02:35.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 Processor -- start 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 -- start start 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e0102ae0 0x7f34e01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108420 0x7f34e0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 0x7f34e0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0112a90 con 0x7f34e0108d70 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0112910 con 0x7f34e0102ae0 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e727f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0112c10 con 0x7f34e0108420 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e581c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 0x7f34e0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e581c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 0x7f34e0199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58098/0 (socket says 172.21.15.92:58098) 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e581c640 1 -- 172.21.15.92:0/865047124 learned_addr learned my addr 172.21.15.92:0/865047124 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e501b640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e0102ae0 0x7f34e01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.021+0000 7f34e481a640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108420 0x7f34e0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:35.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e581c640 1 -- 172.21.15.92:0/865047124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108420 msgr2=0x7f34e0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e581c640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108420 0x7f34e0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e581c640 1 -- 172.21.15.92:0/865047124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e0102ae0 msgr2=0x7f34e01991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e581c640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e0102ae0 0x7f34e01991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e581c640 1 -- 172.21.15.92:0/865047124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e01a8880 con 0x7f34e0108d70 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e481a640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108420 0x7f34e0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e581c640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 0x7f34e0199c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34d800cad0 tx=0x7f34d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800ae20 con 0x7f34e0108d70 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e01a8b70 con 0x7f34e0108d70 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d8005020 con 0x7f34e0108d70 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800f6b0 con 0x7f34e0108d70 2024-09-21T02:02:35.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.022+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a90b0 con 0x7f34e0108d70 2024-09-21T02:02:35.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.023+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e0103d60 con 0x7f34e0108d70 2024-09-21T02:02:35.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.024+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f34d80051d0 con 0x7f34e0108d70 2024-09-21T02:02:35.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.025+0000 7f34d67fc640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34cc0b6510 0x7f34cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:35.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.025+0000 7f34e501b640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34cc0b6510 0x7f34cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:35.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.025+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f34d80e7590 con 0x7f34e0108d70 2024-09-21T02:02:35.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.026+0000 7f34e501b640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34cc0b6510 0x7f34cc0b8a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000c00 tx=0x7f34dc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:35.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.028+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34d8014030 con 0x7f34e0108d70 2024-09-21T02:02:35.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.206+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34e0102ef0 con 0x7f34e0108d70 2024-09-21T02:02:35.207 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:34 smithi092 ceph-mon[31190]: pgmap v286: 97 pgs: 6 creating+peering, 61 active+clean, 30 unknown; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 B/s wr, 7 op/s 2024-09-21T02:02:35.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.208+0000 7f34d67fc640 1 -- 172.21.15.92:0/865047124 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22268 (secure 0 0 0) 0x7f34e0102ef0 con 0x7f34e0108d70 2024-09-21T02:02:35.209 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:35.209 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:02:31.545307+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34cc0b6510 msgr2=0x7f34cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34cc0b6510 0x7f34cc0b8a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000c00 tx=0x7f34dc031400 comp rx=0 tx=0).stop 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 msgr2=0x7f34e0199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 0x7f34e0199c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34d800cad0 tx=0x7f34d800cf90 comp rx=0 tx=0).stop 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 shutdown_connections 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34cc0b6510 0x7f34cc0b8a20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34e0108d70 0x7f34e0199c50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0108420 0x7f34e0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 --2- 172.21.15.92:0/865047124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e0102ae0 0x7f34e01991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 >> 172.21.15.92:0/865047124 conn(0x7f34e0079770 msgr2=0x7f34e0106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 shutdown_connections 2024-09-21T02:02:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:35.210+0000 7f34e727f640 1 -- 172.21.15.92:0/865047124 wait complete. 2024-09-21T02:02:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:34 smithi136 ceph-mon[34754]: pgmap v286: 97 pgs: 6 creating+peering, 61 active+clean, 30 unknown; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 B/s wr, 7 op/s 2024-09-21T02:02:35.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:34 smithi099 ceph-mon[34404]: pgmap v286: 97 pgs: 6 creating+peering, 61 active+clean, 30 unknown; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 B/s wr, 7 op/s 2024-09-21T02:02:35.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:02:36.048 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:36.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:35 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/865047124' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:36.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:35 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/865047124' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:36.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:35 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/865047124' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 -- 172.21.15.92:0/2215729175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c102080 msgr2=0x7efe4c102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 --2- 172.21.15.92:0/2215729175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c102080 0x7efe4c102490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009900 tx=0x7efe3c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 -- 172.21.15.92:0/2215729175 shutdown_connections 2024-09-21T02:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 --2- 172.21.15.92:0/2215729175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe4c109eb0 0x7efe4c1106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 --2- 172.21.15.92:0/2215729175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe4c109560 0x7efe4c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 --2- 172.21.15.92:0/2215729175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c102080 0x7efe4c102490 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 -- 172.21.15.92:0/2215729175 >> 172.21.15.92:0/2215729175 conn(0x7efe4c079ec0 msgr2=0x7efe4c07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 -- 172.21.15.92:0/2215729175 shutdown_connections 2024-09-21T02:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.544+0000 7efe510e0640 1 -- 172.21.15.92:0/2215729175 wait complete. 2024-09-21T02:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.545+0000 7efe510e0640 1 Processor -- start 2024-09-21T02:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.545+0000 7efe510e0640 1 -- start start 2024-09-21T02:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe4c102080 0x7efe4c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe4c109560 0x7efe4c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 0x7efe4c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c113bd0 con 0x7efe4c109eb0 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c113a50 con 0x7efe4c109560 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c113d50 con 0x7efe4c102080 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 0x7efe4c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 0x7efe4c19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58114/0 (socket says 172.21.15.92:58114) 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 -- 172.21.15.92:0/2060625491 learned_addr learned my addr 172.21.15.92:0/2060625491 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4a59c640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe4c109560 0x7efe4c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4ad9d640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe4c102080 0x7efe4c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 -- 172.21.15.92:0/2060625491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe4c102080 msgr2=0x7efe4c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe4c102080 0x7efe4c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 -- 172.21.15.92:0/2060625491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe4c109560 msgr2=0x7efe4c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe4c109560 0x7efe4c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 -- 172.21.15.92:0/2060625491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c1a0450 con 0x7efe4c109eb0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe4b59e640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 0x7efe4c19fde0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efe4400b970 tx=0x7efe4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44019070 con 0x7efe4c109eb0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44014440 con 0x7efe4c109eb0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.546+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1a0740 con 0x7efe4c109eb0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.547+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe440075d0 con 0x7efe4c109eb0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.547+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a6f60 con 0x7efe4c109eb0 2024-09-21T02:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.548+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c102490 con 0x7efe4c109eb0 2024-09-21T02:02:36.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.549+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7efe4401c070 con 0x7efe4c109eb0 2024-09-21T02:02:36.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.550+0000 7efe33fff640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe340b6540 0x7efe340b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:36.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.550+0000 7efe4ad9d640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe340b6540 0x7efe340b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:36.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.550+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7efe440e69f0 con 0x7efe4c109eb0 2024-09-21T02:02:36.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.551+0000 7efe4ad9d640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe340b6540 0x7efe340b8a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efe3c031440 tx=0x7efe3c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:36.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.554+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe440b17b0 con 0x7efe4c109eb0 2024-09-21T02:02:36.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.769+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efe4c19b6d0 con 0x7efe4c109eb0 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.992+0000 7efe33fff640 1 -- 172.21.15.92:0/2060625491 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7efe4c19b6d0 con 0x7efe4c109eb0 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe340b6540 msgr2=0x7efe340b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe340b6540 0x7efe340b8a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efe3c031440 tx=0x7efe3c005e80 comp rx=0 tx=0).stop 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 msgr2=0x7efe4c19fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 0x7efe4c19fde0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efe4400b970 tx=0x7efe4400bd70 comp rx=0 tx=0).stop 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 shutdown_connections 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe340b6540 0x7efe340b8a50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe4c109eb0 0x7efe4c19fde0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe4c109560 0x7efe4c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 --2- 172.21.15.92:0/2060625491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe4c102080 0x7efe4c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.994+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 >> 172.21.15.92:0/2060625491 conn(0x7efe4c079ec0 msgr2=0x7efe4c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.995+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 shutdown_connections 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:36.995+0000 7efe510e0640 1 -- 172.21.15.92:0/2060625491 wait complete. 2024-09-21T02:02:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:36 smithi136 ceph-mon[34754]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-21T02:02:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:36 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2060625491' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:02:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:36 smithi092 ceph-mon[31190]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-21T02:02:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:36 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2060625491' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:02:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:36 smithi099 ceph-mon[34404]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-21T02:02:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:36 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2060625491' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:02:37.660 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T02:02:37.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T02:02:37.906 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:38.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:38 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2060625491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:02:38.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:38 smithi136 ceph-mon[34754]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:38 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2060625491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:02:38.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:38 smithi092 ceph-mon[31190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:38.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:38 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2060625491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:02:38.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:38 smithi099 ceph-mon[34404]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 -- 172.21.15.92:0/2490646873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 msgr2=0x7f80bc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 --2- 172.21.15.92:0/2490646873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc1098b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009900 tx=0x7f80b002a410 comp rx=0 tx=0).stop 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 -- 172.21.15.92:0/2490646873 shutdown_connections 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 --2- 172.21.15.92:0/2490646873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc109df0 0x7f80bc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 --2- 172.21.15.92:0/2490646873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc1098b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 --2- 172.21.15.92:0/2490646873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80bc103b60 0x7f80bc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 -- 172.21.15.92:0/2490646873 >> 172.21.15.92:0/2490646873 conn(0x7f80bc07b850 msgr2=0x7f80bc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 -- 172.21.15.92:0/2490646873 shutdown_connections 2024-09-21T02:02:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 -- 172.21.15.92:0/2490646873 wait complete. 2024-09-21T02:02:38.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 Processor -- start 2024-09-21T02:02:38.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.393+0000 7f80c265a640 1 -- start start 2024-09-21T02:02:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c265a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc103b60 0x7f80bc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c265a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c265a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80bc109df0 0x7f80bc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c265a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc113ab0 con 0x7f80bc1094a0 2024-09-21T02:02:38.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c265a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc113930 con 0x7f80bc103b60 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c265a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc113c30 con 0x7f80bc109df0 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58140/0 (socket says 172.21.15.92:58140) 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 -- 172.21.15.92:0/936316536 learned_addr learned my addr 172.21.15.92:0/936316536 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bbfff640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc103b60 0x7f80bc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80c0bf7640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80bc109df0 0x7f80bc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 -- 172.21.15.92:0/936316536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80bc109df0 msgr2=0x7f80bc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80bc109df0 0x7f80bc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 -- 172.21.15.92:0/936316536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc103b60 msgr2=0x7f80bc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc103b60 0x7f80bc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.394+0000 7f80bb7fe640 1 -- 172.21.15.92:0/936316536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc1a0410 con 0x7f80bc1094a0 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.395+0000 7f80bb7fe640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc19aa40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f80b002a930 tx=0x7f80b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.395+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b003f030 con 0x7f80bc1094a0 2024-09-21T02:02:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.395+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc1a06a0 con 0x7f80bc1094a0 2024-09-21T02:02:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.395+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b0003aa0 con 0x7f80bc1094a0 2024-09-21T02:02:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.395+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0035070 con 0x7f80bc1094a0 2024-09-21T02:02:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.395+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc1a6e60 con 0x7f80bc1094a0 2024-09-21T02:02:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.396+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc104de0 con 0x7f80bc1094a0 2024-09-21T02:02:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.397+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f80b0042030 con 0x7f80bc1094a0 2024-09-21T02:02:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.398+0000 7f80b97fa640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80a40b6510 0x7f80a40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.398+0000 7f80bbfff640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80a40b6510 0x7f80a40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.398+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f80b0107c40 con 0x7f80bc1094a0 2024-09-21T02:02:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.398+0000 7f80bbfff640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80a40b6510 0x7f80a40b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00bbe0 tx=0x7f80ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:38.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.401+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80b00ce620 con 0x7f80bc1094a0 2024-09-21T02:02:38.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:38.617+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f80bc10bbc0 con 0x7f80bc1094a0 2024-09-21T02:02:39.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.010+0000 7f80b97fa640 1 -- 172.21.15.92:0/936316536 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f80b00d2a00 con 0x7f80bc1094a0 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80a40b6510 msgr2=0x7f80a40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80a40b6510 0x7f80a40b8a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00bbe0 tx=0x7f80ac00b390 comp rx=0 tx=0).stop 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 msgr2=0x7f80bc19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc19aa40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f80b002a930 tx=0x7f80b0004270 comp rx=0 tx=0).stop 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 shutdown_connections 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80a40b6510 0x7f80a40b8a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80bc109df0 0x7f80bc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80bc1094a0 0x7f80bc19aa40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 --2- 172.21.15.92:0/936316536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc103b60 0x7f80bc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:39.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 >> 172.21.15.92:0/936316536 conn(0x7f80bc07b850 msgr2=0x7f80bc105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:39.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 shutdown_connections 2024-09-21T02:02:39.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:39.012+0000 7f80c265a640 1 -- 172.21.15.92:0/936316536 wait complete. 2024-09-21T02:02:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:39 smithi136 ceph-mon[34754]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-21T02:02:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:39 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/936316536' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T02:02:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:39 smithi092 ceph-mon[31190]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-21T02:02:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:39 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/936316536' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T02:02:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:39 smithi099 ceph-mon[34404]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-21T02:02:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:39 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/936316536' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T02:02:39.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T02:02:39.912 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T02:02:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/936316536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T02:02:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T02:02:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:40.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-21T02:02:40.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T02:02:40.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:40 smithi136 ceph-mon[34754]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-21T02:02:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T02:02:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/936316536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T02:02:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T02:02:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:40.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-21T02:02:40.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T02:02:40.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:40 smithi092 ceph-mon[31190]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-21T02:02:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T02:02:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/936316536' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T02:02:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T02:02:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T02:02:40.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-21T02:02:40.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T02:02:40.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:40 smithi099 ceph-mon[34404]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-21T02:02:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 -- 172.21.15.92:0/2025196263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f01083b0 msgr2=0x7f61f01087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 --2- 172.21.15.92:0/2025196263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f01083b0 0x7f61f01087c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009900 tx=0x7f61e402a410 comp rx=0 tx=0).stop 2024-09-21T02:02:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 -- 172.21.15.92:0/2025196263 shutdown_connections 2024-09-21T02:02:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 --2- 172.21.15.92:0/2025196263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f0108d00 0x7f61f010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 --2- 172.21.15.92:0/2025196263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f01083b0 0x7f61f01087c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:40.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 --2- 172.21.15.92:0/2025196263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61f0102ac0 0x7f61f0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:40.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 -- 172.21.15.92:0/2025196263 >> 172.21.15.92:0/2025196263 conn(0x7f61f0079720 msgr2=0x7f61f0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:40.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 -- 172.21.15.92:0/2025196263 shutdown_connections 2024-09-21T02:02:40.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.424+0000 7f61f539a640 1 -- 172.21.15.92:0/2025196263 wait complete. 2024-09-21T02:02:40.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 Processor -- start 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 -- start start 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 0x7f61f019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f01083b0 0x7f61f019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61f0108d00 0x7f61f019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0113b00 con 0x7f61f0102ac0 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0113980 con 0x7f61f01083b0 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.425+0000 7f61f539a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0113c80 con 0x7f61f0108d00 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 0x7f61f019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 0x7f61f019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58158/0 (socket says 172.21.15.92:58158) 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 -- 172.21.15.92:0/2649603431 learned_addr learned my addr 172.21.15.92:0/2649603431 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61ef7fe640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61f0108d00 0x7f61f019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61ee7fc640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f01083b0 0x7f61f019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 -- 172.21.15.92:0/2649603431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61f0108d00 msgr2=0x7f61f019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61f0108d00 0x7f61f019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:40.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 -- 172.21.15.92:0/2649603431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f01083b0 msgr2=0x7f61f019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f01083b0 0x7f61f019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 -- 172.21.15.92:0/2649603431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f01a0540 con 0x7f61f0102ac0 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61ee7fc640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f01083b0 0x7f61f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61eeffd640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 0x7f61f019a5d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c970 tx=0x7f61e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0007bb0 con 0x7f61f0102ac0 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61e0007d50 con 0x7f61f0102ac0 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f01a0830 con 0x7f61f0102ac0 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e000f450 con 0x7f61f0102ac0 2024-09-21T02:02:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.426+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f01a6f90 con 0x7f61f0102ac0 2024-09-21T02:02:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.427+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f0103cf0 con 0x7f61f0102ac0 2024-09-21T02:02:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.430+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7f61e0004f00 con 0x7f61f0102ac0 2024-09-21T02:02:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.430+0000 7f61d7fff640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f61d80b6510 0x7f61d80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.430+0000 7f61ee7fc640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f61d80b6510 0x7f61d80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.430+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f61e00e7280 con 0x7f61f0102ac0 2024-09-21T02:02:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.431+0000 7f61ee7fc640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f61d80b6510 0x7f61d80b8a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009230 tx=0x7f61e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:40.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.434+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61e0014030 con 0x7f61f0102ac0 2024-09-21T02:02:40.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:40.649+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f61f0102ed0 con 0x7f61f0102ac0 2024-09-21T02:02:41.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.026+0000 7f61d7fff640 1 -- 172.21.15.92:0/2649603431 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f61f0102ed0 con 0x7f61f0102ac0 2024-09-21T02:02:41.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f61d80b6510 msgr2=0x7f61d80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f61d80b6510 0x7f61d80b8a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009230 tx=0x7f61e4000bc0 comp rx=0 tx=0).stop 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 msgr2=0x7f61f019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 0x7f61f019a5d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c970 tx=0x7f61e000ce30 comp rx=0 tx=0).stop 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 shutdown_connections 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f61d80b6510 0x7f61d80b8a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61f0108d00 0x7f61f019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f01083b0 0x7f61f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 --2- 172.21.15.92:0/2649603431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61f0102ac0 0x7f61f019a5d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 >> 172.21.15.92:0/2649603431 conn(0x7f61f0079720 msgr2=0x7f61f01044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 shutdown_connections 2024-09-21T02:02:41.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:41.029+0000 7f61f539a640 1 -- 172.21.15.92:0/2649603431 wait complete. 2024-09-21T02:02:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:41 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] up:active 2024-09-21T02:02:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:41 smithi136 ceph-mon[34754]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T02:02:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:41 smithi136 ceph-mon[34754]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-21T02:02:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:41 smithi136 ceph-mon[34754]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-21T02:02:41.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:41 smithi136 ceph-mon[34754]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T02:02:41.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:41 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2649603431' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:02:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:41 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] up:active 2024-09-21T02:02:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:41 smithi092 ceph-mon[31190]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T02:02:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:41 smithi092 ceph-mon[31190]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-21T02:02:41.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:41 smithi092 ceph-mon[31190]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-21T02:02:41.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:41 smithi092 ceph-mon[31190]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T02:02:41.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:41 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2649603431' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:02:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:41 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] up:active 2024-09-21T02:02:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:41 smithi099 ceph-mon[34404]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T02:02:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:41 smithi099 ceph-mon[34404]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-21T02:02:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:41 smithi099 ceph-mon[34404]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-21T02:02:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:41 smithi099 ceph-mon[34404]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T02:02:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:41 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2649603431' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:02:41.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T02:02:41.918 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T02:02:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2649603431' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] up:active 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] up:active 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: Cluster is now healthy 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-21T02:02:42.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:42 smithi136 ceph-mon[34754]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-21T02:02:42.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2649603431' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] up:active 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] up:active 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: Cluster is now healthy 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-21T02:02:42.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:42 smithi092 ceph-mon[31190]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-21T02:02:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T02:02:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2649603431' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:02:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] up:active 2024-09-21T02:02:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T02:02:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-21T02:02:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] up:active 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: Cluster is now healthy 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-21T02:02:42.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:42 smithi099 ceph-mon[34404]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 -- 172.21.15.92:0/228679945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda001083a0 msgr2=0x7fda001087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 --2- 172.21.15.92:0/228679945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda001083a0 0x7fda001087b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009900 tx=0x7fd9f002a410 comp rx=0 tx=0).stop 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 -- 172.21.15.92:0/228679945 shutdown_connections 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 --2- 172.21.15.92:0/228679945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda00108cf0 0x7fda0010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 --2- 172.21.15.92:0/228679945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda001083a0 0x7fda001087b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 --2- 172.21.15.92:0/228679945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda00102a60 0x7fda00102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 -- 172.21.15.92:0/228679945 >> 172.21.15.92:0/228679945 conn(0x7fda00079730 msgr2=0x7fda00079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 -- 172.21.15.92:0/228679945 shutdown_connections 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.412+0000 7fda076d5640 1 -- 172.21.15.92:0/228679945 wait complete. 2024-09-21T02:02:42.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 Processor -- start 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 -- start start 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 0x7fda0019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda001083a0 0x7fda0019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda00108cf0 0x7fda0019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda00112960 con 0x7fda00102a60 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda001127e0 con 0x7fda001083a0 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda076d5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda00112ae0 con 0x7fda00108cf0 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 0x7fda0019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 0x7fda0019b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58182/0 (socket says 172.21.15.92:58182) 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 -- 172.21.15.92:0/3695431910 learned_addr learned my addr 172.21.15.92:0/3695431910 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda04c70640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda001083a0 0x7fda0019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 -- 172.21.15.92:0/3695431910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda00108cf0 msgr2=0x7fda0019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda00108cf0 0x7fda0019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:42.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 -- 172.21.15.92:0/3695431910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda001083a0 msgr2=0x7fda0019b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda001083a0 0x7fda0019b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.413+0000 7fda05471640 1 -- 172.21.15.92:0/3695431910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda001a6500 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.414+0000 7fda05471640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 0x7fda0019b270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9f0 tx=0x7fd9fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.414+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cd80 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.414+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc0054d0 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.414+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc010690 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.414+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda001a67f0 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.414+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda001a6cb0 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.415+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda00102e70 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.416+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7fd9fc004e10 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.417+0000 7fd9f67fc640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9ec0b65c0 0x7fd9ec0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.417+0000 7fda04c70640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9ec0b65c0 0x7fd9ec0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.417+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd9fc0e7770 con 0x7fda00102a60 2024-09-21T02:02:42.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.417+0000 7fda04c70640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9ec0b65c0 0x7fd9ec0b8ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:42.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.420+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9fc056710 con 0x7fda00102a60 2024-09-21T02:02:42.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:42.636+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fda0002e670 con 0x7fda00102a60 2024-09-21T02:02:43.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.316+0000 7fd9f67fc640 1 -- 172.21.15.92:0/3695431910 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fd9fc0ae150 con 0x7fda00102a60 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9ec0b65c0 msgr2=0x7fd9ec0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9ec0b65c0 0x7fd9ec0b8ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f0031400 comp rx=0 tx=0).stop 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 msgr2=0x7fda0019b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 0x7fda0019b270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9f0 tx=0x7fd9fc00edf0 comp rx=0 tx=0).stop 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 shutdown_connections 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9ec0b65c0 0x7fd9ec0b8ad0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda00108cf0 0x7fda0019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda001083a0 0x7fda0019b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 --2- 172.21.15.92:0/3695431910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00102a60 0x7fda0019b270 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 >> 172.21.15.92:0/3695431910 conn(0x7fda00079730 msgr2=0x7fda001043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:43.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 shutdown_connections 2024-09-21T02:02:43.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:43.319+0000 7fda076d5640 1 -- 172.21.15.92:0/3695431910 wait complete. 2024-09-21T02:02:43.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:43 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] up:active 2024-09-21T02:02:43.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:43 smithi099 ceph-mon[34404]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T02:02:43.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:43 smithi099 ceph-mon[34404]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:02:43.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:43 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3695431910' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:02:43.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:43 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] up:active 2024-09-21T02:02:43.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:43 smithi092 ceph-mon[31190]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T02:02:43.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:43 smithi092 ceph-mon[31190]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:02:43.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:43 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3695431910' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:02:43.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:43 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] up:active 2024-09-21T02:02:43.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:43 smithi136 ceph-mon[34754]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T02:02:43.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:43 smithi136 ceph-mon[34754]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:02:43.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:43 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3695431910' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:02:44.042 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T02:02:44.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T02:02:44.289 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:44 smithi099 ceph-mon[34404]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-21T02:02:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3695431910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:02:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:44 smithi099 ceph-mon[34404]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:02:44.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-mon[31190]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-21T02:02:44.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3695431910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:02:44.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-mon[31190]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.761+0000 7ff54d33f640 1 -- 172.21.15.92:0/2730184021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 msgr2=0x7ff5481098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.761+0000 7ff54d33f640 1 --2- 172.21.15.92:0/2730184021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff5481098b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009900 tx=0x7ff53c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 -- 172.21.15.92:0/2730184021 shutdown_connections 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 --2- 172.21.15.92:0/2730184021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 0x7ff548110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 --2- 172.21.15.92:0/2730184021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff5481098b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 --2- 172.21.15.92:0/2730184021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff548103b60 0x7ff548103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 -- 172.21.15.92:0/2730184021 >> 172.21.15.92:0/2730184021 conn(0x7ff54807b850 msgr2=0x7ff54807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 -- 172.21.15.92:0/2730184021 shutdown_connections 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 -- 172.21.15.92:0/2730184021 wait complete. 2024-09-21T02:02:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.762+0000 7ff54d33f640 1 Processor -- start 2024-09-21T02:02:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 -- start start 2024-09-21T02:02:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff548103b60 0x7ff54819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff54819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:44.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 0x7ff54819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff548113ab0 con 0x7ff5481094a0 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff548113930 con 0x7ff548109df0 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff54d33f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff548113c30 con 0x7ff548103b60 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff54819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff54819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36106/0 (socket says 172.21.15.92:36106) 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 -- 172.21.15.92:0/608560661 learned_addr learned my addr 172.21.15.92:0/608560661 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff546ffd640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff548103b60 0x7ff54819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5477fe640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 0x7ff54819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 -- 172.21.15.92:0/608560661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff548103b60 msgr2=0x7ff54819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff548103b60 0x7ff54819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 -- 172.21.15.92:0/608560661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 msgr2=0x7ff54819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 0x7ff54819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5467fc640 1 -- 172.21.15.92:0/608560661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5481a0510 con 0x7ff5481094a0 2024-09-21T02:02:44.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.763+0000 7ff5477fe640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 0x7ff54819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.764+0000 7ff5467fc640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff54819ab40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff53c02a930 tx=0x7ff53c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.764+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c03f030 con 0x7ff5481094a0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.764+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c003aa0 con 0x7ff5481094a0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.764+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c035070 con 0x7ff5481094a0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.764+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5481a07a0 con 0x7ff5481094a0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.764+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5481a6f60 con 0x7ff5481094a0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.765+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff548104de0 con 0x7ff5481094a0 2024-09-21T02:02:44.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.767+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 154512+0+0 (secure 0 0 0) 0x7ff53c042030 con 0x7ff5481094a0 2024-09-21T02:02:44.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.768+0000 7ff52ffff640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5300b6510 0x7ff5300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:44.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.768+0000 7ff546ffd640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5300b6510 0x7ff5300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:44.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.768+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff53c107c40 con 0x7ff5481094a0 2024-09-21T02:02:44.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.768+0000 7ff546ffd640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5300b6510 0x7ff5300b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff538005fd0 tx=0x7ff538009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:44.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.771+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff53c0ce620 con 0x7ff5481094a0 2024-09-21T02:02:44.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:44 smithi136 ceph-mon[34754]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-21T02:02:44.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3695431910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:02:44.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:44 smithi136 ceph-mon[34754]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:02:44.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:44.975+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff548063800 con 0x7ff5300b6510 2024-09-21T02:02:45.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.010+0000 7ff52ffff640 1 -- 172.21.15.92:0/608560661 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff548063800 con 0x7ff5300b6510 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5300b6510 msgr2=0x7ff5300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5300b6510 0x7ff5300b8a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff538005fd0 tx=0x7ff538009680 comp rx=0 tx=0).stop 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 msgr2=0x7ff54819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff54819ab40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff53c02a930 tx=0x7ff53c004270 comp rx=0 tx=0).stop 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 shutdown_connections 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5300b6510 0x7ff5300b8a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff548109df0 0x7ff54819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5481094a0 0x7ff54819ab40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:45.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 --2- 172.21.15.92:0/608560661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff548103b60 0x7ff54819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:45.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 >> 172.21.15.92:0/608560661 conn(0x7ff54807b850 msgr2=0x7ff548107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:45.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 shutdown_connections 2024-09-21T02:02:45.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:45.012+0000 7ff54d33f640 1 -- 172.21.15.92:0/608560661 wait complete. 2024-09-21T02:02:45.304 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:02:44.981+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:02:45.304 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:02:44.981+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:02:45.304 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:02:44.981+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:02:45.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:02:44.981+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:02:45.305 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:02:44 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:02:44.981+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:02:45.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:45 smithi099 ceph-mon[34404]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:45.613 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:45 smithi092 ceph-mon[31190]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:45.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T02:02:45.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:45 smithi136 ceph-mon[34754]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 -- 172.21.15.92:0/3707679028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 msgr2=0x7f3050110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 --2- 172.21.15.92:0/3707679028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f3050110330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3048009ca0 tx=0x7f304802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 -- 172.21.15.92:0/3707679028 shutdown_connections 2024-09-21T02:02:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 --2- 172.21.15.92:0/3707679028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f3050110330 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 --2- 172.21.15.92:0/3707679028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 0x7f305010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 --2- 172.21.15.92:0/3707679028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 0x7f3050104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 -- 172.21.15.92:0/3707679028 >> 172.21.15.92:0/3707679028 conn(0x7f3050079790 msgr2=0x7f3050079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 -- 172.21.15.92:0/3707679028 shutdown_connections 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 -- 172.21.15.92:0/3707679028 wait complete. 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.355+0000 7f3057136640 1 Processor -- start 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 -- start start 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 0x7f305019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 0x7f305019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f305019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3050113b20 con 0x7f305010de60 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30501139a0 con 0x7f3050104a30 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3057136640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3050113ca0 con 0x7f3050105380 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f305019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f305019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36142/0 (socket says 172.21.15.92:36142) 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 -- 172.21.15.92:0/2076806217 learned_addr learned my addr 172.21.15.92:0/2076806217 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3054ed2640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 0x7f305019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 -- 172.21.15.92:0/2076806217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 msgr2=0x7f305019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f304ffff640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 0x7f305019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 0x7f305019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 -- 172.21.15.92:0/2076806217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 msgr2=0x7f305019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 0x7f305019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 -- 172.21.15.92:0/2076806217 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30501a03e0 con 0x7f305010de60 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f3054ed2640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 0x7f305019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f304ffff640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 0x7f305019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.356+0000 7f30556d3640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f305019fcb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3048005cc0 tx=0x7f304802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.357+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048003cf0 con 0x7f305010de60 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.357+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30501a0670 con 0x7f305010de60 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.357+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30501a6e30 con 0x7f305010de60 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.357+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30480044f0 con 0x7f305010de60 2024-09-21T02:02:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.357+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304802fc70 con 0x7f305010de60 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.358+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30340052c0 con 0x7f305010de60 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.359+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3048004040 con 0x7f305010de60 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.360+0000 7f304dffb640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30400b65a0 0x7f30400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.360+0000 7f3054ed2640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30400b65a0 0x7f30400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.360+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f3048108a10 con 0x7f305010de60 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.360+0000 7f3054ed2640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30400b65a0 0x7f30400b8ab0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3044006fd0 tx=0x7f3044008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.363+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30480cf380 con 0x7f305010de60 2024-09-21T02:02:46.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:46 smithi092 ceph-mon[31190]: pgmap v292: 97 pgs: 97 active+clean; 584 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-21T02:02:46.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:46 smithi092 ceph-mon[31190]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-21T02:02:46.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.570+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3034000bf0 con 0x7f30400b65a0 2024-09-21T02:02:46.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:46 smithi099 ceph-mon[34404]: pgmap v292: 97 pgs: 97 active+clean; 584 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-21T02:02:46.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:46 smithi099 ceph-mon[34404]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-21T02:02:46.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.599+0000 7f304dffb640 1 -- 172.21.15.92:0/2076806217 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3034000bf0 con 0x7f30400b65a0 2024-09-21T02:02:46.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.601+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30400b65a0 msgr2=0x7f30400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.601+0000 7f3057136640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30400b65a0 0x7f30400b8ab0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3044006fd0 tx=0x7f3044008040 comp rx=0 tx=0).stop 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.601+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 msgr2=0x7f305019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.601+0000 7f3057136640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f305019fcb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3048005cc0 tx=0x7f304802f470 comp rx=0 tx=0).stop 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 shutdown_connections 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30400b65a0 0x7f30400b8ab0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305010de60 0x7f305019fcb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3050105380 0x7f305019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 --2- 172.21.15.92:0/2076806217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050104a30 0x7f305019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 >> 172.21.15.92:0/2076806217 conn(0x7f3050079790 msgr2=0x7f3050100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 shutdown_connections 2024-09-21T02:02:46.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:46.602+0000 7f3057136640 1 -- 172.21.15.92:0/2076806217 wait complete. 2024-09-21T02:02:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:46 smithi136 ceph-mon[34754]: pgmap v292: 97 pgs: 97 active+clean; 584 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.8 KiB/s wr, 19 op/s 2024-09-21T02:02:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:46 smithi136 ceph-mon[34754]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-21T02:02:47.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T02:02:47.430 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:47.586 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:47 smithi092 ceph-mon[31190]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:47.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:47 smithi136 ceph-mon[34754]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:47.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:47 smithi099 ceph-mon[34404]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 -- 172.21.15.92:0/2852723670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109ec0 msgr2=0x7ff4701106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 --2- 172.21.15.92:0/2852723670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109ec0 0x7ff4701106f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff468009900 tx=0x7ff46802a410 comp rx=0 tx=0).stop 2024-09-21T02:02:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 -- 172.21.15.92:0/2852723670 shutdown_connections 2024-09-21T02:02:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 --2- 172.21.15.92:0/2852723670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109ec0 0x7ff4701106f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 --2- 172.21.15.92:0/2852723670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff470109570 0x7ff470109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 --2- 172.21.15.92:0/2852723670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470103c30 0x7ff470104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 -- 172.21.15.92:0/2852723670 >> 172.21.15.92:0/2852723670 conn(0x7ff47007b8d0 msgr2=0x7ff47007bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 -- 172.21.15.92:0/2852723670 shutdown_connections 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.950+0000 7ff4760dd640 1 -- 172.21.15.92:0/2852723670 wait complete. 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 Processor -- start 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 -- start start 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff470103c30 0x7ff47019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 0x7ff47019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470109ec0 0x7ff47019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff470113ae0 con 0x7ff470109570 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff470113960 con 0x7ff470103c30 2024-09-21T02:02:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff4760dd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff470113c60 con 0x7ff470109ec0 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff46effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 0x7ff47019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff46effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 0x7ff47019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36160/0 (socket says 172.21.15.92:36160) 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff46effd640 1 -- 172.21.15.92:0/1153051998 learned_addr learned my addr 172.21.15.92:0/1153051998 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff46f7fe640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff470103c30 0x7ff47019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.951+0000 7ff46ffff640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470109ec0 0x7ff47019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46effd640 1 -- 172.21.15.92:0/1153051998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470109ec0 msgr2=0x7ff47019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46effd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470109ec0 0x7ff47019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46effd640 1 -- 172.21.15.92:0/1153051998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff470103c30 msgr2=0x7ff47019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46effd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff470103c30 0x7ff47019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46effd640 1 -- 172.21.15.92:0/1153051998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4701a04e0 con 0x7ff470109570 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46ffff640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470109ec0 0x7ff47019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46effd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 0x7ff47019ab10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c970 tx=0x7ff45800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458007bb0 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff458007d50 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4701a07d0 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45800f450 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.952+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4701a6f30 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.953+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff470104040 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.955+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff458004eb0 con 0x7ff470109570 2024-09-21T02:02:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.955+0000 7ff46cff9640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff45c0b6620 0x7ff45c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.955+0000 7ff46f7fe640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff45c0b6620 0x7ff45c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.956+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff4580e7400 con 0x7ff470109570 2024-09-21T02:02:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.956+0000 7ff46f7fe640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff45c0b6620 0x7ff45c0b8b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff460005fd0 tx=0x7ff460009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:47.959+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff458014030 con 0x7ff470109570 2024-09-21T02:02:48.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.167+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff470063800 con 0x7ff45c0b6620 2024-09-21T02:02:48.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.173+0000 7ff46cff9640 1 -- 172.21.15.92:0/1153051998 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff470063800 con 0x7ff45c0b6620 2024-09-21T02:02:48.174 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/qa/sv_0/2ae9596d-c5e2-4f8e-b5e1-ef31982908e5 2024-09-21T02:02:48.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.175+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff45c0b6620 msgr2=0x7ff45c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:48.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.175+0000 7ff4760dd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff45c0b6620 0x7ff45c0b8b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff460005fd0 tx=0x7ff460009680 comp rx=0 tx=0).stop 2024-09-21T02:02:48.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.175+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 msgr2=0x7ff47019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:48.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.175+0000 7ff4760dd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 0x7ff47019ab10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c970 tx=0x7ff45800ce30 comp rx=0 tx=0).stop 2024-09-21T02:02:48.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 shutdown_connections 2024-09-21T02:02:48.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff45c0b6620 0x7ff45c0b8b30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:48.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff470109ec0 0x7ff47019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:48.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff470109570 0x7ff47019ab10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:48.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 --2- 172.21.15.92:0/1153051998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff470103c30 0x7ff47019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:48.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 >> 172.21.15.92:0/1153051998 conn(0x7ff47007b8d0 msgr2=0x7ff4701076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:48.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 shutdown_connections 2024-09-21T02:02:48.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:48.176+0000 7ff4760dd640 1 -- 172.21.15.92:0/1153051998 wait complete. 2024-09-21T02:02:48.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:48 smithi092 ceph-mon[31190]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-09-21T02:02:48.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:48 smithi136 ceph-mon[34754]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-09-21T02:02:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:48 smithi099 ceph-mon[34404]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-09-21T02:02:48.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T02:02:49.129 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:49.534 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:49 smithi092 ceph-mon[31190]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 -- 172.21.15.92:0/289903710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47781083a0 msgr2=0x7f47781087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 --2- 172.21.15.92:0/289903710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47781083a0 0x7f47781087b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f476c009900 tx=0x7f476c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 -- 172.21.15.92:0/289903710 shutdown_connections 2024-09-21T02:02:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 --2- 172.21.15.92:0/289903710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4778108cf0 0x7f477810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 --2- 172.21.15.92:0/289903710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47781083a0 0x7f47781087b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 --2- 172.21.15.92:0/289903710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4778102a60 0x7f4778102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 -- 172.21.15.92:0/289903710 >> 172.21.15.92:0/289903710 conn(0x7f4778079730 msgr2=0x7f4778079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 -- 172.21.15.92:0/289903710 shutdown_connections 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.629+0000 7f477e2ea640 1 -- 172.21.15.92:0/289903710 wait complete. 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 Processor -- start 2024-09-21T02:02:49.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 -- start start 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4778102a60 0x7f477819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47781083a0 0x7f477819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 0x7f477819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4778112960 con 0x7f4778108cf0 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47781127e0 con 0x7f47781083a0 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.630+0000 7f477e2ea640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4778112ae0 con 0x7f4778102a60 2024-09-21T02:02:49.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 0x7f477819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 0x7f477819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36182/0 (socket says 172.21.15.92:36182) 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 -- 172.21.15.92:0/858215175 learned_addr learned my addr 172.21.15.92:0/858215175 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f47777fe640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47781083a0 0x7f477819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f4777fff640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4778102a60 0x7f477819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 -- 172.21.15.92:0/858215175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4778102a60 msgr2=0x7f477819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4778102a60 0x7f477819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 -- 172.21.15.92:0/858215175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47781083a0 msgr2=0x7f477819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47781083a0 0x7f477819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 -- 172.21.15.92:0/858215175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a0450 con 0x7f4778108cf0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477c887640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 0x7f477819fd20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f477000eaa0 tx=0x7f477000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477000cdb0 con 0x7f4778108cf0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781a0740 con 0x7f4778108cf0 2024-09-21T02:02:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4770005020 con 0x7f4778108cf0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700116d0 con 0x7f4778108cf0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.631+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a6ea0 con 0x7f4778108cf0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.632+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778102e70 con 0x7f4778108cf0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.634+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f47700054a0 con 0x7f4778108cf0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.634+0000 7f47757fa640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47600b65a0 0x7f47600b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.635+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f47700e74f0 con 0x7f4778108cf0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.635+0000 7f4777fff640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47600b65a0 0x7f47600b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.636+0000 7f4777fff640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47600b65a0 0x7f47600b8ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4768005fd0 tx=0x7f4768005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.640+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4770016030 con 0x7f4778108cf0 2024-09-21T02:02:49.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:49 smithi136 ceph-mon[34754]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:49.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:49 smithi099 ceph-mon[34404]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.846+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4778063800 con 0x7f47600b65a0 2024-09-21T02:02:49.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.874+0000 7f47757fa640 1 -- 172.21.15.92:0/858215175 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4778063800 con 0x7f47600b65a0 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47600b65a0 msgr2=0x7f47600b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47600b65a0 0x7f47600b8ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4768005fd0 tx=0x7f4768005850 comp rx=0 tx=0).stop 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 msgr2=0x7f477819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 0x7f477819fd20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f477000eaa0 tx=0x7f477000ef60 comp rx=0 tx=0).stop 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 shutdown_connections 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47600b65a0 0x7f47600b8ab0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4778108cf0 0x7f477819fd20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47781083a0 0x7f477819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 --2- 172.21.15.92:0/858215175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4778102a60 0x7f477819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 >> 172.21.15.92:0/858215175 conn(0x7f4778079730 msgr2=0x7f477806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 shutdown_connections 2024-09-21T02:02:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:49.876+0000 7f477e2ea640 1 -- 172.21.15.92:0/858215175 wait complete. 2024-09-21T02:02:50.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:50 smithi092 ceph-mon[31190]: pgmap v294: 97 pgs: 97 active+clean; 612 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-09-21T02:02:50.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:50 smithi092 ceph-mon[31190]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:50.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T02:02:50.739 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:50.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:50 smithi136 ceph-mon[34754]: pgmap v294: 97 pgs: 97 active+clean; 612 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-09-21T02:02:50.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:50 smithi136 ceph-mon[34754]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:50.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:50 smithi099 ceph-mon[34404]: pgmap v294: 97 pgs: 97 active+clean; 612 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-09-21T02:02:50.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:50 smithi099 ceph-mon[34404]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:51.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 -- 172.21.15.92:0/1502828762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd608104860 msgr2=0x7fd608104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 --2- 172.21.15.92:0/1502828762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd608104860 0x7fd608104c70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ca0 tx=0x7fd5f802a8e0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 -- 172.21.15.92:0/1502828762 shutdown_connections 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 --2- 172.21.15.92:0/1502828762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd60810de10 0x7fd6081102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 --2- 172.21.15.92:0/1502828762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd608105290 0x7fd60810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 --2- 172.21.15.92:0/1502828762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd608104860 0x7fd608104c70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 -- 172.21.15.92:0/1502828762 >> 172.21.15.92:0/1502828762 conn(0x7fd608079730 msgr2=0x7fd608079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 -- 172.21.15.92:0/1502828762 shutdown_connections 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.227+0000 7fd60f714640 1 -- 172.21.15.92:0/1502828762 wait complete. 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 Processor -- start 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 -- start start 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd608104860 0x7fd60819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd608105290 0x7fd60819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 0x7fd60819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd608113ac0 con 0x7fd60810de10 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd608113940 con 0x7fd608104860 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60f714640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd608113c40 con 0x7fd608105290 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 0x7fd60819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 0x7fd60819fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36190/0 (socket says 172.21.15.92:36190) 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 -- 172.21.15.92:0/818923508 learned_addr learned my addr 172.21.15.92:0/818923508 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60d4b0640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd608104860 0x7fd60819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 -- 172.21.15.92:0/818923508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd608105290 msgr2=0x7fd60819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd608105290 0x7fd60819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 -- 172.21.15.92:0/818923508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd608104860 msgr2=0x7fd60819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd608104860 0x7fd60819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.228+0000 7fd60dcb1640 1 -- 172.21.15.92:0/818923508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6081a04c0 con 0x7fd60810de10 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.229+0000 7fd60dcb1640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 0x7fd60819fd90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd60000d750 tx=0x7fd60000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.229+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600017070 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.229+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6000054d0 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.229+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600004c60 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.229+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6081a07b0 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.229+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6081a6f10 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.230+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd608104c70 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.232+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd600009bd0 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.233+0000 7fd6067fc640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5f00b65a0 0x7fd5f00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.233+0000 7fd60d4b0640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5f00b65a0 0x7fd5f00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.233+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd6000e7140 con 0x7fd60810de10 2024-09-21T02:02:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.234+0000 7fd60d4b0640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5f00b65a0 0x7fd5f00b8ab0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8000f80 tx=0x7fd5f80093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.236+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6000ada20 con 0x7fd60810de10 2024-09-21T02:02:51.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.445+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd608063800 con 0x7fd5f00b65a0 2024-09-21T02:02:51.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.450+0000 7fd6067fc640 1 -- 172.21.15.92:0/818923508 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd608063800 con 0x7fd5f00b65a0 2024-09-21T02:02:51.451 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/qa/sv_1/80f7c1c1-cbdd-4a71-a4cb-e351ed1898ee 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5f00b65a0 msgr2=0x7fd5f00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5f00b65a0 0x7fd5f00b8ab0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8000f80 tx=0x7fd5f80093f0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 msgr2=0x7fd60819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 0x7fd60819fd90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd60000d750 tx=0x7fd60000dc10 comp rx=0 tx=0).stop 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 shutdown_connections 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5f00b65a0 0x7fd5f00b8ab0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd60810de10 0x7fd60819fd90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd608105290 0x7fd60819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 --2- 172.21.15.92:0/818923508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd608104860 0x7fd60819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:51.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 >> 172.21.15.92:0/818923508 conn(0x7fd608079730 msgr2=0x7fd6080ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:51.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 shutdown_connections 2024-09-21T02:02:51.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:51.452+0000 7fd60f714640 1 -- 172.21.15.92:0/818923508 wait complete. 2024-09-21T02:02:52.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:02:52.349 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:52.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:52 smithi092 ceph-mon[31190]: pgmap v295: 97 pgs: 97 active+clean; 613 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.2 KiB/s wr, 24 op/s 2024-09-21T02:02:52.470 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:52 smithi092 ceph-mon[31190]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:52.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 -- 172.21.15.92:0/1468387974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c108d30 msgr2=0x7f1b4c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 --2- 172.21.15.92:0/1468387974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c108d30 0x7f1b4c10f560 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009ca0 tx=0x7f1b4402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 -- 172.21.15.92:0/1468387974 shutdown_connections 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 --2- 172.21.15.92:0/1468387974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c108d30 0x7f1b4c10f560 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 --2- 172.21.15.92:0/1468387974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 --2- 172.21.15.92:0/1468387974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b4c102aa0 0x7f1b4c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 -- 172.21.15.92:0/1468387974 >> 172.21.15.92:0/1468387974 conn(0x7f1b4c0797e0 msgr2=0x7f1b4c079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 -- 172.21.15.92:0/1468387974 shutdown_connections 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.779+0000 7f1b52038640 1 -- 172.21.15.92:0/1468387974 wait complete. 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 Processor -- start 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 -- start start 2024-09-21T02:02:52.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b4c102aa0 0x7f1b4c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b4c108d30 0x7f1b4c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c112a20 con 0x7f1b4c1083e0 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1128a0 con 0x7f1b4c102aa0 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b52038640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c112ba0 con 0x7f1b4c108d30 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b4affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.780+0000 7f1b4affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36204/0 (socket says 172.21.15.92:36204) 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 -- 172.21.15.92:0/1281419815 learned_addr learned my addr 172.21.15.92:0/1281419815 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4b7fe640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b4c102aa0 0x7f1b4c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4bfff640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b4c108d30 0x7f1b4c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 -- 172.21.15.92:0/1281419815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b4c108d30 msgr2=0x7f1b4c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b4c108d30 0x7f1b4c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 -- 172.21.15.92:0/1281419815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b4c102aa0 msgr2=0x7f1b4c19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:52.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b4c102aa0 0x7f1b4c19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 -- 172.21.15.92:0/1281419815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c1a6540 con 0x7f1b4c1083e0 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b4affd640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c19b770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b450 tx=0x7f1b4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40015020 con 0x7f1b4c1083e0 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b40011d40 con 0x7f1b4c1083e0 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1a6830 con 0x7f1b4c1083e0 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000f340 con 0x7f1b4c1083e0 2024-09-21T02:02:52.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.781+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1a6df0 con 0x7f1b4c1083e0 2024-09-21T02:02:52.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:52 smithi136 ceph-mon[34754]: pgmap v295: 97 pgs: 97 active+clean; 613 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.2 KiB/s wr, 24 op/s 2024-09-21T02:02:52.786 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:52 smithi136 ceph-mon[34754]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.782+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c102eb0 con 0x7f1b4c1083e0 2024-09-21T02:02:52.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.784+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1b40018020 con 0x7f1b4c1083e0 2024-09-21T02:02:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.784+0000 7f1b48ff9640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1b340b65a0 0x7f1b340b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.784+0000 7f1b4b7fe640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1b340b65a0 0x7f1b340b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.784+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f1b400e7230 con 0x7f1b4c1083e0 2024-09-21T02:02:52.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.785+0000 7f1b4b7fe640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1b340b65a0 0x7f1b340b8ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0058c0 tx=0x7f1b3c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:52.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:52.788+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1b4001f020 con 0x7f1b4c1083e0 2024-09-21T02:02:52.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:52 smithi099 ceph-mon[34404]: pgmap v295: 97 pgs: 97 active+clean; 613 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.2 KiB/s wr, 24 op/s 2024-09-21T02:02:52.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:52 smithi099 ceph-mon[34404]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:02:53.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.005+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4c06bc90 con 0x7f1b4c1083e0 2024-09-21T02:02:53.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.006+0000 7f1b48ff9640 1 -- 172.21.15.92:0/1281419815 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11963 (secure 0 0 0) 0x7f1b4c06bc90 con 0x7f1b4c1083e0 2024-09-21T02:02:53.007 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:53.007 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"btime":"2024-09-21T02:02:43:311503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:02:42.638635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[14799,24610,24623,24619,24637]},"id":1}]} 2024-09-21T02:02:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1b340b65a0 msgr2=0x7f1b340b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1b340b65a0 0x7f1b340b8ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0058c0 tx=0x7f1b3c00a250 comp rx=0 tx=0).stop 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 msgr2=0x7f1b4c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c19b770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b450 tx=0x7f1b4000b910 comp rx=0 tx=0).stop 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 shutdown_connections 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1b340b65a0 0x7f1b340b8ab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b4c108d30 0x7f1b4c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b4c1083e0 0x7f1b4c19b770 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 --2- 172.21.15.92:0/1281419815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b4c102aa0 0x7f1b4c19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 >> 172.21.15.92:0/1281419815 conn(0x7f1b4c0797e0 msgr2=0x7f1b4c105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 shutdown_connections 2024-09-21T02:02:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:53.008+0000 7f1b52038640 1 -- 172.21.15.92:0/1281419815 wait complete. 2024-09-21T02:02:53.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:53 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1281419815' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:53.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:02:53.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:53 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1281419815' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:53.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:53 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1281419815' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:02:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:54.424 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:54 smithi092 ceph-mon[31190]: pgmap v296: 97 pgs: 97 active+clean; 613 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 24 op/s 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.423+0000 7f3143acd640 1 -- 172.21.15.92:0/565518684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c1094a0 msgr2=0x7f313c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.423+0000 7f3143acd640 1 --2- 172.21.15.92:0/565518684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c1094a0 0x7f313c1098b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f312c009900 tx=0x7f312c02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 -- 172.21.15.92:0/565518684 shutdown_connections 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 --2- 172.21.15.92:0/565518684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f313c109df0 0x7f313c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 --2- 172.21.15.92:0/565518684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c1094a0 0x7f313c1098b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 --2- 172.21.15.92:0/565518684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 0x7f313c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 -- 172.21.15.92:0/565518684 >> 172.21.15.92:0/565518684 conn(0x7f313c07b850 msgr2=0x7f313c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 -- 172.21.15.92:0/565518684 shutdown_connections 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.424+0000 7f3143acd640 1 -- 172.21.15.92:0/565518684 wait complete. 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 Processor -- start 2024-09-21T02:02:54.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 -- start start 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 0x7f313c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f313c1094a0 0x7f313c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 0x7f313c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f313c113ad0 con 0x7f313c109df0 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f313c113950 con 0x7f313c1094a0 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3143acd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f313c113c50 con 0x7f313c103b60 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 0x7f313c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 0x7f313c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36286/0 (socket says 172.21.15.92:36286) 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 -- 172.21.15.92:0/396726917 learned_addr learned my addr 172.21.15.92:0/396726917 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3141068640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f313c1094a0 0x7f313c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f3141869640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 0x7f313c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 -- 172.21.15.92:0/396726917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 msgr2=0x7f313c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 0x7f313c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 -- 172.21.15.92:0/396726917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f313c1094a0 msgr2=0x7f313c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.425+0000 7f314206a640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f313c1094a0 0x7f313c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f314206a640 1 -- 172.21.15.92:0/396726917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c1a0430 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f3141869640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 0x7f313c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f314206a640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 0x7f313c19fd00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f313400eaa0 tx=0x7f313400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313400cdb0 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3134005020 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c1a0720 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340116d0 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.426+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c1a6e80 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.427+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c103f70 con 0x7f313c109df0 2024-09-21T02:02:54.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.428+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f31340051d0 con 0x7f313c109df0 2024-09-21T02:02:54.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.429+0000 7f3132ffd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31280b65a0 0x7f31280b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:54.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.430+0000 7f3141869640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31280b65a0 0x7f31280b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:54.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.430+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f31340e79e0 con 0x7f313c109df0 2024-09-21T02:02:54.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.430+0000 7f3141869640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31280b65a0 0x7f31280b8ab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:54.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.433+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3134016030 con 0x7f313c109df0 2024-09-21T02:02:54.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.610+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f313c19b5f0 con 0x7f313c109df0 2024-09-21T02:02:54.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.612+0000 7f3132ffd640 1 -- 172.21.15.92:0/396726917 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22268 (secure 0 0 0) 0x7f313c19b5f0 con 0x7f313c109df0 2024-09-21T02:02:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:54.613 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:02:31.545307+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31280b65a0 msgr2=0x7f31280b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31280b65a0 0x7f31280b8ab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138008040 comp rx=0 tx=0).stop 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 msgr2=0x7f313c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 0x7f313c19fd00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f313400eaa0 tx=0x7f313400ef60 comp rx=0 tx=0).stop 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 shutdown_connections 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31280b65a0 0x7f31280b8ab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313c109df0 0x7f313c19fd00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f313c1094a0 0x7f313c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 --2- 172.21.15.92:0/396726917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f313c103b60 0x7f313c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 >> 172.21.15.92:0/396726917 conn(0x7f313c07b850 msgr2=0x7f313c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 shutdown_connections 2024-09-21T02:02:54.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:54.614+0000 7f3143acd640 1 -- 172.21.15.92:0/396726917 wait complete. 2024-09-21T02:02:54.732 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:54 smithi099 ceph-mon[34404]: pgmap v296: 97 pgs: 97 active+clean; 613 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 24 op/s 2024-09-21T02:02:54.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:54 smithi136 ceph-mon[34754]: pgmap v296: 97 pgs: 97 active+clean; 613 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.8 KiB/s wr, 24 op/s 2024-09-21T02:02:55.403 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T02:02:55.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:02:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:55.685 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:55 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/396726917' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:55.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:55 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/396726917' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:55.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:55 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/396726917' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 -- 172.21.15.92:0/3491012240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 msgr2=0x7efe6c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 --2- 172.21.15.92:0/3491012240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c110650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efe64009ca0 tx=0x7efe6402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 -- 172.21.15.92:0/3491012240 shutdown_connections 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 --2- 172.21.15.92:0/3491012240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c110650 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 --2- 172.21.15.92:0/3491012240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c1094d0 0x7efe6c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 --2- 172.21.15.92:0/3491012240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe6c103b90 0x7efe6c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 -- 172.21.15.92:0/3491012240 >> 172.21.15.92:0/3491012240 conn(0x7efe6c0ff600 msgr2=0x7efe6c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 -- 172.21.15.92:0/3491012240 shutdown_connections 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.064+0000 7efe73451640 1 -- 172.21.15.92:0/3491012240 wait complete. 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 Processor -- start 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 -- start start 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c103b90 0x7efe6c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe6c1094d0 0x7efe6c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c113ae0 con 0x7efe6c109e20 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c113960 con 0x7efe6c103b90 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe73451640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c113c60 con 0x7efe6c1094d0 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe719ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe711ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c103b90 0x7efe6c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe719ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36304/0 (socket says 172.21.15.92:36304) 2024-09-21T02:02:56.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.065+0000 7efe719ee640 1 -- 172.21.15.92:0/124392640 learned_addr learned my addr 172.21.15.92:0/124392640 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe709ec640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe6c1094d0 0x7efe6c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe719ee640 1 -- 172.21.15.92:0/124392640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe6c1094d0 msgr2=0x7efe6c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe719ee640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe6c1094d0 0x7efe6c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe719ee640 1 -- 172.21.15.92:0/124392640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c103b90 msgr2=0x7efe6c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe719ee640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c103b90 0x7efe6c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe719ee640 1 -- 172.21.15.92:0/124392640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1a03e0 con 0x7efe6c109e20 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe711ed640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c103b90 0x7efe6c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe719ee640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c19fcb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe640098d0 tx=0x7efe640317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64003b70 con 0x7efe6c109e20 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe64003d10 con 0x7efe6c109e20 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1a0670 con 0x7efe6c109e20 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6402fa20 con 0x7efe6c109e20 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.066+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1a6e30 con 0x7efe6c109e20 2024-09-21T02:02:56.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.067+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c103fa0 con 0x7efe6c109e20 2024-09-21T02:02:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.070+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efe6402fbc0 con 0x7efe6c109e20 2024-09-21T02:02:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.070+0000 7efe627fc640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe580b6620 0x7efe580b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.070+0000 7efe711ed640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe580b6620 0x7efe580b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.071+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7efe641086b0 con 0x7efe6c109e20 2024-09-21T02:02:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.071+0000 7efe711ed640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe580b6620 0x7efe580b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe68005fd0 tx=0x7efe6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.074+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe640cef90 con 0x7efe6c109e20 2024-09-21T02:02:56.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.252+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe6c10bc40 con 0x7efe580b6620 2024-09-21T02:02:56.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.257+0000 7efe627fc640 1 -- 172.21.15.92:0/124392640 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258747 (secure 0 0 0) 0x7efe6c10bc40 con 0x7efe580b6620 2024-09-21T02:02:56.259 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:56.260 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:02:56.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe580b6620 msgr2=0x7efe580b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe580b6620 0x7efe580b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe68005fd0 tx=0x7efe6800b390 comp rx=0 tx=0).stop 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 msgr2=0x7efe6c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c19fcb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe640098d0 tx=0x7efe640317b0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 shutdown_connections 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe580b6620 0x7efe580b8b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe6c109e20 0x7efe6c19fcb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe6c1094d0 0x7efe6c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 --2- 172.21.15.92:0/124392640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6c103b90 0x7efe6c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 >> 172.21.15.92:0/124392640 conn(0x7efe6c0ff600 msgr2=0x7efe6c108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 shutdown_connections 2024-09-21T02:02:56.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:56.261+0000 7efe73451640 1 -- 172.21.15.92:0/124392640 wait complete. 2024-09-21T02:02:56.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:56 smithi136 ceph-mon[34754]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-21T02:02:56.784 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:56 smithi092 ceph-mon[31190]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-21T02:02:56.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:56 smithi099 ceph-mon[34404]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-21T02:02:57.058 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-21T02:02:55.027219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652329,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":243,"num_read_kb":216,"num_write":359,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":406756,"kb_used_data":11748,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124535068,"statfs":{"total":1151940427776,"available":1151523909632,"internally_reserved":0,"allocated":12029952,"data_stored":6611919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56689,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":148,"num_read_kb":135,"num_write":109,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003349"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966600+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:02:31.966600+0000","last_peered":"2024-09-21T02:02:31.966600+0000","last_clean":"2024-09-21T02:02:31.966600+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:02:31.966600+0000","last_undegraded":"2024-09-21T02:02:31.966600+0000","last_fullsized":"2024-09-21T02:02:31.966600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984212+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:02:31.984212+0000","last_peered":"2024-09-21T02:02:31.984212+0000","last_clean":"2024-09-21T02:02:31.984212+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:02:31.984212+0000","last_undegraded":"2024-09-21T02:02:31.984212+0000","last_fullsized":"2024-09-21T02:02:31.984212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984212+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:02:31.984212+0000","last_peered":"2024-09-21T02:02:31.984212+0000","last_clean":"2024-09-21T02:02:31.984212+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:02:31.984212+0000","last_undegraded":"2024-09-21T02:02:31.984212+0000","last_fullsized":"2024-09-21T02:02:31.984212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984122+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:02:31.984122+0000","last_peered":"2024-09-21T02:02:31.984122+0000","last_clean":"2024-09-21T02:02:31.984122+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:02:31.984122+0000","last_undegraded":"2024-09-21T02:02:31.984122+0000","last_fullsized":"2024-09-21T02:02:31.984122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979942+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:02:31.979942+0000","last_peered":"2024-09-21T02:02:31.979942+0000","last_clean":"2024-09-21T02:02:31.979942+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:02:31.979942+0000","last_undegraded":"2024-09-21T02:02:31.979942+0000","last_fullsized":"2024-09-21T02:02:31.979942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.964059+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:02:31.964059+0000","last_peered":"2024-09-21T02:02:31.964059+0000","last_clean":"2024-09-21T02:02:31.964059+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:02:31.964059+0000","last_undegraded":"2024-09-21T02:02:31.964059+0000","last_fullsized":"2024-09-21T02:02:31.964059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979814+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:02:31.979814+0000","last_peered":"2024-09-21T02:02:31.979814+0000","last_clean":"2024-09-21T02:02:31.979814+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:02:31.979814+0000","last_undegraded":"2024-09-21T02:02:31.979814+0000","last_fullsized":"2024-09-21T02:02:31.979814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980365+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:02:31.980365+0000","last_peered":"2024-09-21T02:02:31.980365+0000","last_clean":"2024-09-21T02:02:31.980365+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:02:31.980365+0000","last_undegraded":"2024-09-21T02:02:31.980365+0000","last_fullsized":"2024-09-21T02:02:31.980365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982507+0000","last_change":"2024-09-21T02:02:30.975000+0000","last_active":"2024-09-21T02:02:31.982507+0000","last_peered":"2024-09-21T02:02:31.982507+0000","last_clean":"2024-09-21T02:02:31.982507+0000","last_became_active":"2024-09-21T02:02:30.974507+0000","last_became_peered":"2024-09-21T02:02:30.974507+0000","last_unstale":"2024-09-21T02:02:31.982507+0000","last_undegraded":"2024-09-21T02:02:31.982507+0000","last_fullsized":"2024-09-21T02:02:31.982507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:19.152162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.964552+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:02:31.964552+0000","last_peered":"2024-09-21T02:02:31.964552+0000","last_clean":"2024-09-21T02:02:31.964552+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:02:31.964552+0000","last_undegraded":"2024-09-21T02:02:31.964552+0000","last_fullsized":"2024-09-21T02:02:31.964552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981302+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:02:31.981302+0000","last_peered":"2024-09-21T02:02:31.981302+0000","last_clean":"2024-09-21T02:02:31.981302+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:02:31.981302+0000","last_undegraded":"2024-09-21T02:02:31.981302+0000","last_fullsized":"2024-09-21T02:02:31.981302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984392+0000","last_change":"2024-09-21T02:02:30.975791+0000","last_active":"2024-09-21T02:02:31.984392+0000","last_peered":"2024-09-21T02:02:31.984392+0000","last_clean":"2024-09-21T02:02:31.984392+0000","last_became_active":"2024-09-21T02:02:30.974622+0000","last_became_peered":"2024-09-21T02:02:30.974622+0000","last_unstale":"2024-09-21T02:02:31.984392+0000","last_undegraded":"2024-09-21T02:02:31.984392+0000","last_fullsized":"2024-09-21T02:02:31.984392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:59.033132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980429+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:02:31.980429+0000","last_peered":"2024-09-21T02:02:31.980429+0000","last_clean":"2024-09-21T02:02:31.980429+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:02:31.980429+0000","last_undegraded":"2024-09-21T02:02:31.980429+0000","last_fullsized":"2024-09-21T02:02:31.980429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984127+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:02:31.984127+0000","last_peered":"2024-09-21T02:02:31.984127+0000","last_clean":"2024-09-21T02:02:31.984127+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:02:31.984127+0000","last_undegraded":"2024-09-21T02:02:31.984127+0000","last_fullsized":"2024-09-21T02:02:31.984127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.983847+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:02:31.983847+0000","last_peered":"2024-09-21T02:02:31.983847+0000","last_clean":"2024-09-21T02:02:31.983847+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:02:31.983847+0000","last_undegraded":"2024-09-21T02:02:31.983847+0000","last_fullsized":"2024-09-21T02:02:31.983847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981419+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:02:31.981419+0000","last_peered":"2024-09-21T02:02:31.981419+0000","last_clean":"2024-09-21T02:02:31.981419+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:02:31.981419+0000","last_undegraded":"2024-09-21T02:02:31.981419+0000","last_fullsized":"2024-09-21T02:02:31.981419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966769+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:02:31.966769+0000","last_peered":"2024-09-21T02:02:31.966769+0000","last_clean":"2024-09-21T02:02:31.966769+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:02:31.966769+0000","last_undegraded":"2024-09-21T02:02:31.966769+0000","last_fullsized":"2024-09-21T02:02:31.966769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.965578+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:02:31.965578+0000","last_peered":"2024-09-21T02:02:31.965578+0000","last_clean":"2024-09-21T02:02:31.965578+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:02:31.965578+0000","last_undegraded":"2024-09-21T02:02:31.965578+0000","last_fullsized":"2024-09-21T02:02:31.965578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.986741+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:02:31.986741+0000","last_peered":"2024-09-21T02:02:31.986741+0000","last_clean":"2024-09-21T02:02:31.986741+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:02:31.986741+0000","last_undegraded":"2024-09-21T02:02:31.986741+0000","last_fullsized":"2024-09-21T02:02:31.986741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966245+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:02:31.966245+0000","last_peered":"2024-09-21T02:02:31.966245+0000","last_clean":"2024-09-21T02:02:31.966245+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:02:31.966245+0000","last_undegraded":"2024-09-21T02:02:31.966245+0000","last_fullsized":"2024-09-21T02:02:31.966245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980643+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:02:31.980643+0000","last_peered":"2024-09-21T02:02:31.980643+0000","last_clean":"2024-09-21T02:02:31.980643+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:02:31.980643+0000","last_undegraded":"2024-09-21T02:02:31.980643+0000","last_fullsized":"2024-09-21T02:02:31.980643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982484+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:02:31.982484+0000","last_peered":"2024-09-21T02:02:31.982484+0000","last_clean":"2024-09-21T02:02:31.982484+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:02:31.982484+0000","last_undegraded":"2024-09-21T02:02:31.982484+0000","last_fullsized":"2024-09-21T02:02:31.982484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980277+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:02:31.980277+0000","last_peered":"2024-09-21T02:02:31.980277+0000","last_clean":"2024-09-21T02:02:31.980277+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:02:31.980277+0000","last_undegraded":"2024-09-21T02:02:31.980277+0000","last_fullsized":"2024-09-21T02:02:31.980277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981000+0000","last_change":"2024-09-21T02:02:30.973693+0000","last_active":"2024-09-21T02:02:31.981000+0000","last_peered":"2024-09-21T02:02:31.981000+0000","last_clean":"2024-09-21T02:02:31.981000+0000","last_became_active":"2024-09-21T02:02:30.973350+0000","last_became_peered":"2024-09-21T02:02:30.973350+0000","last_unstale":"2024-09-21T02:02:31.981000+0000","last_undegraded":"2024-09-21T02:02:31.981000+0000","last_fullsized":"2024-09-21T02:02:31.981000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:54.848940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981004+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:02:31.981004+0000","last_peered":"2024-09-21T02:02:31.981004+0000","last_clean":"2024-09-21T02:02:31.981004+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:02:31.981004+0000","last_undegraded":"2024-09-21T02:02:31.981004+0000","last_fullsized":"2024-09-21T02:02:31.981004+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.985951+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:02:31.985951+0000","last_peered":"2024-09-21T02:02:31.985951+0000","last_clean":"2024-09-21T02:02:31.985951+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:02:31.985951+0000","last_undegraded":"2024-09-21T02:02:31.985951+0000","last_fullsized":"2024-09-21T02:02:31.985951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966470+0000","last_change":"2024-09-21T02:02:30.976395+0000","last_active":"2024-09-21T02:02:31.966470+0000","last_peered":"2024-09-21T02:02:31.966470+0000","last_clean":"2024-09-21T02:02:31.966470+0000","last_became_active":"2024-09-21T02:02:30.976001+0000","last_became_peered":"2024-09-21T02:02:30.976001+0000","last_unstale":"2024-09-21T02:02:31.966470+0000","last_undegraded":"2024-09-21T02:02:31.966470+0000","last_fullsized":"2024-09-21T02:02:31.966470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:39.724131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980134+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:02:31.980134+0000","last_peered":"2024-09-21T02:02:31.980134+0000","last_clean":"2024-09-21T02:02:31.980134+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:02:31.980134+0000","last_undegraded":"2024-09-21T02:02:31.980134+0000","last_fullsized":"2024-09-21T02:02:31.980134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984239+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:02:31.984239+0000","last_peered":"2024-09-21T02:02:31.984239+0000","last_clean":"2024-09-21T02:02:31.984239+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:02:31.984239+0000","last_undegraded":"2024-09-21T02:02:31.984239+0000","last_fullsized":"2024-09-21T02:02:31.984239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981470+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:02:31.981470+0000","last_peered":"2024-09-21T02:02:31.981470+0000","last_clean":"2024-09-21T02:02:31.981470+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:02:31.981470+0000","last_undegraded":"2024-09-21T02:02:31.981470+0000","last_fullsized":"2024-09-21T02:02:31.981470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984906+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:02:31.984906+0000","last_peered":"2024-09-21T02:02:31.984906+0000","last_clean":"2024-09-21T02:02:31.984906+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:02:31.984906+0000","last_undegraded":"2024-09-21T02:02:31.984906+0000","last_fullsized":"2024-09-21T02:02:31.984906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966378+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:02:31.966378+0000","last_peered":"2024-09-21T02:02:31.966378+0000","last_clean":"2024-09-21T02:02:31.966378+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:02:31.966378+0000","last_undegraded":"2024-09-21T02:02:31.966378+0000","last_fullsized":"2024-09-21T02:02:31.966378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979570+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:02:31.979570+0000","last_peered":"2024-09-21T02:02:31.979570+0000","last_clean":"2024-09-21T02:02:31.979570+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:02:31.979570+0000","last_undegraded":"2024-09-21T02:02:31.979570+0000","last_fullsized":"2024-09-21T02:02:31.979570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:51.323669+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:02:51.323669+0000","last_peered":"2024-09-21T02:02:51.323669+0000","last_clean":"2024-09-21T02:02:51.323669+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:02:51.323669+0000","last_undegraded":"2024-09-21T02:02:51.323669+0000","last_fullsized":"2024-09-21T02:02:51.323669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980764+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:02:31.980764+0000","last_peered":"2024-09-21T02:02:31.980764+0000","last_clean":"2024-09-21T02:02:31.980764+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:02:31.980764+0000","last_undegraded":"2024-09-21T02:02:31.980764+0000","last_fullsized":"2024-09-21T02:02:31.980764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.046806+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:02:41.046806+0000","last_peered":"2024-09-21T02:02:41.046806+0000","last_clean":"2024-09-21T02:02:41.046806+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:02:41.046806+0000","last_undegraded":"2024-09-21T02:02:41.046806+0000","last_fullsized":"2024-09-21T02:02:41.046806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:40.035494+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:02:40.035494+0000","last_peered":"2024-09-21T02:02:40.035494+0000","last_clean":"2024-09-21T02:02:40.035494+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:02:40.035494+0000","last_undegraded":"2024-09-21T02:02:40.035494+0000","last_fullsized":"2024-09-21T02:02:40.035494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980613+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:02:31.980613+0000","last_peered":"2024-09-21T02:02:31.980613+0000","last_clean":"2024-09-21T02:02:31.980613+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:02:31.980613+0000","last_undegraded":"2024-09-21T02:02:31.980613+0000","last_fullsized":"2024-09-21T02:02:31.980613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.306642+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:02:41.306642+0000","last_peered":"2024-09-21T02:02:41.306642+0000","last_clean":"2024-09-21T02:02:41.306642+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:02:41.306642+0000","last_undegraded":"2024-09-21T02:02:41.306642+0000","last_fullsized":"2024-09-21T02:02:41.306642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.965431+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:02:31.965431+0000","last_peered":"2024-09-21T02:02:31.965431+0000","last_clean":"2024-09-21T02:02:31.965431+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:02:31.965431+0000","last_undegraded":"2024-09-21T02:02:31.965431+0000","last_fullsized":"2024-09-21T02:02:31.965431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":49,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:53.325598+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:02:53.325598+0000","last_peered":"2024-09-21T02:02:53.325598+0000","last_clean":"2024-09-21T02:02:53.325598+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:02:53.325598+0000","last_undegraded":"2024-09-21T02:02:53.325598+0000","last_fullsized":"2024-09-21T02:02:53.325598+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982371+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:02:31.982371+0000","last_peered":"2024-09-21T02:02:31.982371+0000","last_clean":"2024-09-21T02:02:31.982371+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:02:31.982371+0000","last_undegraded":"2024-09-21T02:02:31.982371+0000","last_fullsized":"2024-09-21T02:02:31.982371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.310624+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:02:41.310624+0000","last_peered":"2024-09-21T02:02:41.310624+0000","last_clean":"2024-09-21T02:02:41.310624+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:02:41.310624+0000","last_undegraded":"2024-09-21T02:02:41.310624+0000","last_fullsized":"2024-09-21T02:02:41.310624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984789+0000","last_change":"2024-09-21T02:02:30.980769+0000","last_active":"2024-09-21T02:02:31.984789+0000","last_peered":"2024-09-21T02:02:31.984789+0000","last_clean":"2024-09-21T02:02:31.984789+0000","last_became_active":"2024-09-21T02:02:30.980432+0000","last_became_peered":"2024-09-21T02:02:30.980432+0000","last_unstale":"2024-09-21T02:02:31.984789+0000","last_undegraded":"2024-09-21T02:02:31.984789+0000","last_fullsized":"2024-09-21T02:02:31.984789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:33.038686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.047415+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:02:41.047415+0000","last_peered":"2024-09-21T02:02:41.047415+0000","last_clean":"2024-09-21T02:02:41.047415+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:02:41.047415+0000","last_undegraded":"2024-09-21T02:02:41.047415+0000","last_fullsized":"2024-09-21T02:02:41.047415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981929+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:02:31.981929+0000","last_peered":"2024-09-21T02:02:31.981929+0000","last_clean":"2024-09-21T02:02:31.981929+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:02:31.981929+0000","last_undegraded":"2024-09-21T02:02:31.981929+0000","last_fullsized":"2024-09-21T02:02:31.981929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:53.329821+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:02:53.329821+0000","last_peered":"2024-09-21T02:02:53.329821+0000","last_clean":"2024-09-21T02:02:53.329821+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:02:53.329821+0000","last_undegraded":"2024-09-21T02:02:53.329821+0000","last_fullsized":"2024-09-21T02:02:53.329821+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53741,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":68,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982068+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:02:31.982068+0000","last_peered":"2024-09-21T02:02:31.982068+0000","last_clean":"2024-09-21T02:02:31.982068+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:02:31.982068+0000","last_undegraded":"2024-09-21T02:02:31.982068+0000","last_fullsized":"2024-09-21T02:02:31.982068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.313291+0000","last_change":"2024-09-21T02:02:28.954979+0000","last_active":"2024-09-21T02:02:41.313291+0000","last_peered":"2024-09-21T02:02:41.313291+0000","last_clean":"2024-09-21T02:02:41.313291+0000","last_became_active":"2024-09-21T02:02:28.954536+0000","last_became_peered":"2024-09-21T02:02:28.954536+0000","last_unstale":"2024-09-21T02:02:41.313291+0000","last_undegraded":"2024-09-21T02:02:41.313291+0000","last_fullsized":"2024-09-21T02:02:41.313291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:50.203415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980044+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:02:31.980044+0000","last_peered":"2024-09-21T02:02:31.980044+0000","last_clean":"2024-09-21T02:02:31.980044+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:02:31.980044+0000","last_undegraded":"2024-09-21T02:02:31.980044+0000","last_fullsized":"2024-09-21T02:02:31.980044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.312379+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:02:41.312379+0000","last_peered":"2024-09-21T02:02:41.312379+0000","last_clean":"2024-09-21T02:02:41.312379+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:02:41.312379+0000","last_undegraded":"2024-09-21T02:02:41.312379+0000","last_fullsized":"2024-09-21T02:02:41.312379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979769+0000","last_change":"2024-09-21T02:02:30.974054+0000","last_active":"2024-09-21T02:02:31.979769+0000","last_peered":"2024-09-21T02:02:31.979769+0000","last_clean":"2024-09-21T02:02:31.979769+0000","last_became_active":"2024-09-21T02:02:30.973627+0000","last_became_peered":"2024-09-21T02:02:30.973627+0000","last_unstale":"2024-09-21T02:02:31.979769+0000","last_undegraded":"2024-09-21T02:02:31.979769+0000","last_fullsized":"2024-09-21T02:02:31.979769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:16:30.775839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:53.326021+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:02:53.326021+0000","last_peered":"2024-09-21T02:02:53.326021+0000","last_clean":"2024-09-21T02:02:53.326021+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:02:53.326021+0000","last_undegraded":"2024-09-21T02:02:53.326021+0000","last_fullsized":"2024-09-21T02:02:53.326021+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980274+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:02:31.980274+0000","last_peered":"2024-09-21T02:02:31.980274+0000","last_clean":"2024-09-21T02:02:31.980274+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:02:31.980274+0000","last_undegraded":"2024-09-21T02:02:31.980274+0000","last_fullsized":"2024-09-21T02:02:31.980274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.302614+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:02:41.302614+0000","last_peered":"2024-09-21T02:02:41.302614+0000","last_clean":"2024-09-21T02:02:41.302614+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:02:41.302614+0000","last_undegraded":"2024-09-21T02:02:41.302614+0000","last_fullsized":"2024-09-21T02:02:41.302614+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.983428+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:02:31.983428+0000","last_peered":"2024-09-21T02:02:31.983428+0000","last_clean":"2024-09-21T02:02:31.983428+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:02:31.983428+0000","last_undegraded":"2024-09-21T02:02:31.983428+0000","last_fullsized":"2024-09-21T02:02:31.983428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.031966+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:02:41.031966+0000","last_peered":"2024-09-21T02:02:41.031966+0000","last_clean":"2024-09-21T02:02:41.031966+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:02:41.031966+0000","last_undegraded":"2024-09-21T02:02:41.031966+0000","last_fullsized":"2024-09-21T02:02:41.031966+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966393+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:02:31.966393+0000","last_peered":"2024-09-21T02:02:31.966393+0000","last_clean":"2024-09-21T02:02:31.966393+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:02:31.966393+0000","last_undegraded":"2024-09-21T02:02:31.966393+0000","last_fullsized":"2024-09-21T02:02:31.966393+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.309476+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:02:41.309476+0000","last_peered":"2024-09-21T02:02:41.309476+0000","last_clean":"2024-09-21T02:02:41.309476+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:02:41.309476+0000","last_undegraded":"2024-09-21T02:02:41.309476+0000","last_fullsized":"2024-09-21T02:02:41.309476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980309+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:02:31.980309+0000","last_peered":"2024-09-21T02:02:31.980309+0000","last_clean":"2024-09-21T02:02:31.980309+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:02:31.980309+0000","last_undegraded":"2024-09-21T02:02:31.980309+0000","last_fullsized":"2024-09-21T02:02:31.980309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.302650+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:02:41.302650+0000","last_peered":"2024-09-21T02:02:41.302650+0000","last_clean":"2024-09-21T02:02:41.302650+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:02:41.302650+0000","last_undegraded":"2024-09-21T02:02:41.302650+0000","last_fullsized":"2024-09-21T02:02:41.302650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:33.144675+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:02:33.144675+0000","last_peered":"2024-09-21T02:02:33.144675+0000","last_clean":"2024-09-21T02:02:33.144675+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:02:33.144675+0000","last_undegraded":"2024-09-21T02:02:33.144675+0000","last_fullsized":"2024-09-21T02:02:33.144675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:52.324848+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:02:52.324848+0000","last_peered":"2024-09-21T02:02:52.324848+0000","last_clean":"2024-09-21T02:02:52.324848+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:02:52.324848+0000","last_undegraded":"2024-09-21T02:02:52.324848+0000","last_fullsized":"2024-09-21T02:02:52.324848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981541+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:02:31.981541+0000","last_peered":"2024-09-21T02:02:31.981541+0000","last_clean":"2024-09-21T02:02:31.981541+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:02:31.981541+0000","last_undegraded":"2024-09-21T02:02:31.981541+0000","last_fullsized":"2024-09-21T02:02:31.981541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980371+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:02:31.980371+0000","last_peered":"2024-09-21T02:02:31.980371+0000","last_clean":"2024-09-21T02:02:31.980371+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:02:31.980371+0000","last_undegraded":"2024-09-21T02:02:31.980371+0000","last_fullsized":"2024-09-21T02:02:31.980371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.314734+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:02:41.314734+0000","last_peered":"2024-09-21T02:02:41.314734+0000","last_clean":"2024-09-21T02:02:41.314734+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:02:41.314734+0000","last_undegraded":"2024-09-21T02:02:41.314734+0000","last_fullsized":"2024-09-21T02:02:41.314734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981241+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:02:31.981241+0000","last_peered":"2024-09-21T02:02:31.981241+0000","last_clean":"2024-09-21T02:02:31.981241+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:02:31.981241+0000","last_undegraded":"2024-09-21T02:02:31.981241+0000","last_fullsized":"2024-09-21T02:02:31.981241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:40.036553+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:02:40.036553+0000","last_peered":"2024-09-21T02:02:40.036553+0000","last_clean":"2024-09-21T02:02:40.036553+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:02:40.036553+0000","last_undegraded":"2024-09-21T02:02:40.036553+0000","last_fullsized":"2024-09-21T02:02:40.036553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979876+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:02:31.979876+0000","last_peered":"2024-09-21T02:02:31.979876+0000","last_clean":"2024-09-21T02:02:31.979876+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:02:31.979876+0000","last_undegraded":"2024-09-21T02:02:31.979876+0000","last_fullsized":"2024-09-21T02:02:31.979876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:52.324040+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:02:52.324040+0000","last_peered":"2024-09-21T02:02:52.324040+0000","last_clean":"2024-09-21T02:02:52.324040+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:02:52.324040+0000","last_undegraded":"2024-09-21T02:02:52.324040+0000","last_fullsized":"2024-09-21T02:02:52.324040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981257+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:02:31.981257+0000","last_peered":"2024-09-21T02:02:31.981257+0000","last_clean":"2024-09-21T02:02:31.981257+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:02:31.981257+0000","last_undegraded":"2024-09-21T02:02:31.981257+0000","last_fullsized":"2024-09-21T02:02:31.981257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:51.332948+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:02:51.332948+0000","last_peered":"2024-09-21T02:02:51.332948+0000","last_clean":"2024-09-21T02:02:51.332948+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:02:51.332948+0000","last_undegraded":"2024-09-21T02:02:51.332948+0000","last_fullsized":"2024-09-21T02:02:51.332948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.964072+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:02:31.964072+0000","last_peered":"2024-09-21T02:02:31.964072+0000","last_clean":"2024-09-21T02:02:31.964072+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:02:31.964072+0000","last_undegraded":"2024-09-21T02:02:31.964072+0000","last_fullsized":"2024-09-21T02:02:31.964072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:32.870201+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:02:32.870201+0000","last_peered":"2024-09-21T02:02:32.870201+0000","last_clean":"2024-09-21T02:02:32.870201+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:02:32.870201+0000","last_undegraded":"2024-09-21T02:02:32.870201+0000","last_fullsized":"2024-09-21T02:02:32.870201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:33.966656+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:02:33.966656+0000","last_peered":"2024-09-21T02:02:33.966656+0000","last_clean":"2024-09-21T02:02:33.966656+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:02:33.966656+0000","last_undegraded":"2024-09-21T02:02:33.966656+0000","last_fullsized":"2024-09-21T02:02:33.966656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.302231+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:02:41.302231+0000","last_peered":"2024-09-21T02:02:41.302231+0000","last_clean":"2024-09-21T02:02:41.302231+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:02:41.302231+0000","last_undegraded":"2024-09-21T02:02:41.302231+0000","last_fullsized":"2024-09-21T02:02:41.302231+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980520+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:02:31.980520+0000","last_peered":"2024-09-21T02:02:31.980520+0000","last_clean":"2024-09-21T02:02:31.980520+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:02:31.980520+0000","last_undegraded":"2024-09-21T02:02:31.980520+0000","last_fullsized":"2024-09-21T02:02:31.980520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'11","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:48.977917+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:02:48.977917+0000","last_peered":"2024-09-21T02:02:48.977917+0000","last_clean":"2024-09-21T02:02:48.977917+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:02:48.977917+0000","last_undegraded":"2024-09-21T02:02:48.977917+0000","last_fullsized":"2024-09-21T02:02:48.977917+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981383+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:02:31.981383+0000","last_peered":"2024-09-21T02:02:31.981383+0000","last_clean":"2024-09-21T02:02:31.981383+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:02:31.981383+0000","last_undegraded":"2024-09-21T02:02:31.981383+0000","last_fullsized":"2024-09-21T02:02:31.981383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.314241+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:02:41.314241+0000","last_peered":"2024-09-21T02:02:41.314241+0000","last_clean":"2024-09-21T02:02:41.314241+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:02:41.314241+0000","last_undegraded":"2024-09-21T02:02:41.314241+0000","last_fullsized":"2024-09-21T02:02:41.314241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984618+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:02:31.984618+0000","last_peered":"2024-09-21T02:02:31.984618+0000","last_clean":"2024-09-21T02:02:31.984618+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:02:31.984618+0000","last_undegraded":"2024-09-21T02:02:31.984618+0000","last_fullsized":"2024-09-21T02:02:31.984618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:39.029671+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:02:39.029671+0000","last_peered":"2024-09-21T02:02:39.029671+0000","last_clean":"2024-09-21T02:02:39.029671+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:02:39.029671+0000","last_undegraded":"2024-09-21T02:02:39.029671+0000","last_fullsized":"2024-09-21T02:02:39.029671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979600+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:02:31.979600+0000","last_peered":"2024-09-21T02:02:31.979600+0000","last_clean":"2024-09-21T02:02:31.979600+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:02:31.979600+0000","last_undegraded":"2024-09-21T02:02:31.979600+0000","last_fullsized":"2024-09-21T02:02:31.979600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.312067+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:02:41.312067+0000","last_peered":"2024-09-21T02:02:41.312067+0000","last_clean":"2024-09-21T02:02:41.312067+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:02:41.312067+0000","last_undegraded":"2024-09-21T02:02:41.312067+0000","last_fullsized":"2024-09-21T02:02:41.312067+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966481+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:02:31.966481+0000","last_peered":"2024-09-21T02:02:31.966481+0000","last_clean":"2024-09-21T02:02:31.966481+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:02:31.966481+0000","last_undegraded":"2024-09-21T02:02:31.966481+0000","last_fullsized":"2024-09-21T02:02:31.966481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.046937+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:02:41.046937+0000","last_peered":"2024-09-21T02:02:41.046937+0000","last_clean":"2024-09-21T02:02:41.046937+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:02:41.046937+0000","last_undegraded":"2024-09-21T02:02:41.046937+0000","last_fullsized":"2024-09-21T02:02:41.046937+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981040+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:02:31.981040+0000","last_peered":"2024-09-21T02:02:31.981040+0000","last_clean":"2024-09-21T02:02:31.981040+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:02:31.981040+0000","last_undegraded":"2024-09-21T02:02:31.981040+0000","last_fullsized":"2024-09-21T02:02:31.981040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:39.028870+0000","last_change":"2024-09-21T02:02:28.956340+0000","last_active":"2024-09-21T02:02:39.028870+0000","last_peered":"2024-09-21T02:02:39.028870+0000","last_clean":"2024-09-21T02:02:39.028870+0000","last_became_active":"2024-09-21T02:02:28.955784+0000","last_became_peered":"2024-09-21T02:02:28.955784+0000","last_unstale":"2024-09-21T02:02:39.028870+0000","last_undegraded":"2024-09-21T02:02:39.028870+0000","last_fullsized":"2024-09-21T02:02:39.028870+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:04.434320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980553+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:02:31.980553+0000","last_peered":"2024-09-21T02:02:31.980553+0000","last_clean":"2024-09-21T02:02:31.980553+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:02:31.980553+0000","last_undegraded":"2024-09-21T02:02:31.980553+0000","last_fullsized":"2024-09-21T02:02:31.980553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984557+0000","last_change":"2024-09-21T02:02:30.979134+0000","last_active":"2024-09-21T02:02:31.984557+0000","last_peered":"2024-09-21T02:02:31.984557+0000","last_clean":"2024-09-21T02:02:31.984557+0000","last_became_active":"2024-09-21T02:02:30.978759+0000","last_became_peered":"2024-09-21T02:02:30.978759+0000","last_unstale":"2024-09-21T02:02:31.984557+0000","last_undegraded":"2024-09-21T02:02:31.984557+0000","last_fullsized":"2024-09-21T02:02:31.984557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:17.416650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.313555+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:02:41.313555+0000","last_peered":"2024-09-21T02:02:41.313555+0000","last_clean":"2024-09-21T02:02:41.313555+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:02:41.313555+0000","last_undegraded":"2024-09-21T02:02:41.313555+0000","last_fullsized":"2024-09-21T02:02:41.313555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981665+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:02:31.981665+0000","last_peered":"2024-09-21T02:02:31.981665+0000","last_clean":"2024-09-21T02:02:31.981665+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:02:31.981665+0000","last_undegraded":"2024-09-21T02:02:31.981665+0000","last_fullsized":"2024-09-21T02:02:31.981665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.314617+0000","last_change":"2024-09-21T02:02:28.956646+0000","last_active":"2024-09-21T02:02:41.314617+0000","last_peered":"2024-09-21T02:02:41.314617+0000","last_clean":"2024-09-21T02:02:41.314617+0000","last_became_active":"2024-09-21T02:02:28.956248+0000","last_became_peered":"2024-09-21T02:02:28.956248+0000","last_unstale":"2024-09-21T02:02:41.314617+0000","last_undegraded":"2024-09-21T02:02:41.314617+0000","last_fullsized":"2024-09-21T02:02:41.314617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:03.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984768+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:02:31.984768+0000","last_peered":"2024-09-21T02:02:31.984768+0000","last_clean":"2024-09-21T02:02:31.984768+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:02:31.984768+0000","last_undegraded":"2024-09-21T02:02:31.984768+0000","last_fullsized":"2024-09-21T02:02:31.984768+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:32.894529+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:02:32.894529+0000","last_peered":"2024-09-21T02:02:32.894529+0000","last_clean":"2024-09-21T02:02:32.894529+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:02:32.894529+0000","last_undegraded":"2024-09-21T02:02:32.894529+0000","last_fullsized":"2024-09-21T02:02:32.894529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981286+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:02:31.981286+0000","last_peered":"2024-09-21T02:02:31.981286+0000","last_clean":"2024-09-21T02:02:31.981286+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:02:31.981286+0000","last_undegraded":"2024-09-21T02:02:31.981286+0000","last_fullsized":"2024-09-21T02:02:31.981286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:51.332267+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:02:51.332267+0000","last_peered":"2024-09-21T02:02:51.332267+0000","last_clean":"2024-09-21T02:02:51.332267+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:02:51.332267+0000","last_undegraded":"2024-09-21T02:02:51.332267+0000","last_fullsized":"2024-09-21T02:02:51.332267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61727,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":167,"num_read_kb":152,"num_write":234,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":185181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":389494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1511424,"data_stored":1034264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":389033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":839680,"data_stored":389411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":982711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":827392,"data_stored":389141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1449984,"data_stored":981607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":827392,"data_stored":389002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":388575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444093,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":444406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":443795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":390480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":55884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":56495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:02:57.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:02:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:57.632 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:57 smithi092 ceph-mon[31190]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 -- 172.21.15.92:0/3333301847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc1094c0 msgr2=0x7faddc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 --2- 172.21.15.92:0/3333301847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc1094c0 0x7faddc1098d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009900 tx=0x7fadcc02a410 comp rx=0 tx=0).stop 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 -- 172.21.15.92:0/3333301847 shutdown_connections 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 --2- 172.21.15.92:0/3333301847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc109e10 0x7faddc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 --2- 172.21.15.92:0/3333301847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc1094c0 0x7faddc1098d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 --2- 172.21.15.92:0/3333301847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faddc101fe0 0x7faddc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 -- 172.21.15.92:0/3333301847 >> 172.21.15.92:0/3333301847 conn(0x7faddc077bc0 msgr2=0x7faddc078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:57.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 -- 172.21.15.92:0/3333301847 shutdown_connections 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.720+0000 7fade14c7640 1 -- 172.21.15.92:0/3333301847 wait complete. 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 Processor -- start 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 -- start start 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faddc101fe0 0x7faddc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc1094c0 0x7faddc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 0x7faddc19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113ac0 con 0x7faddc109e10 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113940 con 0x7faddc1094c0 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7fade14c7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113c40 con 0x7faddc101fe0 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 0x7faddc19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:57.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 0x7faddc19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36308/0 (socket says 172.21.15.92:36308) 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.721+0000 7faddb7fe640 1 -- 172.21.15.92:0/3836473303 learned_addr learned my addr 172.21.15.92:0/3836473303 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddaffd640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faddc101fe0 0x7faddc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7fadda7fc640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc1094c0 0x7faddc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddb7fe640 1 -- 172.21.15.92:0/3836473303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faddc101fe0 msgr2=0x7faddc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddb7fe640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faddc101fe0 0x7faddc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddb7fe640 1 -- 172.21.15.92:0/3836473303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc1094c0 msgr2=0x7faddc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddb7fe640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc1094c0 0x7faddc19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddb7fe640 1 -- 172.21.15.92:0/3836473303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a0500 con 0x7faddc109e10 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7faddb7fe640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 0x7faddc19fdd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fadd400b970 tx=0x7fadd400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4019070 con 0x7faddc109e10 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a07f0 con 0x7faddc109e10 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4014440 con 0x7faddc109e10 2024-09-21T02:02:57.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40075d0 con 0x7faddc109e10 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.722+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a6f50 con 0x7faddc109e10 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.723+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc1023f0 con 0x7faddc109e10 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.725+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fadd401c070 con 0x7faddc109e10 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.725+0000 7fadc3fff640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fadc40b65a0 0x7fadc40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.725+0000 7faddaffd640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fadc40b65a0 0x7fadc40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.726+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fadd40e6cc0 con 0x7faddc109e10 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.726+0000 7faddaffd640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fadc40b65a0 0x7fadc40b8ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadd00058c0 tx=0x7fadd000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:57.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.729+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fadd40ad5a0 con 0x7faddc109e10 2024-09-21T02:02:57.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:57 smithi136 ceph-mon[34754]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:02:57.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:57 smithi099 ceph-mon[34404]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:02:57.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.904+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faddc063800 con 0x7fadc40b65a0 2024-09-21T02:02:57.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.908+0000 7fadc3fff640 1 -- 172.21.15.92:0/3836473303 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258741 (secure 0 0 0) 0x7faddc063800 con 0x7fadc40b65a0 2024-09-21T02:02:57.911 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:02:57.911 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fadc40b65a0 msgr2=0x7fadc40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fadc40b65a0 0x7fadc40b8ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadd00058c0 tx=0x7fadd000a4c0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 msgr2=0x7faddc19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 0x7faddc19fdd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fadd400b970 tx=0x7fadd400be30 comp rx=0 tx=0).stop 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 shutdown_connections 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fadc40b65a0 0x7fadc40b8ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faddc109e10 0x7faddc19fdd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc1094c0 0x7faddc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 --2- 172.21.15.92:0/3836473303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faddc101fe0 0x7faddc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 >> 172.21.15.92:0/3836473303 conn(0x7faddc077bc0 msgr2=0x7faddc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 shutdown_connections 2024-09-21T02:02:57.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:57.911+0000 7fade14c7640 1 -- 172.21.15.92:0/3836473303 wait complete. 2024-09-21T02:02:58.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:58 smithi092 ceph-mon[31190]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-21T02:02:58.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:02:58 smithi092 ceph-mon[31190]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:02:58.717 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-21T02:02:57.027907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652329,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":243,"num_read_kb":216,"num_write":359,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":406756,"kb_used_data":11748,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124535068,"statfs":{"total":1151940427776,"available":1151523909632,"internally_reserved":0,"allocated":12029952,"data_stored":6611919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53927,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003450"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966600+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:02:31.966600+0000","last_peered":"2024-09-21T02:02:31.966600+0000","last_clean":"2024-09-21T02:02:31.966600+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:02:31.966600+0000","last_undegraded":"2024-09-21T02:02:31.966600+0000","last_fullsized":"2024-09-21T02:02:31.966600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984212+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:02:31.984212+0000","last_peered":"2024-09-21T02:02:31.984212+0000","last_clean":"2024-09-21T02:02:31.984212+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:02:31.984212+0000","last_undegraded":"2024-09-21T02:02:31.984212+0000","last_fullsized":"2024-09-21T02:02:31.984212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984212+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:02:31.984212+0000","last_peered":"2024-09-21T02:02:31.984212+0000","last_clean":"2024-09-21T02:02:31.984212+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:02:31.984212+0000","last_undegraded":"2024-09-21T02:02:31.984212+0000","last_fullsized":"2024-09-21T02:02:31.984212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984122+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:02:31.984122+0000","last_peered":"2024-09-21T02:02:31.984122+0000","last_clean":"2024-09-21T02:02:31.984122+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:02:31.984122+0000","last_undegraded":"2024-09-21T02:02:31.984122+0000","last_fullsized":"2024-09-21T02:02:31.984122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979942+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:02:31.979942+0000","last_peered":"2024-09-21T02:02:31.979942+0000","last_clean":"2024-09-21T02:02:31.979942+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:02:31.979942+0000","last_undegraded":"2024-09-21T02:02:31.979942+0000","last_fullsized":"2024-09-21T02:02:31.979942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.964059+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:02:31.964059+0000","last_peered":"2024-09-21T02:02:31.964059+0000","last_clean":"2024-09-21T02:02:31.964059+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:02:31.964059+0000","last_undegraded":"2024-09-21T02:02:31.964059+0000","last_fullsized":"2024-09-21T02:02:31.964059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979814+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:02:31.979814+0000","last_peered":"2024-09-21T02:02:31.979814+0000","last_clean":"2024-09-21T02:02:31.979814+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:02:31.979814+0000","last_undegraded":"2024-09-21T02:02:31.979814+0000","last_fullsized":"2024-09-21T02:02:31.979814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980365+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:02:31.980365+0000","last_peered":"2024-09-21T02:02:31.980365+0000","last_clean":"2024-09-21T02:02:31.980365+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:02:31.980365+0000","last_undegraded":"2024-09-21T02:02:31.980365+0000","last_fullsized":"2024-09-21T02:02:31.980365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982507+0000","last_change":"2024-09-21T02:02:30.975000+0000","last_active":"2024-09-21T02:02:31.982507+0000","last_peered":"2024-09-21T02:02:31.982507+0000","last_clean":"2024-09-21T02:02:31.982507+0000","last_became_active":"2024-09-21T02:02:30.974507+0000","last_became_peered":"2024-09-21T02:02:30.974507+0000","last_unstale":"2024-09-21T02:02:31.982507+0000","last_undegraded":"2024-09-21T02:02:31.982507+0000","last_fullsized":"2024-09-21T02:02:31.982507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:19.152162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.964552+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:02:31.964552+0000","last_peered":"2024-09-21T02:02:31.964552+0000","last_clean":"2024-09-21T02:02:31.964552+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:02:31.964552+0000","last_undegraded":"2024-09-21T02:02:31.964552+0000","last_fullsized":"2024-09-21T02:02:31.964552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981302+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:02:31.981302+0000","last_peered":"2024-09-21T02:02:31.981302+0000","last_clean":"2024-09-21T02:02:31.981302+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:02:31.981302+0000","last_undegraded":"2024-09-21T02:02:31.981302+0000","last_fullsized":"2024-09-21T02:02:31.981302+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984392+0000","last_change":"2024-09-21T02:02:30.975791+0000","last_active":"2024-09-21T02:02:31.984392+0000","last_peered":"2024-09-21T02:02:31.984392+0000","last_clean":"2024-09-21T02:02:31.984392+0000","last_became_active":"2024-09-21T02:02:30.974622+0000","last_became_peered":"2024-09-21T02:02:30.974622+0000","last_unstale":"2024-09-21T02:02:31.984392+0000","last_undegraded":"2024-09-21T02:02:31.984392+0000","last_fullsized":"2024-09-21T02:02:31.984392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:59.033132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980429+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:02:31.980429+0000","last_peered":"2024-09-21T02:02:31.980429+0000","last_clean":"2024-09-21T02:02:31.980429+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:02:31.980429+0000","last_undegraded":"2024-09-21T02:02:31.980429+0000","last_fullsized":"2024-09-21T02:02:31.980429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984127+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:02:31.984127+0000","last_peered":"2024-09-21T02:02:31.984127+0000","last_clean":"2024-09-21T02:02:31.984127+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:02:31.984127+0000","last_undegraded":"2024-09-21T02:02:31.984127+0000","last_fullsized":"2024-09-21T02:02:31.984127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.983847+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:02:31.983847+0000","last_peered":"2024-09-21T02:02:31.983847+0000","last_clean":"2024-09-21T02:02:31.983847+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:02:31.983847+0000","last_undegraded":"2024-09-21T02:02:31.983847+0000","last_fullsized":"2024-09-21T02:02:31.983847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981419+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:02:31.981419+0000","last_peered":"2024-09-21T02:02:31.981419+0000","last_clean":"2024-09-21T02:02:31.981419+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:02:31.981419+0000","last_undegraded":"2024-09-21T02:02:31.981419+0000","last_fullsized":"2024-09-21T02:02:31.981419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966769+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:02:31.966769+0000","last_peered":"2024-09-21T02:02:31.966769+0000","last_clean":"2024-09-21T02:02:31.966769+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:02:31.966769+0000","last_undegraded":"2024-09-21T02:02:31.966769+0000","last_fullsized":"2024-09-21T02:02:31.966769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.965578+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:02:31.965578+0000","last_peered":"2024-09-21T02:02:31.965578+0000","last_clean":"2024-09-21T02:02:31.965578+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:02:31.965578+0000","last_undegraded":"2024-09-21T02:02:31.965578+0000","last_fullsized":"2024-09-21T02:02:31.965578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.986741+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:02:31.986741+0000","last_peered":"2024-09-21T02:02:31.986741+0000","last_clean":"2024-09-21T02:02:31.986741+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:02:31.986741+0000","last_undegraded":"2024-09-21T02:02:31.986741+0000","last_fullsized":"2024-09-21T02:02:31.986741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966245+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:02:31.966245+0000","last_peered":"2024-09-21T02:02:31.966245+0000","last_clean":"2024-09-21T02:02:31.966245+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:02:31.966245+0000","last_undegraded":"2024-09-21T02:02:31.966245+0000","last_fullsized":"2024-09-21T02:02:31.966245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980643+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:02:31.980643+0000","last_peered":"2024-09-21T02:02:31.980643+0000","last_clean":"2024-09-21T02:02:31.980643+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:02:31.980643+0000","last_undegraded":"2024-09-21T02:02:31.980643+0000","last_fullsized":"2024-09-21T02:02:31.980643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982484+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:02:31.982484+0000","last_peered":"2024-09-21T02:02:31.982484+0000","last_clean":"2024-09-21T02:02:31.982484+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:02:31.982484+0000","last_undegraded":"2024-09-21T02:02:31.982484+0000","last_fullsized":"2024-09-21T02:02:31.982484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980277+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:02:31.980277+0000","last_peered":"2024-09-21T02:02:31.980277+0000","last_clean":"2024-09-21T02:02:31.980277+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:02:31.980277+0000","last_undegraded":"2024-09-21T02:02:31.980277+0000","last_fullsized":"2024-09-21T02:02:31.980277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981000+0000","last_change":"2024-09-21T02:02:30.973693+0000","last_active":"2024-09-21T02:02:31.981000+0000","last_peered":"2024-09-21T02:02:31.981000+0000","last_clean":"2024-09-21T02:02:31.981000+0000","last_became_active":"2024-09-21T02:02:30.973350+0000","last_became_peered":"2024-09-21T02:02:30.973350+0000","last_unstale":"2024-09-21T02:02:31.981000+0000","last_undegraded":"2024-09-21T02:02:31.981000+0000","last_fullsized":"2024-09-21T02:02:31.981000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:54.848940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981004+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:02:31.981004+0000","last_peered":"2024-09-21T02:02:31.981004+0000","last_clean":"2024-09-21T02:02:31.981004+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:02:31.981004+0000","last_undegraded":"2024-09-21T02:02:31.981004+0000","last_fullsized":"2024-09-21T02:02:31.981004+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.985951+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:02:31.985951+0000","last_peered":"2024-09-21T02:02:31.985951+0000","last_clean":"2024-09-21T02:02:31.985951+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:02:31.985951+0000","last_undegraded":"2024-09-21T02:02:31.985951+0000","last_fullsized":"2024-09-21T02:02:31.985951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966470+0000","last_change":"2024-09-21T02:02:30.976395+0000","last_active":"2024-09-21T02:02:31.966470+0000","last_peered":"2024-09-21T02:02:31.966470+0000","last_clean":"2024-09-21T02:02:31.966470+0000","last_became_active":"2024-09-21T02:02:30.976001+0000","last_became_peered":"2024-09-21T02:02:30.976001+0000","last_unstale":"2024-09-21T02:02:31.966470+0000","last_undegraded":"2024-09-21T02:02:31.966470+0000","last_fullsized":"2024-09-21T02:02:31.966470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:39.724131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980134+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:02:31.980134+0000","last_peered":"2024-09-21T02:02:31.980134+0000","last_clean":"2024-09-21T02:02:31.980134+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:02:31.980134+0000","last_undegraded":"2024-09-21T02:02:31.980134+0000","last_fullsized":"2024-09-21T02:02:31.980134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984239+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:02:31.984239+0000","last_peered":"2024-09-21T02:02:31.984239+0000","last_clean":"2024-09-21T02:02:31.984239+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:02:31.984239+0000","last_undegraded":"2024-09-21T02:02:31.984239+0000","last_fullsized":"2024-09-21T02:02:31.984239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981470+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:02:31.981470+0000","last_peered":"2024-09-21T02:02:31.981470+0000","last_clean":"2024-09-21T02:02:31.981470+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:02:31.981470+0000","last_undegraded":"2024-09-21T02:02:31.981470+0000","last_fullsized":"2024-09-21T02:02:31.981470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984906+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:02:31.984906+0000","last_peered":"2024-09-21T02:02:31.984906+0000","last_clean":"2024-09-21T02:02:31.984906+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:02:31.984906+0000","last_undegraded":"2024-09-21T02:02:31.984906+0000","last_fullsized":"2024-09-21T02:02:31.984906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966378+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:02:31.966378+0000","last_peered":"2024-09-21T02:02:31.966378+0000","last_clean":"2024-09-21T02:02:31.966378+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:02:31.966378+0000","last_undegraded":"2024-09-21T02:02:31.966378+0000","last_fullsized":"2024-09-21T02:02:31.966378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979570+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:02:31.979570+0000","last_peered":"2024-09-21T02:02:31.979570+0000","last_clean":"2024-09-21T02:02:31.979570+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:02:31.979570+0000","last_undegraded":"2024-09-21T02:02:31.979570+0000","last_fullsized":"2024-09-21T02:02:31.979570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:51.323669+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:02:51.323669+0000","last_peered":"2024-09-21T02:02:51.323669+0000","last_clean":"2024-09-21T02:02:51.323669+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:02:51.323669+0000","last_undegraded":"2024-09-21T02:02:51.323669+0000","last_fullsized":"2024-09-21T02:02:51.323669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980764+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:02:31.980764+0000","last_peered":"2024-09-21T02:02:31.980764+0000","last_clean":"2024-09-21T02:02:31.980764+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:02:31.980764+0000","last_undegraded":"2024-09-21T02:02:31.980764+0000","last_fullsized":"2024-09-21T02:02:31.980764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.046806+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:02:41.046806+0000","last_peered":"2024-09-21T02:02:41.046806+0000","last_clean":"2024-09-21T02:02:41.046806+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:02:41.046806+0000","last_undegraded":"2024-09-21T02:02:41.046806+0000","last_fullsized":"2024-09-21T02:02:41.046806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:40.035494+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:02:40.035494+0000","last_peered":"2024-09-21T02:02:40.035494+0000","last_clean":"2024-09-21T02:02:40.035494+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:02:40.035494+0000","last_undegraded":"2024-09-21T02:02:40.035494+0000","last_fullsized":"2024-09-21T02:02:40.035494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980613+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:02:31.980613+0000","last_peered":"2024-09-21T02:02:31.980613+0000","last_clean":"2024-09-21T02:02:31.980613+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:02:31.980613+0000","last_undegraded":"2024-09-21T02:02:31.980613+0000","last_fullsized":"2024-09-21T02:02:31.980613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.306642+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:02:41.306642+0000","last_peered":"2024-09-21T02:02:41.306642+0000","last_clean":"2024-09-21T02:02:41.306642+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:02:41.306642+0000","last_undegraded":"2024-09-21T02:02:41.306642+0000","last_fullsized":"2024-09-21T02:02:41.306642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.965431+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:02:31.965431+0000","last_peered":"2024-09-21T02:02:31.965431+0000","last_clean":"2024-09-21T02:02:31.965431+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:02:31.965431+0000","last_undegraded":"2024-09-21T02:02:31.965431+0000","last_fullsized":"2024-09-21T02:02:31.965431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":49,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:53.325598+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:02:53.325598+0000","last_peered":"2024-09-21T02:02:53.325598+0000","last_clean":"2024-09-21T02:02:53.325598+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:02:53.325598+0000","last_undegraded":"2024-09-21T02:02:53.325598+0000","last_fullsized":"2024-09-21T02:02:53.325598+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982371+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:02:31.982371+0000","last_peered":"2024-09-21T02:02:31.982371+0000","last_clean":"2024-09-21T02:02:31.982371+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:02:31.982371+0000","last_undegraded":"2024-09-21T02:02:31.982371+0000","last_fullsized":"2024-09-21T02:02:31.982371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.310624+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:02:41.310624+0000","last_peered":"2024-09-21T02:02:41.310624+0000","last_clean":"2024-09-21T02:02:41.310624+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:02:41.310624+0000","last_undegraded":"2024-09-21T02:02:41.310624+0000","last_fullsized":"2024-09-21T02:02:41.310624+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984789+0000","last_change":"2024-09-21T02:02:30.980769+0000","last_active":"2024-09-21T02:02:31.984789+0000","last_peered":"2024-09-21T02:02:31.984789+0000","last_clean":"2024-09-21T02:02:31.984789+0000","last_became_active":"2024-09-21T02:02:30.980432+0000","last_became_peered":"2024-09-21T02:02:30.980432+0000","last_unstale":"2024-09-21T02:02:31.984789+0000","last_undegraded":"2024-09-21T02:02:31.984789+0000","last_fullsized":"2024-09-21T02:02:31.984789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:33.038686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.047415+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:02:41.047415+0000","last_peered":"2024-09-21T02:02:41.047415+0000","last_clean":"2024-09-21T02:02:41.047415+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:02:41.047415+0000","last_undegraded":"2024-09-21T02:02:41.047415+0000","last_fullsized":"2024-09-21T02:02:41.047415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981929+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:02:31.981929+0000","last_peered":"2024-09-21T02:02:31.981929+0000","last_clean":"2024-09-21T02:02:31.981929+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:02:31.981929+0000","last_undegraded":"2024-09-21T02:02:31.981929+0000","last_fullsized":"2024-09-21T02:02:31.981929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:53.329821+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:02:53.329821+0000","last_peered":"2024-09-21T02:02:53.329821+0000","last_clean":"2024-09-21T02:02:53.329821+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:02:53.329821+0000","last_undegraded":"2024-09-21T02:02:53.329821+0000","last_fullsized":"2024-09-21T02:02:53.329821+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53741,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":68,"num_write":29,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.982068+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:02:31.982068+0000","last_peered":"2024-09-21T02:02:31.982068+0000","last_clean":"2024-09-21T02:02:31.982068+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:02:31.982068+0000","last_undegraded":"2024-09-21T02:02:31.982068+0000","last_fullsized":"2024-09-21T02:02:31.982068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.313291+0000","last_change":"2024-09-21T02:02:28.954979+0000","last_active":"2024-09-21T02:02:41.313291+0000","last_peered":"2024-09-21T02:02:41.313291+0000","last_clean":"2024-09-21T02:02:41.313291+0000","last_became_active":"2024-09-21T02:02:28.954536+0000","last_became_peered":"2024-09-21T02:02:28.954536+0000","last_unstale":"2024-09-21T02:02:41.313291+0000","last_undegraded":"2024-09-21T02:02:41.313291+0000","last_fullsized":"2024-09-21T02:02:41.313291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:50.203415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980044+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:02:31.980044+0000","last_peered":"2024-09-21T02:02:31.980044+0000","last_clean":"2024-09-21T02:02:31.980044+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:02:31.980044+0000","last_undegraded":"2024-09-21T02:02:31.980044+0000","last_fullsized":"2024-09-21T02:02:31.980044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.312379+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:02:41.312379+0000","last_peered":"2024-09-21T02:02:41.312379+0000","last_clean":"2024-09-21T02:02:41.312379+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:02:41.312379+0000","last_undegraded":"2024-09-21T02:02:41.312379+0000","last_fullsized":"2024-09-21T02:02:41.312379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979769+0000","last_change":"2024-09-21T02:02:30.974054+0000","last_active":"2024-09-21T02:02:31.979769+0000","last_peered":"2024-09-21T02:02:31.979769+0000","last_clean":"2024-09-21T02:02:31.979769+0000","last_became_active":"2024-09-21T02:02:30.973627+0000","last_became_peered":"2024-09-21T02:02:30.973627+0000","last_unstale":"2024-09-21T02:02:31.979769+0000","last_undegraded":"2024-09-21T02:02:31.979769+0000","last_fullsized":"2024-09-21T02:02:31.979769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:16:30.775839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:53.326021+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:02:53.326021+0000","last_peered":"2024-09-21T02:02:53.326021+0000","last_clean":"2024-09-21T02:02:53.326021+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:02:53.326021+0000","last_undegraded":"2024-09-21T02:02:53.326021+0000","last_fullsized":"2024-09-21T02:02:53.326021+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980274+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:02:31.980274+0000","last_peered":"2024-09-21T02:02:31.980274+0000","last_clean":"2024-09-21T02:02:31.980274+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:02:31.980274+0000","last_undegraded":"2024-09-21T02:02:31.980274+0000","last_fullsized":"2024-09-21T02:02:31.980274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.302614+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:02:41.302614+0000","last_peered":"2024-09-21T02:02:41.302614+0000","last_clean":"2024-09-21T02:02:41.302614+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:02:41.302614+0000","last_undegraded":"2024-09-21T02:02:41.302614+0000","last_fullsized":"2024-09-21T02:02:41.302614+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.983428+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:02:31.983428+0000","last_peered":"2024-09-21T02:02:31.983428+0000","last_clean":"2024-09-21T02:02:31.983428+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:02:31.983428+0000","last_undegraded":"2024-09-21T02:02:31.983428+0000","last_fullsized":"2024-09-21T02:02:31.983428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.031966+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:02:41.031966+0000","last_peered":"2024-09-21T02:02:41.031966+0000","last_clean":"2024-09-21T02:02:41.031966+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:02:41.031966+0000","last_undegraded":"2024-09-21T02:02:41.031966+0000","last_fullsized":"2024-09-21T02:02:41.031966+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966393+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:02:31.966393+0000","last_peered":"2024-09-21T02:02:31.966393+0000","last_clean":"2024-09-21T02:02:31.966393+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:02:31.966393+0000","last_undegraded":"2024-09-21T02:02:31.966393+0000","last_fullsized":"2024-09-21T02:02:31.966393+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.309476+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:02:41.309476+0000","last_peered":"2024-09-21T02:02:41.309476+0000","last_clean":"2024-09-21T02:02:41.309476+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:02:41.309476+0000","last_undegraded":"2024-09-21T02:02:41.309476+0000","last_fullsized":"2024-09-21T02:02:41.309476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980309+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:02:31.980309+0000","last_peered":"2024-09-21T02:02:31.980309+0000","last_clean":"2024-09-21T02:02:31.980309+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:02:31.980309+0000","last_undegraded":"2024-09-21T02:02:31.980309+0000","last_fullsized":"2024-09-21T02:02:31.980309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.302650+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:02:41.302650+0000","last_peered":"2024-09-21T02:02:41.302650+0000","last_clean":"2024-09-21T02:02:41.302650+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:02:41.302650+0000","last_undegraded":"2024-09-21T02:02:41.302650+0000","last_fullsized":"2024-09-21T02:02:41.302650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:33.144675+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:02:33.144675+0000","last_peered":"2024-09-21T02:02:33.144675+0000","last_clean":"2024-09-21T02:02:33.144675+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:02:33.144675+0000","last_undegraded":"2024-09-21T02:02:33.144675+0000","last_fullsized":"2024-09-21T02:02:33.144675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:52.324848+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:02:52.324848+0000","last_peered":"2024-09-21T02:02:52.324848+0000","last_clean":"2024-09-21T02:02:52.324848+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:02:52.324848+0000","last_undegraded":"2024-09-21T02:02:52.324848+0000","last_fullsized":"2024-09-21T02:02:52.324848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981541+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:02:31.981541+0000","last_peered":"2024-09-21T02:02:31.981541+0000","last_clean":"2024-09-21T02:02:31.981541+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:02:31.981541+0000","last_undegraded":"2024-09-21T02:02:31.981541+0000","last_fullsized":"2024-09-21T02:02:31.981541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980371+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:02:31.980371+0000","last_peered":"2024-09-21T02:02:31.980371+0000","last_clean":"2024-09-21T02:02:31.980371+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:02:31.980371+0000","last_undegraded":"2024-09-21T02:02:31.980371+0000","last_fullsized":"2024-09-21T02:02:31.980371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.314734+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:02:41.314734+0000","last_peered":"2024-09-21T02:02:41.314734+0000","last_clean":"2024-09-21T02:02:41.314734+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:02:41.314734+0000","last_undegraded":"2024-09-21T02:02:41.314734+0000","last_fullsized":"2024-09-21T02:02:41.314734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981241+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:02:31.981241+0000","last_peered":"2024-09-21T02:02:31.981241+0000","last_clean":"2024-09-21T02:02:31.981241+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:02:31.981241+0000","last_undegraded":"2024-09-21T02:02:31.981241+0000","last_fullsized":"2024-09-21T02:02:31.981241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:40.036553+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:02:40.036553+0000","last_peered":"2024-09-21T02:02:40.036553+0000","last_clean":"2024-09-21T02:02:40.036553+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:02:40.036553+0000","last_undegraded":"2024-09-21T02:02:40.036553+0000","last_fullsized":"2024-09-21T02:02:40.036553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979876+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:02:31.979876+0000","last_peered":"2024-09-21T02:02:31.979876+0000","last_clean":"2024-09-21T02:02:31.979876+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:02:31.979876+0000","last_undegraded":"2024-09-21T02:02:31.979876+0000","last_fullsized":"2024-09-21T02:02:31.979876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:52.324040+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:02:52.324040+0000","last_peered":"2024-09-21T02:02:52.324040+0000","last_clean":"2024-09-21T02:02:52.324040+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:02:52.324040+0000","last_undegraded":"2024-09-21T02:02:52.324040+0000","last_fullsized":"2024-09-21T02:02:52.324040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981257+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:02:31.981257+0000","last_peered":"2024-09-21T02:02:31.981257+0000","last_clean":"2024-09-21T02:02:31.981257+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:02:31.981257+0000","last_undegraded":"2024-09-21T02:02:31.981257+0000","last_fullsized":"2024-09-21T02:02:31.981257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:51.332948+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:02:51.332948+0000","last_peered":"2024-09-21T02:02:51.332948+0000","last_clean":"2024-09-21T02:02:51.332948+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:02:51.332948+0000","last_undegraded":"2024-09-21T02:02:51.332948+0000","last_fullsized":"2024-09-21T02:02:51.332948+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.964072+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:02:31.964072+0000","last_peered":"2024-09-21T02:02:31.964072+0000","last_clean":"2024-09-21T02:02:31.964072+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:02:31.964072+0000","last_undegraded":"2024-09-21T02:02:31.964072+0000","last_fullsized":"2024-09-21T02:02:31.964072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:32.870201+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:02:32.870201+0000","last_peered":"2024-09-21T02:02:32.870201+0000","last_clean":"2024-09-21T02:02:32.870201+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:02:32.870201+0000","last_undegraded":"2024-09-21T02:02:32.870201+0000","last_fullsized":"2024-09-21T02:02:32.870201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:33.966656+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:02:33.966656+0000","last_peered":"2024-09-21T02:02:33.966656+0000","last_clean":"2024-09-21T02:02:33.966656+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:02:33.966656+0000","last_undegraded":"2024-09-21T02:02:33.966656+0000","last_fullsized":"2024-09-21T02:02:33.966656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.302231+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:02:41.302231+0000","last_peered":"2024-09-21T02:02:41.302231+0000","last_clean":"2024-09-21T02:02:41.302231+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:02:41.302231+0000","last_undegraded":"2024-09-21T02:02:41.302231+0000","last_fullsized":"2024-09-21T02:02:41.302231+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980520+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:02:31.980520+0000","last_peered":"2024-09-21T02:02:31.980520+0000","last_clean":"2024-09-21T02:02:31.980520+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:02:31.980520+0000","last_undegraded":"2024-09-21T02:02:31.980520+0000","last_fullsized":"2024-09-21T02:02:31.980520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'11","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:48.977917+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:02:48.977917+0000","last_peered":"2024-09-21T02:02:48.977917+0000","last_clean":"2024-09-21T02:02:48.977917+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:02:48.977917+0000","last_undegraded":"2024-09-21T02:02:48.977917+0000","last_fullsized":"2024-09-21T02:02:48.977917+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981383+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:02:31.981383+0000","last_peered":"2024-09-21T02:02:31.981383+0000","last_clean":"2024-09-21T02:02:31.981383+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:02:31.981383+0000","last_undegraded":"2024-09-21T02:02:31.981383+0000","last_fullsized":"2024-09-21T02:02:31.981383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.314241+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:02:41.314241+0000","last_peered":"2024-09-21T02:02:41.314241+0000","last_clean":"2024-09-21T02:02:41.314241+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:02:41.314241+0000","last_undegraded":"2024-09-21T02:02:41.314241+0000","last_fullsized":"2024-09-21T02:02:41.314241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984618+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:02:31.984618+0000","last_peered":"2024-09-21T02:02:31.984618+0000","last_clean":"2024-09-21T02:02:31.984618+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:02:31.984618+0000","last_undegraded":"2024-09-21T02:02:31.984618+0000","last_fullsized":"2024-09-21T02:02:31.984618+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:39.029671+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:02:39.029671+0000","last_peered":"2024-09-21T02:02:39.029671+0000","last_clean":"2024-09-21T02:02:39.029671+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:02:39.029671+0000","last_undegraded":"2024-09-21T02:02:39.029671+0000","last_fullsized":"2024-09-21T02:02:39.029671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.979600+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:02:31.979600+0000","last_peered":"2024-09-21T02:02:31.979600+0000","last_clean":"2024-09-21T02:02:31.979600+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:02:31.979600+0000","last_undegraded":"2024-09-21T02:02:31.979600+0000","last_fullsized":"2024-09-21T02:02:31.979600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.312067+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:02:41.312067+0000","last_peered":"2024-09-21T02:02:41.312067+0000","last_clean":"2024-09-21T02:02:41.312067+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:02:41.312067+0000","last_undegraded":"2024-09-21T02:02:41.312067+0000","last_fullsized":"2024-09-21T02:02:41.312067+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.966481+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:02:31.966481+0000","last_peered":"2024-09-21T02:02:31.966481+0000","last_clean":"2024-09-21T02:02:31.966481+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:02:31.966481+0000","last_undegraded":"2024-09-21T02:02:31.966481+0000","last_fullsized":"2024-09-21T02:02:31.966481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.046937+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:02:41.046937+0000","last_peered":"2024-09-21T02:02:41.046937+0000","last_clean":"2024-09-21T02:02:41.046937+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:02:41.046937+0000","last_undegraded":"2024-09-21T02:02:41.046937+0000","last_fullsized":"2024-09-21T02:02:41.046937+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981040+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:02:31.981040+0000","last_peered":"2024-09-21T02:02:31.981040+0000","last_clean":"2024-09-21T02:02:31.981040+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:02:31.981040+0000","last_undegraded":"2024-09-21T02:02:31.981040+0000","last_fullsized":"2024-09-21T02:02:31.981040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:39.028870+0000","last_change":"2024-09-21T02:02:28.956340+0000","last_active":"2024-09-21T02:02:39.028870+0000","last_peered":"2024-09-21T02:02:39.028870+0000","last_clean":"2024-09-21T02:02:39.028870+0000","last_became_active":"2024-09-21T02:02:28.955784+0000","last_became_peered":"2024-09-21T02:02:28.955784+0000","last_unstale":"2024-09-21T02:02:39.028870+0000","last_undegraded":"2024-09-21T02:02:39.028870+0000","last_fullsized":"2024-09-21T02:02:39.028870+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:04.434320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.980553+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:02:31.980553+0000","last_peered":"2024-09-21T02:02:31.980553+0000","last_clean":"2024-09-21T02:02:31.980553+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:02:31.980553+0000","last_undegraded":"2024-09-21T02:02:31.980553+0000","last_fullsized":"2024-09-21T02:02:31.980553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984557+0000","last_change":"2024-09-21T02:02:30.979134+0000","last_active":"2024-09-21T02:02:31.984557+0000","last_peered":"2024-09-21T02:02:31.984557+0000","last_clean":"2024-09-21T02:02:31.984557+0000","last_became_active":"2024-09-21T02:02:30.978759+0000","last_became_peered":"2024-09-21T02:02:30.978759+0000","last_unstale":"2024-09-21T02:02:31.984557+0000","last_undegraded":"2024-09-21T02:02:31.984557+0000","last_fullsized":"2024-09-21T02:02:31.984557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:17.416650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.313555+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:02:41.313555+0000","last_peered":"2024-09-21T02:02:41.313555+0000","last_clean":"2024-09-21T02:02:41.313555+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:02:41.313555+0000","last_undegraded":"2024-09-21T02:02:41.313555+0000","last_fullsized":"2024-09-21T02:02:41.313555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981665+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:02:31.981665+0000","last_peered":"2024-09-21T02:02:31.981665+0000","last_clean":"2024-09-21T02:02:31.981665+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:02:31.981665+0000","last_undegraded":"2024-09-21T02:02:31.981665+0000","last_fullsized":"2024-09-21T02:02:31.981665+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:41.314617+0000","last_change":"2024-09-21T02:02:28.956646+0000","last_active":"2024-09-21T02:02:41.314617+0000","last_peered":"2024-09-21T02:02:41.314617+0000","last_clean":"2024-09-21T02:02:41.314617+0000","last_became_active":"2024-09-21T02:02:28.956248+0000","last_became_peered":"2024-09-21T02:02:28.956248+0000","last_unstale":"2024-09-21T02:02:41.314617+0000","last_undegraded":"2024-09-21T02:02:41.314617+0000","last_fullsized":"2024-09-21T02:02:41.314617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:03.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.984768+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:02:31.984768+0000","last_peered":"2024-09-21T02:02:31.984768+0000","last_clean":"2024-09-21T02:02:31.984768+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:02:31.984768+0000","last_undegraded":"2024-09-21T02:02:31.984768+0000","last_fullsized":"2024-09-21T02:02:31.984768+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:32.894529+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:02:32.894529+0000","last_peered":"2024-09-21T02:02:32.894529+0000","last_clean":"2024-09-21T02:02:32.894529+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:02:32.894529+0000","last_undegraded":"2024-09-21T02:02:32.894529+0000","last_fullsized":"2024-09-21T02:02:32.894529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:31.981286+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:02:31.981286+0000","last_peered":"2024-09-21T02:02:31.981286+0000","last_clean":"2024-09-21T02:02:31.981286+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:02:31.981286+0000","last_undegraded":"2024-09-21T02:02:31.981286+0000","last_fullsized":"2024-09-21T02:02:31.981286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:02:51.332267+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:02:51.332267+0000","last_peered":"2024-09-21T02:02:51.332267+0000","last_clean":"2024-09-21T02:02:51.332267+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:02:51.332267+0000","last_undegraded":"2024-09-21T02:02:51.332267+0000","last_fullsized":"2024-09-21T02:02:51.332267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61727,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":167,"num_read_kb":152,"num_write":234,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":185181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":389494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1511424,"data_stored":1034264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":389033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266720,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":839680,"data_stored":389411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":982711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":827392,"data_stored":389141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1449984,"data_stored":981607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":827392,"data_stored":389002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":388575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444093,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":444406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":443795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":390480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":55884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":56495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:02:58.721 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T02:02:58.721 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T02:02:58.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T02:02:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:58 smithi136 ceph-mon[34754]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-21T02:02:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:02:58 smithi136 ceph-mon[34754]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:02:58.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:58 smithi099 ceph-mon[34404]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-09-21T02:02:58.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:02:58 smithi099 ceph-mon[34404]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:02:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.454+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3524579868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4103750 msgr2=0x7f71a4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.454+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3524579868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4103750 0x7f71a4103bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f719402a950 comp rx=0 tx=0).stop 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3524579868 shutdown_connections 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3524579868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a4105180 0x7f71a410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3524579868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a41047e0 0x7f71a4104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3524579868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4103750 0x7f71a4103bc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3524579868 >> 172.21.15.92:0/3524579868 conn(0x7f71a40ff0c0 msgr2=0x7f71a4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3524579868 shutdown_connections 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3524579868 wait complete. 2024-09-21T02:02:59.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 Processor -- start 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.455+0000 7f71aaa9a640 1 -- start start 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a4103750 0x7f71a4076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a41047e0 0x7f71a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 0x7f71a4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71a410f370 con 0x7f71a4105180 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71a410f1f0 con 0x7f71a41047e0 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71a410f4f0 con 0x7f71a4103750 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 0x7f71a4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 0x7f71a4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36328/0 (socket says 172.21.15.92:36328) 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 -- 172.21.15.92:0/3211458118 learned_addr learned my addr 172.21.15.92:0/3211458118 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a8836640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a4103750 0x7f71a4076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a3fff640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a41047e0 0x7f71a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 -- 172.21.15.92:0/3211458118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a4103750 msgr2=0x7f71a4076e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a4103750 0x7f71a4076e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 -- 172.21.15.92:0/3211458118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a41047e0 msgr2=0x7f71a4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a41047e0 0x7f71a4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 -- 172.21.15.92:0/3211458118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a40762c0 con 0x7f71a4105180 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a8836640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a4103750 0x7f71a4076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a9037640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 0x7f71a4075c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f719c00d750 tx=0x7f719c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c017070 con 0x7f71a4105180 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a41a4020 con 0x7f71a4105180 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f719c0054d0 con 0x7f71a4105180 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.456+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c004c60 con 0x7f71a4105180 2024-09-21T02:02:59.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.457+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a41a4410 con 0x7f71a4105180 2024-09-21T02:02:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.458+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a4103bc0 con 0x7f71a4105180 2024-09-21T02:02:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.459+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f719c009bd0 con 0x7f71a4105180 2024-09-21T02:02:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.459+0000 7f71a1ffb640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71900b65a0 0x7f71900b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:02:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.460+0000 7f71a8836640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71900b65a0 0x7f71900b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:02:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.460+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f719c0e6db0 con 0x7f71a4105180 2024-09-21T02:02:59.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.460+0000 7f71a8836640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71900b65a0 0x7f71900b8ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7194006010 tx=0x7f7194009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:02:59.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.464+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f719c0ad690 con 0x7f71a4105180 2024-09-21T02:02:59.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.711+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f71a4063fc0 con 0x7f71a4105180 2024-09-21T02:02:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.714+0000 7f71a1ffb640 1 -- 172.21.15.92:0/3211458118 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f719c0b1a70 con 0x7f71a4105180 2024-09-21T02:02:59.715 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-21T02:02:59.715 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBTKe5mRk+DKhAAV22MmqjvaNVDPs8/icR6eg== 2024-09-21T02:02:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.716+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71900b65a0 msgr2=0x7f71900b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.716+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71900b65a0 0x7f71900b8ab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7194006010 tx=0x7f7194009360 comp rx=0 tx=0).stop 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 msgr2=0x7f71a4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 0x7f71a4075c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f719c00d750 tx=0x7f719c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 shutdown_connections 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71900b65a0 0x7f71900b8ab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71a4105180 0x7f71a4075c50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71a41047e0 0x7f71a4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 --2- 172.21.15.92:0/3211458118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71a4103750 0x7f71a4076e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 >> 172.21.15.92:0/3211458118 conn(0x7f71a40ff0c0 msgr2=0x7f71a4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 shutdown_connections 2024-09-21T02:02:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:02:59.717+0000 7f71aaa9a640 1 -- 172.21.15.92:0/3211458118 wait complete. 2024-09-21T02:03:00.431 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T02:03:00.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T02:03:00.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T02:03:00.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T02:03:00.749 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:00 smithi136 ceph-mon[34754]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-09-21T02:03:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:00 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3211458118' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:03:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:00 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3211458118' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:03:00.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:00 smithi092 ceph-mon[31190]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-09-21T02:03:00.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:00 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3211458118' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:03:00.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:00 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3211458118' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:03:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:00 smithi099 ceph-mon[34404]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-09-21T02:03:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:00 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3211458118' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:03:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:00 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3211458118' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.213+0000 7fe48f98c640 1 -- 172.21.15.92:0/3091150582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 msgr2=0x7fe488102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.213+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3091150582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488102e70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe478009900 tx=0x7fe47802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 -- 172.21.15.92:0/3091150582 shutdown_connections 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3091150582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe488108cf0 0x7fe48810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3091150582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4881083a0 0x7fe4881087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3091150582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488102e70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 -- 172.21.15.92:0/3091150582 >> 172.21.15.92:0/3091150582 conn(0x7fe488079730 msgr2=0x7fe488079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 -- 172.21.15.92:0/3091150582 shutdown_connections 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 -- 172.21.15.92:0/3091150582 wait complete. 2024-09-21T02:03:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 Processor -- start 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.214+0000 7fe48f98c640 1 -- start start 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48f98c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48f98c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881083a0 0x7fe488104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48f98c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe488108cf0 0x7fe488104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48f98c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112960 con 0x7fe488102a60 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48f98c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881127e0 con 0x7fe488108cf0 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48f98c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112ae0 con 0x7fe4881083a0 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36344/0 (socket says 172.21.15.92:36344) 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 -- 172.21.15.92:0/3050783777 learned_addr learned my addr 172.21.15.92:0/3050783777 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48df29640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe488108cf0 0x7fe488104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48cf27640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881083a0 0x7fe488104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 -- 172.21.15.92:0/3050783777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881083a0 msgr2=0x7fe488104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881083a0 0x7fe488104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 -- 172.21.15.92:0/3050783777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe488108cf0 msgr2=0x7fe488104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe488108cf0 0x7fe488104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 -- 172.21.15.92:0/3050783777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a8700 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe48d728640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488104250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe478005940 tx=0x7fe4780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.215+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47803f030 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.216+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478003cb0 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.216+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a8990 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.216+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478035070 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.216+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a8ec0 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.217+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488102e70 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.218+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe478042030 con 0x7fe488102a60 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.219+0000 7fe47e7fc640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4740b6620 0x7fe4740b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.219+0000 7fe48cf27640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4740b6620 0x7fe4740b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.219+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fe478107d80 con 0x7fe488102a60 2024-09-21T02:03:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.219+0000 7fe48cf27640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4740b6620 0x7fe4740b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe484005fd0 tx=0x7fe484005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.222+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4780ce660 con 0x7fe488102a60 2024-09-21T02:03:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.402+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fe4881a9290 con 0x7fe488102a60 2024-09-21T02:03:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.403+0000 7fe47e7fc640 1 -- 172.21.15.92:0/3050783777 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7fe4780d2a40 con 0x7fe488102a60 2024-09-21T02:03:01.403 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4740b6620 msgr2=0x7fe4740b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4740b6620 0x7fe4740b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe484005fd0 tx=0x7fe484005850 comp rx=0 tx=0).stop 2024-09-21T02:03:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 msgr2=0x7fe488104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488104250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe478005940 tx=0x7fe4780042f0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 shutdown_connections 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe4740b6620 0x7fe4740b8b30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe488108cf0 0x7fe488104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881083a0 0x7fe488104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 --2- 172.21.15.92:0/3050783777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe488102a60 0x7fe488104250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 >> 172.21.15.92:0/3050783777 conn(0x7fe488079730 msgr2=0x7fe488101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 shutdown_connections 2024-09-21T02:03:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:01.405+0000 7fe48f98c640 1 -- 172.21.15.92:0/3050783777 wait complete. 2024-09-21T02:03:01.990 INFO:teuthology.orchestra.run.smithi092.stdout:[{"version":55,"timestamp":"2024-09-21T02:02:21.415216+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T02:03:01.991 INFO:tasks.ceph_manager:config epoch is 55 2024-09-21T02:03:01.991 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T02:03:01.991 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T02:03:01.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T02:03:02.235 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:02.631 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:02 smithi092 ceph-mon[31190]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-09-21T02:03:02.631 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:02 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3050783777' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 -- 172.21.15.92:0/1096387753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc410de10 msgr2=0x7fccc4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 --2- 172.21.15.92:0/1096387753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc410de10 0x7fccc4110300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009900 tx=0x7fccbc02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 -- 172.21.15.92:0/1096387753 shutdown_connections 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 --2- 172.21.15.92:0/1096387753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc410de10 0x7fccc4110300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 --2- 172.21.15.92:0/1096387753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4105300 0x7fccc410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 --2- 172.21.15.92:0/1096387753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 0x7fccc4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 -- 172.21.15.92:0/1096387753 >> 172.21.15.92:0/1096387753 conn(0x7fccc4079730 msgr2=0x7fccc4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 -- 172.21.15.92:0/1096387753 shutdown_connections 2024-09-21T02:03:02.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 -- 172.21.15.92:0/1096387753 wait complete. 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.720+0000 7fccc9394640 1 Processor -- start 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 -- start start 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 0x7fccc419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 0x7fccc419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc410de10 0x7fccc419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4113b10 con 0x7fccc4105300 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4113990 con 0x7fccc410de10 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc9394640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4113c90 con 0x7fccc41049b0 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 0x7fccc419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 0x7fccc419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36356/0 (socket says 172.21.15.92:36356) 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 -- 172.21.15.92:0/4231988844 learned_addr learned my addr 172.21.15.92:0/4231988844 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc2ffd640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 0x7fccc419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:02.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 -- 172.21.15.92:0/4231988844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 msgr2=0x7fccc419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 0x7fccc419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc37fe640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc410de10 0x7fccc419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 -- 172.21.15.92:0/4231988844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc410de10 msgr2=0x7fccc419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc410de10 0x7fccc419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc27fc640 1 -- 172.21.15.92:0/4231988844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc41a0430 con 0x7fccc4105300 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.721+0000 7fccc2ffd640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 0x7fccc419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.722+0000 7fccc27fc640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 0x7fccc419aa60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fccac00c970 tx=0x7fccac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.722+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac007bb0 con 0x7fccc4105300 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.722+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc41a0720 con 0x7fccc4105300 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.722+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccac007d50 con 0x7fccc4105300 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.722+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac004e10 con 0x7fccc4105300 2024-09-21T02:03:02.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.722+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc41a6f00 con 0x7fccc4105300 2024-09-21T02:03:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.723+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc4104dc0 con 0x7fccc4105300 2024-09-21T02:03:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.724+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fccac005000 con 0x7fccc4105300 2024-09-21T02:03:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.725+0000 7fccbbfff640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccb00b65a0 0x7fccb00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.725+0000 7fccc2ffd640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccb00b65a0 0x7fccb00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.725+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fccac0e7520 con 0x7fccc4105300 2024-09-21T02:03:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.726+0000 7fccc2ffd640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccb00b65a0 0x7fccb00b8ab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fccc406bc60 tx=0x7fccb4009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:02.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.728+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fccac014030 con 0x7fccc4105300 2024-09-21T02:03:02.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:02 smithi136 ceph-mon[34754]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-09-21T02:03:02.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:02 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3050783777' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:03:02.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:02 smithi099 ceph-mon[34404]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-09-21T02:03:02.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:02 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3050783777' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:03:02.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.954+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fccc419b5f0 con 0x7fccc4105300 2024-09-21T02:03:02.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.959+0000 7fccbbfff640 1 -- 172.21.15.92:0/4231988844 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+296435 (secure 0 0 0) 0x7fccc419b5f0 con 0x7fccc4105300 2024-09-21T02:03:02.963 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccb00b65a0 msgr2=0x7fccb00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccb00b65a0 0x7fccb00b8ab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fccc406bc60 tx=0x7fccb4009570 comp rx=0 tx=0).stop 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 msgr2=0x7fccc419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 0x7fccc419aa60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fccac00c970 tx=0x7fccac00ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 shutdown_connections 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fccb00b65a0 0x7fccb00b8ab0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc410de10 0x7fccc419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccc4105300 0x7fccc419aa60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 --2- 172.21.15.92:0/4231988844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccc41049b0 0x7fccc419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.962+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 >> 172.21.15.92:0/4231988844 conn(0x7fccc4079730 msgr2=0x7fccc40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:02.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.963+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 shutdown_connections 2024-09-21T02:03:02.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:02.963+0000 7fccc9394640 1 -- 172.21.15.92:0/4231988844 wait complete. 2024-09-21T02:03:03.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:03 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4231988844' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:03:03.655 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":1870152537},{"type":"v1","addr":"172.21.15.92:6801","nonce":1870152537}]},"active_addr":"172.21.15.92:6801/1870152537","active_change":"2024-09-21T01:55:10.932523+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24127,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24140,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.92:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3402643159}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":330833430}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":910473048}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":966327342}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2075292940}]}]} 2024-09-21T02:03:03.661 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T02:03:03.661 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T02:03:03.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:03:03.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:03 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4231988844' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:03:03.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:03 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4231988844' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:03:03.905 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 -- 172.21.15.92:0/2401052383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25941083a0 msgr2=0x7f25941087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 --2- 172.21.15.92:0/2401052383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25941083a0 0x7f25941087b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2584009900 tx=0x7f258402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 -- 172.21.15.92:0/2401052383 shutdown_connections 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 --2- 172.21.15.92:0/2401052383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2594108cf0 0x7f259410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 --2- 172.21.15.92:0/2401052383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25941083a0 0x7f25941087b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 --2- 172.21.15.92:0/2401052383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2594102a60 0x7f2594102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 -- 172.21.15.92:0/2401052383 >> 172.21.15.92:0/2401052383 conn(0x7f2594079720 msgr2=0x7f2594079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 -- 172.21.15.92:0/2401052383 shutdown_connections 2024-09-21T02:03:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.388+0000 7f259b72e640 1 -- 172.21.15.92:0/2401052383 wait complete. 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 Processor -- start 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 -- start start 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2594102a60 0x7f2594199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25941083a0 0x7f2594199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 0x7f259419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f25941129b0 con 0x7f2594108cf0 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2594112830 con 0x7f2594102a60 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f259b72e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2594112b30 con 0x7f25941083a0 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 0x7f259419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 0x7f259419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50828/0 (socket says 172.21.15.92:50828) 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 -- 172.21.15.92:0/2599502604 learned_addr learned my addr 172.21.15.92:0/2599502604 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f25994ca640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2594102a60 0x7f2594199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2598cc9640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25941083a0 0x7f2594199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 -- 172.21.15.92:0/2599502604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25941083a0 msgr2=0x7f2594199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25941083a0 0x7f2594199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 -- 172.21.15.92:0/2599502604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2594102a60 msgr2=0x7f2594199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2594102a60 0x7f2594199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2599ccb640 1 -- 172.21.15.92:0/2599502604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259419f2d0 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.389+0000 7f2598cc9640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25941083a0 0x7f2594199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.390+0000 7f2599ccb640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 0x7f259419eba0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f258c00eaa0 tx=0x7f258c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.390+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c00cdb0 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.390+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259419f5c0 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.390+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f258c005020 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.390+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c0116d0 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.390+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259406bb80 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.391+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259406b3b0 con 0x7f2594108cf0 2024-09-21T02:03:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.392+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f258c0054a0 con 0x7f2594108cf0 2024-09-21T02:03:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.393+0000 7f258a7fc640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f25800b65a0 0x7f25800b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.393+0000 7f25994ca640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f25800b65a0 0x7f25800b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.393+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f258c0e78b0 con 0x7f2594108cf0 2024-09-21T02:03:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.393+0000 7f25994ca640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f25800b65a0 0x7f25800b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f259000bbe0 tx=0x7f259000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.396+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f258c016030 con 0x7f2594108cf0 2024-09-21T02:03:04.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:04 smithi092 ceph-mon[31190]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-21T02:03:04.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.575+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2594103ce0 con 0x7f2594108cf0 2024-09-21T02:03:04.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.576+0000 7f258a7fc640 1 -- 172.21.15.92:0/2599502604 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22268 (secure 0 0 0) 0x7f2594103ce0 con 0x7f2594108cf0 2024-09-21T02:03:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:04.578 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:02:31.545307+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:04.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f25800b65a0 msgr2=0x7f25800b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f25800b65a0 0x7f25800b8ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f259000bbe0 tx=0x7f259000b390 comp rx=0 tx=0).stop 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 msgr2=0x7f259419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 0x7f259419eba0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f258c00eaa0 tx=0x7f258c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 shutdown_connections 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f25800b65a0 0x7f25800b8ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2594108cf0 0x7f259419eba0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25941083a0 0x7f2594199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 --2- 172.21.15.92:0/2599502604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2594102a60 0x7f2594199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 >> 172.21.15.92:0/2599502604 conn(0x7f2594079720 msgr2=0x7f25941043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 shutdown_connections 2024-09-21T02:03:04.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:04.579+0000 7f259b72e640 1 -- 172.21.15.92:0/2599502604 wait complete. 2024-09-21T02:03:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:04 smithi136 ceph-mon[34754]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-21T02:03:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:04 smithi099 ceph-mon[34404]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-21T02:03:05.427 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T02:03:05.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:03:05.671 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:05.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:05 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2599502604' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:05.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2599502604' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:05.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:05 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2599502604' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 -- 172.21.15.92:0/645752174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 msgr2=0x7f6fdc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/645752174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc1087b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009900 tx=0x7f6fc402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 -- 172.21.15.92:0/645752174 shutdown_connections 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/645752174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 0x7f6fdc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/645752174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc1087b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/645752174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 0x7f6fdc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 -- 172.21.15.92:0/645752174 >> 172.21.15.92:0/645752174 conn(0x7f6fdc079730 msgr2=0x7f6fdc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 -- 172.21.15.92:0/645752174 shutdown_connections 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.204+0000 7f6fe28ce640 1 -- 172.21.15.92:0/645752174 wait complete. 2024-09-21T02:03:06.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 Processor -- start 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 -- start start 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 0x7f6fdc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 0x7f6fdc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112960 con 0x7f6fdc1083a0 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1127e0 con 0x7f6fdc108cf0 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe28ce640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112ae0 con 0x7f6fdc102a60 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50840/0 (socket says 172.21.15.92:50840) 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 -- 172.21.15.92:0/2061114488 learned_addr learned my addr 172.21.15.92:0/2061114488 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdbfff640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 0x7f6fdc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe0e6b640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 0x7f6fdc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:06.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 -- 172.21.15.92:0/2061114488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 msgr2=0x7f6fdc19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 0x7f6fdc19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 -- 172.21.15.92:0/2061114488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 msgr2=0x7f6fdc19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 0x7f6fdc19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdb7fe640 1 -- 172.21.15.92:0/2061114488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc1a6540 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fdbfff640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 0x7f6fdc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.205+0000 7f6fe0e6b640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 0x7f6fdc19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.206+0000 7f6fdb7fe640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc19b7f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4032fd0 tx=0x7f6fc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.206+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc403f030 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.206+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc4004490 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.206+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc1a67d0 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.206+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4035070 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.206+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1a6cb0 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.207+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc102e70 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.208+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6fc4042030 con 0x7f6fdc1083a0 2024-09-21T02:03:06.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.209+0000 7f6fd97fa640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6fc80b65d0 0x7f6fc80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.209+0000 7f6fdbfff640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6fc80b65d0 0x7f6fc80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.209+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6fc41078a0 con 0x7f6fdc1083a0 2024-09-21T02:03:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.210+0000 7f6fdbfff640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6fc80b65d0 0x7f6fc80b8ae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005fd0 tx=0x7f6fcc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:06.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.213+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fc40ce180 con 0x7f6fdc1083a0 2024-09-21T02:03:06.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.394+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fdc10a550 con 0x7f6fdc1083a0 2024-09-21T02:03:06.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.396+0000 7f6fd97fa640 1 -- 172.21.15.92:0/2061114488 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22268 (secure 0 0 0) 0x7f6fc40d2560 con 0x7f6fdc1083a0 2024-09-21T02:03:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:06.397 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":75,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:02:31.545307+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6fc80b65d0 msgr2=0x7f6fc80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6fc80b65d0 0x7f6fc80b8ae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005fd0 tx=0x7f6fcc005850 comp rx=0 tx=0).stop 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 msgr2=0x7f6fdc19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc19b7f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4032fd0 tx=0x7f6fc4004270 comp rx=0 tx=0).stop 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 shutdown_connections 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6fc80b65d0 0x7f6fc80b8ae0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc108cf0 0x7f6fdc19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fdc1083a0 0x7f6fdc19b7f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 --2- 172.21.15.92:0/2061114488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fdc102a60 0x7f6fdc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 >> 172.21.15.92:0/2061114488 conn(0x7f6fdc079730 msgr2=0x7f6fdc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.399+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 shutdown_connections 2024-09-21T02:03:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:06.400+0000 7f6fe28ce640 1 -- 172.21.15.92:0/2061114488 wait complete. 2024-09-21T02:03:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:06 smithi136 ceph-mon[34754]: pgmap v302: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T02:03:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:06 smithi092 ceph-mon[31190]: pgmap v302: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T02:03:06.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:06 smithi099 ceph-mon[34404]: pgmap v302: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-21T02:03:07.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T02:03:07.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T02:03:07.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T02:03:07.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T02:03:07.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T02:03:07.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T02:03:07.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T02:03:07.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T02:03:07.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T02:03:07.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T02:03:07.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T02:03:07.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T02:03:07.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:07 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2061114488' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:07.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:07 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2061114488' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:07.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:07 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2061114488' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:07.832 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 -- 172.21.15.92:0/1149419334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a14104860 msgr2=0x7f6a14104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 --2- 172.21.15.92:0/1149419334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a14104860 0x7f6a14104c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009ca0 tx=0x7f6a0402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 -- 172.21.15.92:0/1149419334 shutdown_connections 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 --2- 172.21.15.92:0/1149419334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a1410de10 0x7f6a14110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 --2- 172.21.15.92:0/1149419334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a14105290 0x7f6a1410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 --2- 172.21.15.92:0/1149419334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a14104860 0x7f6a14104c70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 -- 172.21.15.92:0/1149419334 >> 172.21.15.92:0/1149419334 conn(0x7f6a14079700 msgr2=0x7f6a14079b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 -- 172.21.15.92:0/1149419334 shutdown_connections 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 -- 172.21.15.92:0/1149419334 wait complete. 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.392+0000 7f6a1934c640 1 Processor -- start 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 -- start start 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a14104860 0x7f6a1419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a14105290 0x7f6a1419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1410de10 0x7f6a1419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14113af0 con 0x7f6a1410de10 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14113970 con 0x7f6a14104860 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a1934c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14113c70 con 0x7f6a14105290 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1410de10 0x7f6a1419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1410de10 0x7f6a1419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50854/0 (socket says 172.21.15.92:50854) 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 -- 172.21.15.92:0/3539519676 learned_addr learned my addr 172.21.15.92:0/3539519676 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a12ffd640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a14104860 0x7f6a1419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a127fc640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a14105290 0x7f6a1419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 -- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a14105290 msgr2=0x7f6a1419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a14105290 0x7f6a1419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 -- 172.21.15.92:0/3539519676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a14104860 msgr2=0x7f6a1419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a14104860 0x7f6a1419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a137fe640 1 -- 172.21.15.92:0/3539519676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a141a04b0 con 0x7f6a1410de10 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.393+0000 7f6a127fc640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a14105290 0x7f6a1419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.394+0000 7f6a137fe640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1410de10 0x7f6a1419fd80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00c970 tx=0x7f6a0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.394+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c007bb0 con 0x7f6a1410de10 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.394+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c007d50 con 0x7f6a1410de10 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.394+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a141a07a0 con 0x7f6a1410de10 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.394+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c004e60 con 0x7f6a1410de10 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.394+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a141a6f00 con 0x7f6a1410de10 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.395+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a141083b0 con 0x7f6a1410de10 2024-09-21T02:03:08.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.397+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6a0c005000 con 0x7f6a1410de10 2024-09-21T02:03:08.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.397+0000 7f6a0bfff640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a000b65a0 0x7f6a000b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:08.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.397+0000 7f6a12ffd640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a000b65a0 0x7f6a000b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:08.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.398+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6a0c0e7410 con 0x7f6a1410de10 2024-09-21T02:03:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.398+0000 7f6a0bfff640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] conn(0x7f6a000bfd30 0x7f6a000c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.398+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 --> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a000c2850 con 0x7f6a000bfd30 2024-09-21T02:03:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.398+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6a0c0e78f0 con 0x7f6a1410de10 2024-09-21T02:03:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.398+0000 7f6a12ffd640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a000b65a0 0x7f6a000b8ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a0402f500 tx=0x7f6a0402f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.398+0000 7f6a127fc640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] conn(0x7f6a000bfd30 0x7f6a000c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:08.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.399+0000 7f6a127fc640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] conn(0x7f6a000bfd30 0x7f6a000c2180 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:08.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.399+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== osd.2 v2:172.21.15.136:6800/2651717363 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a000c2850 con 0x7f6a000bfd30 2024-09-21T02:03:08.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.413+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 --> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a1406b3b0 con 0x7f6a000bfd30 2024-09-21T02:03:08.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.413+0000 7f6a0bfff640 1 -- 172.21.15.92:0/3539519676 <== osd.2 v2:172.21.15.136:6800/2651717363 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6a1406b3b0 con 0x7f6a000bfd30 2024-09-21T02:03:08.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] conn(0x7f6a000bfd30 msgr2=0x7f6a000c2180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:08.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] conn(0x7f6a000bfd30 0x7f6a000c2180 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:08.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a000b65a0 msgr2=0x7f6a000b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:08.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6a000b65a0 0x7f6a000b8ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a0402f500 tx=0x7f6a0402f310 comp rx=0 tx=0).stop 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1410de10 msgr2=0x7f6a1419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 --2- 172.21.15.92:0/3539519676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1410de10 0x7f6a1419fd80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00c970 tx=0x7f6a0c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a137fe640 1 -- 172.21.15.92:0/3539519676 reap_dead start 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 shutdown_connections 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 >> 172.21.15.92:0/3539519676 conn(0x7f6a14079700 msgr2=0x7f6a14107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 shutdown_connections 2024-09-21T02:03:08.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:08.414+0000 7f6a1934c640 1 -- 172.21.15.92:0/3539519676 wait complete. 2024-09-21T02:03:08.428 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:08.429 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:08.429 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:08.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:08 smithi092 ceph-mon[31190]: pgmap v303: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:03:08.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:08 smithi136 ceph-mon[34754]: pgmap v303: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:03:08.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:08 smithi099 ceph-mon[34404]: pgmap v303: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:03:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 -- 172.21.15.92:0/4136068600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484103190 msgr2=0x7fa4841035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 --2- 172.21.15.92:0/4136068600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484103190 0x7fa4841035a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa478009900 tx=0x7fa47802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 -- 172.21.15.92:0/4136068600 shutdown_connections 2024-09-21T02:03:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 --2- 172.21.15.92:0/4136068600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa484108e30 0x7fa48410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 --2- 172.21.15.92:0/4136068600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841084e0 0x7fa4841088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 --2- 172.21.15.92:0/4136068600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484103190 0x7fa4841035a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 -- 172.21.15.92:0/4136068600 >> 172.21.15.92:0/4136068600 conn(0x7fa484079ec0 msgr2=0x7fa48407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 -- 172.21.15.92:0/4136068600 shutdown_connections 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.393+0000 7fa48c362640 1 -- 172.21.15.92:0/4136068600 wait complete. 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 Processor -- start 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 -- start start 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa484103190 0x7fa4840feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841084e0 0x7fa4840ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484108e30 0x7fa4840ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112b50 con 0x7fa484108e30 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841129d0 con 0x7fa4841084e0 2024-09-21T02:03:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.394+0000 7fa48c362640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112cd0 con 0x7fa484103190 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484108e30 0x7fa4840ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484108e30 0x7fa4840ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50872/0 (socket says 172.21.15.92:50872) 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 -- 172.21.15.92:0/1792123968 learned_addr learned my addr 172.21.15.92:0/1792123968 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa4898fd640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841084e0 0x7fa4840ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a0fe640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa484103190 0x7fa4840feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 -- 172.21.15.92:0/1792123968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa484103190 msgr2=0x7fa4840feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa484103190 0x7fa4840feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 -- 172.21.15.92:0/1792123968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841084e0 msgr2=0x7fa4840ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841084e0 0x7fa4840ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 -- 172.21.15.92:0/1792123968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4841a8950 con 0x7fa484108e30 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48a8ff640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484108e30 0x7fa4840ff970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00eaa0 tx=0x7fa47c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004d20 con 0x7fa484108e30 2024-09-21T02:03:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4841a8c40 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c005020 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0116d0 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.395+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a9180 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.396+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa484063a90 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.398+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa47c00caf0 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa4777fe640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4700b65a0 0x7fa4700b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa48a0fe640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4700b65a0 0x7fa4700b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fa47c0e7950 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa4777fe640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] conn(0x7fa4700bfd30 0x7fa4700c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 --> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa47c014ec0 con 0x7fa4700bfd30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fa47c0e7e30 con 0x7fa484108e30 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa48a0fe640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4700b65a0 0x7fa4700b8ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa478000c00 tx=0x7fa478031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.399+0000 7fa4898fd640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] conn(0x7fa4700bfd30 0x7fa4700c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.400+0000 7fa4898fd640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] conn(0x7fa4700bfd30 0x7fa4700c2180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.400+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== osd.3 v2:172.21.15.92:6810/2531772265 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa47c014ec0 con 0x7fa4700bfd30 2024-09-21T02:03:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.414+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 --> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa484063e90 con 0x7fa4700bfd30 2024-09-21T02:03:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.414+0000 7fa4777fe640 1 -- 172.21.15.92:0/1792123968 <== osd.3 v2:172.21.15.92:6810/2531772265 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa484063e90 con 0x7fa4700bfd30 2024-09-21T02:03:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] conn(0x7fa4700bfd30 msgr2=0x7fa4700c2180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] conn(0x7fa4700bfd30 0x7fa4700c2180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4700b65a0 msgr2=0x7fa4700b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4700b65a0 0x7fa4700b8ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa478000c00 tx=0x7fa478031400 comp rx=0 tx=0).stop 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484108e30 msgr2=0x7fa4840ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 --2- 172.21.15.92:0/1792123968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa484108e30 0x7fa4840ff970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00eaa0 tx=0x7fa47c00eea0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48a8ff640 1 -- 172.21.15.92:0/1792123968 reap_dead start 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 shutdown_connections 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 >> 172.21.15.92:0/1792123968 conn(0x7fa484079ec0 msgr2=0x7fa484101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 shutdown_connections 2024-09-21T02:03:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.415+0000 7fa48c362640 1 -- 172.21.15.92:0/1792123968 wait complete. 2024-09-21T02:03:09.663 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:09.665 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:09.666 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:09.666 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:09.667 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 -- 172.21.15.92:0/1176779040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868102a60 msgr2=0x7f5868102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 --2- 172.21.15.92:0/1176779040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868102a60 0x7f5868102e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f585c009900 tx=0x7f585c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 -- 172.21.15.92:0/1176779040 shutdown_connections 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 --2- 172.21.15.92:0/1176779040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5868108cf0 0x7f586810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 --2- 172.21.15.92:0/1176779040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58681083a0 0x7f58681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 --2- 172.21.15.92:0/1176779040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868102a60 0x7f5868102e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 -- 172.21.15.92:0/1176779040 >> 172.21.15.92:0/1176779040 conn(0x7f5868079730 msgr2=0x7f5868079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:09.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 -- 172.21.15.92:0/1176779040 shutdown_connections 2024-09-21T02:03:09.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 -- 172.21.15.92:0/1176779040 wait complete. 2024-09-21T02:03:09.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.922+0000 7f586ed76640 1 Processor -- start 2024-09-21T02:03:09.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 -- start start 2024-09-21T02:03:09.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5868102a60 0x7f586819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58681083a0 0x7f586819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868108cf0 0x7f586819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5868112960 con 0x7f5868108cf0 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58681127e0 con 0x7f58681083a0 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586ed76640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5868112ae0 con 0x7f5868102a60 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868108cf0 0x7f586819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868108cf0 0x7f586819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50902/0 (socket says 172.21.15.92:50902) 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 -- 172.21.15.92:0/2395536642 learned_addr learned my addr 172.21.15.92:0/2395536642 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f5867fff640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58681083a0 0x7f586819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586cb12640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5868102a60 0x7f586819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 -- 172.21.15.92:0/2395536642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5868102a60 msgr2=0x7f586819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5868102a60 0x7f586819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 -- 172.21.15.92:0/2395536642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58681083a0 msgr2=0x7f586819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58681083a0 0x7f586819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.923+0000 7f586d313640 1 -- 172.21.15.92:0/2395536642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681a0310 con 0x7f5868108cf0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.924+0000 7f586d313640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868108cf0 0x7f586819fca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f586000cce0 tx=0x7f586000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.924+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586000f450 con 0x7f5868108cf0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.924+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681a0600 con 0x7f5868108cf0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.924+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586000faa0 con 0x7f5868108cf0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.924+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860018b70 con 0x7f5868108cf0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.924+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5868104130 con 0x7f5868108cf0 2024-09-21T02:03:09.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.925+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f586806b1e0 con 0x7f5868108cf0 2024-09-21T02:03:09.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.926+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5860018d10 con 0x7f5868108cf0 2024-09-21T02:03:09.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.927+0000 7f5865ffb640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f58580b65d0 0x7f58580b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.927+0000 7f586cb12640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f58580b65d0 0x7f58580b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.927+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f58600e7950 con 0x7f5868108cf0 2024-09-21T02:03:09.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.927+0000 7f5865ffb640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] conn(0x7f58580bfd60 0x7f58580c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.927+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 --> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58580c2880 con 0x7f58580bfd60 2024-09-21T02:03:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.927+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f58600e7e30 con 0x7f5868108cf0 2024-09-21T02:03:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.928+0000 7f5867fff640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] conn(0x7f58580bfd60 0x7f58580c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.928+0000 7f586cb12640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f58580b65d0 0x7f58580b8ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f585c031440 tx=0x7f585c03c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.928+0000 7f5867fff640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] conn(0x7f58580bfd60 0x7f58580c21b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.929+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== osd.6 v2:172.21.15.92:6818/1034212183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58580c2880 con 0x7f58580bfd60 2024-09-21T02:03:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.942+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 --> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58681a0af0 con 0x7f58580bfd60 2024-09-21T02:03:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.942+0000 7f5865ffb640 1 -- 172.21.15.92:0/2395536642 <== osd.6 v2:172.21.15.92:6818/1034212183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f58681a0af0 con 0x7f58580bfd60 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] conn(0x7f58580bfd60 msgr2=0x7f58580c21b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] conn(0x7f58580bfd60 0x7f58580c21b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f58580b65d0 msgr2=0x7f58580b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f58580b65d0 0x7f58580b8ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f585c031440 tx=0x7f585c03c040 comp rx=0 tx=0).stop 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868108cf0 msgr2=0x7f586819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 --2- 172.21.15.92:0/2395536642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5868108cf0 0x7f586819fca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f586000cce0 tx=0x7f586000a620 comp rx=0 tx=0).stop 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586d313640 1 -- 172.21.15.92:0/2395536642 reap_dead start 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 shutdown_connections 2024-09-21T02:03:09.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 >> 172.21.15.92:0/2395536642 conn(0x7f5868079730 msgr2=0x7f5868101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:09.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 shutdown_connections 2024-09-21T02:03:09.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:09.943+0000 7f586ed76640 1 -- 172.21.15.92:0/2395536642 wait complete. 2024-09-21T02:03:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 -- 172.21.15.92:0/2578065507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f98102a60 msgr2=0x7f6f98102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/2578065507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f98102a60 0x7f6f98102e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009ca0 tx=0x7f6f8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 -- 172.21.15.92:0/2578065507 shutdown_connections 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/2578065507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f98108cf0 0x7f6f9810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/2578065507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f981083a0 0x7f6f981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/2578065507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f98102a60 0x7f6f98102e70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.336+0000 7f6f9eb66640 1 -- 172.21.15.92:0/2578065507 >> 172.21.15.92:0/2578065507 conn(0x7f6f98079730 msgr2=0x7f6f98079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 -- 172.21.15.92:0/2578065507 shutdown_connections 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 -- 172.21.15.92:0/2578065507 wait complete. 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 Processor -- start 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 -- start start 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f98102a60 0x7f6f9819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f981083a0 0x7f6f9819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f98108cf0 0x7f6f9819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98112a40 con 0x7f6f981083a0 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981128c0 con 0x7f6f98108cf0 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.337+0000 7f6f9eb66640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98112bc0 con 0x7f6f98102a60 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f981083a0 0x7f6f9819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f981083a0 0x7f6f9819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50924/0 (socket says 172.21.15.92:50924) 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 -- 172.21.15.92:0/1551679049 learned_addr learned my addr 172.21.15.92:0/1551679049 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:10.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f9c902640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f98102a60 0x7f6f9819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f9d103640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f98108cf0 0x7f6f9819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 -- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f98102a60 msgr2=0x7f6f9819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f98102a60 0x7f6f9819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 -- 172.21.15.92:0/1551679049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f98108cf0 msgr2=0x7f6f9819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f98108cf0 0x7f6f9819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 -- 172.21.15.92:0/1551679049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f981a66c0 con 0x7f6f981083a0 2024-09-21T02:03:10.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f9d103640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f98108cf0 0x7f6f9819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f97fff640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f981083a0 0x7f6f9819b8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b450 tx=0x7f6f8400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84015020 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f84011d40 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f981a69b0 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.338+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400f340 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.339+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f981a6ef0 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.339+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f98063b50 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.341+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6f84018020 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f95ffb640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f880b65a0 0x7f6f880b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f9c902640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f880b65a0 0x7f6f880b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6f840e7430 con 0x7f6f981083a0 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f95ffb640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] conn(0x7f6f880bfd30 0x7f6f880c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:10.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 --> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f880c2850 con 0x7f6f880bfd30 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6f840e7910 con 0x7f6f981083a0 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f9c902640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f880b65a0 0x7f6f880b8ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c02f500 tx=0x7f6f8c0094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.342+0000 7f6f9d103640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] conn(0x7f6f880bfd30 0x7f6f880c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.343+0000 7f6f9d103640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] conn(0x7f6f880bfd30 0x7f6f880c2180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.344+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== osd.11 v2:172.21.15.136:6824/2995433302 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f880c2850 con 0x7f6f880bfd30 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.357+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 --> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f98063fd0 con 0x7f6f880bfd30 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f95ffb640 1 -- 172.21.15.92:0/1551679049 <== osd.11 v2:172.21.15.136:6824/2995433302 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f98063fd0 con 0x7f6f880bfd30 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] conn(0x7f6f880bfd30 msgr2=0x7f6f880c2180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] conn(0x7f6f880bfd30 0x7f6f880c2180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f880b65a0 msgr2=0x7f6f880b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f880b65a0 0x7f6f880b8ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c02f500 tx=0x7f6f8c0094a0 comp rx=0 tx=0).stop 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f981083a0 msgr2=0x7f6f9819b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 --2- 172.21.15.92:0/1551679049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f981083a0 0x7f6f9819b8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b450 tx=0x7f6f8400b910 comp rx=0 tx=0).stop 2024-09-21T02:03:10.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9d103640 1 -- 172.21.15.92:0/1551679049 reap_dead start 2024-09-21T02:03:10.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 shutdown_connections 2024-09-21T02:03:10.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 >> 172.21.15.92:0/1551679049 conn(0x7f6f98079730 msgr2=0x7f6f98101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:10.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 shutdown_connections 2024-09-21T02:03:10.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:10.358+0000 7f6f9eb66640 1 -- 172.21.15.92:0/1551679049 wait complete. 2024-09-21T02:03:10.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:10 smithi092 ceph-mon[31190]: pgmap v304: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T02:03:10.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:10 smithi136 ceph-mon[34754]: pgmap v304: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T02:03:10.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:10 smithi099 ceph-mon[34404]: pgmap v304: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T02:03:11.445 INFO:teuthology.orchestra.run.smithi092.stdout:296352743444 2024-09-21T02:03:11.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T02:03:11.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T02:03:11.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-09-21T02:03:11.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]} : dispatch 2024-09-21T02:03:11.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T02:03:11.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]} : dispatch 2024-09-21T02:03:11.563 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T02:03:11.564 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T02:03:11.564 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]} : dispatch 2024-09-21T02:03:11.564 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T02:03:11.564 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:11 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]} : dispatch 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 -- 172.21.15.92:0/1754002453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 msgr2=0x7efe981098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 --2- 172.21.15.92:0/1754002453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981098d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe88009900 tx=0x7efe8802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 -- 172.21.15.92:0/1754002453 shutdown_connections 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 --2- 172.21.15.92:0/1754002453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe98109e10 0x7efe981106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 --2- 172.21.15.92:0/1754002453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981098d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 --2- 172.21.15.92:0/1754002453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98102010 0x7efe98102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 -- 172.21.15.92:0/1754002453 >> 172.21.15.92:0/1754002453 conn(0x7efe98077c60 msgr2=0x7efe980780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 -- 172.21.15.92:0/1754002453 shutdown_connections 2024-09-21T02:03:11.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.656+0000 7efe9cf65640 1 -- 172.21.15.92:0/1754002453 wait complete. 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 Processor -- start 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 -- start start 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98102010 0x7efe98102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe98109e10 0x7efe98103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe98113ac0 con 0x7efe981094c0 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe98113940 con 0x7efe98102010 2024-09-21T02:03:11.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9cf65640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe98113c40 con 0x7efe98109e10 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe95d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe95d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50930/0 (socket says 172.21.15.92:50930) 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe95d9b640 1 -- 172.21.15.92:0/2830995659 learned_addr learned my addr 172.21.15.92:0/2830995659 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.657+0000 7efe9659c640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98102010 0x7efe98102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe96d9d640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe98109e10 0x7efe98103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe95d9b640 1 -- 172.21.15.92:0/2830995659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe98109e10 msgr2=0x7efe98103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe95d9b640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe98109e10 0x7efe98103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe95d9b640 1 -- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98102010 msgr2=0x7efe98102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe95d9b640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98102010 0x7efe98102fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe95d9b640 1 -- 172.21.15.92:0/2830995659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981a8810 con 0x7efe981094c0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe95d9b640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981034f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe8802a930 tx=0x7efe88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8803f030 con 0x7efe981094c0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981a8aa0 con 0x7efe981094c0 2024-09-21T02:03:11.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe88003aa0 con 0x7efe981094c0 2024-09-21T02:03:11.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88035070 con 0x7efe981094c0 2024-09-21T02:03:11.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.658+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a8fd0 con 0x7efe981094c0 2024-09-21T02:03:11.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.659+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe98063b50 con 0x7efe981094c0 2024-09-21T02:03:11.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.661+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7efe88042030 con 0x7efe981094c0 2024-09-21T02:03:11.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.662+0000 7efe877fe640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe800b6620 0x7efe800b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:11.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.662+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8269+0+0 (secure 0 0 0) 0x7efe88107fe0 con 0x7efe981094c0 2024-09-21T02:03:11.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe877fe640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] conn(0x7efe800c0210 0x7efe800c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:11.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 --> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe800c2dd0 con 0x7efe800c0210 2024-09-21T02:03:11.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7efe88108430 con 0x7efe981094c0 2024-09-21T02:03:11.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe9659c640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe800b6620 0x7efe800b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:11.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe96d9d640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] conn(0x7efe800c0210 0x7efe800c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:11.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe9659c640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe800b6620 0x7efe800b8b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efe98104a00 tx=0x7efe8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:11.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.663+0000 7efe96d9d640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] conn(0x7efe800c0210 0x7efe800c2700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:11.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.664+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== osd.1 v2:172.21.15.99:6800/3284764622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe800c2dd0 con 0x7efe800c0210 2024-09-21T02:03:11.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.678+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 --> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efe98063f50 con 0x7efe800c0210 2024-09-21T02:03:11.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.678+0000 7efe877fe640 1 -- 172.21.15.92:0/2830995659 <== osd.1 v2:172.21.15.99:6800/3284764622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efe98063f50 con 0x7efe800c0210 2024-09-21T02:03:11.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] conn(0x7efe800c0210 msgr2=0x7efe800c2700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] conn(0x7efe800c0210 0x7efe800c2700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe800b6620 msgr2=0x7efe800b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7efe800b6620 0x7efe800b8b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efe98104a00 tx=0x7efe8c00b390 comp rx=0 tx=0).stop 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 msgr2=0x7efe981034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 --2- 172.21.15.92:0/2830995659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe981094c0 0x7efe981034f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe8802a930 tx=0x7efe88004270 comp rx=0 tx=0).stop 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe96d9d640 1 -- 172.21.15.92:0/2830995659 reap_dead start 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 shutdown_connections 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 >> 172.21.15.92:0/2830995659 conn(0x7efe98077c60 msgr2=0x7efe981055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 shutdown_connections 2024-09-21T02:03:11.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:11.679+0000 7efe9cf65640 1 -- 172.21.15.92:0/2830995659 wait complete. 2024-09-21T02:03:11.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T02:03:11.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-09-21T02:03:11.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]} : dispatch 2024-09-21T02:03:11.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T02:03:11.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]} : dispatch 2024-09-21T02:03:11.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T02:03:11.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T02:03:11.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]} : dispatch 2024-09-21T02:03:11.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T02:03:11.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:11 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]} : dispatch 2024-09-21T02:03:11.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T02:03:11.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-09-21T02:03:11.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T02:03:11.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:11 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]} : dispatch 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: pgmap v305: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-21T02:03:12.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:03:12.724 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:12 smithi092 ceph-mon[31190]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:03:12.750 INFO:teuthology.orchestra.run.smithi092.stdout:55834574920 2024-09-21T02:03:12.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T02:03:12.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: pgmap v305: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:03:12.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:12 smithi136 ceph-mon[34754]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:03:12.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: pgmap v305: 97 pgs: 97 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-21T02:03:12.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T02:03:12.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:03:12.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:12 smithi099 ceph-mon[34404]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:03:13.581 INFO:teuthology.orchestra.run.smithi092.stdout:107374182461 2024-09-21T02:03:13.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T02:03:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 -- 172.21.15.92:0/2904566677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c109e40 msgr2=0x7fe37c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 --2- 172.21.15.92:0/2904566677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c109e40 0x7fe37c110670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe374009ca0 tx=0x7fe37402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 -- 172.21.15.92:0/2904566677 shutdown_connections 2024-09-21T02:03:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 --2- 172.21.15.92:0/2904566677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c109e40 0x7fe37c110670 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 --2- 172.21.15.92:0/2904566677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c1094f0 0x7fe37c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 --2- 172.21.15.92:0/2904566677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c103bb0 0x7fe37c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.963+0000 7fe383af3640 1 -- 172.21.15.92:0/2904566677 >> 172.21.15.92:0/2904566677 conn(0x7fe37c0ff600 msgr2=0x7fe37c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.964+0000 7fe383af3640 1 -- 172.21.15.92:0/2904566677 shutdown_connections 2024-09-21T02:03:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.964+0000 7fe383af3640 1 -- 172.21.15.92:0/2904566677 wait complete. 2024-09-21T02:03:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.964+0000 7fe383af3640 1 Processor -- start 2024-09-21T02:03:13.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.964+0000 7fe383af3640 1 -- start start 2024-09-21T02:03:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe383af3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c103bb0 0x7fe37c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe383af3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c1094f0 0x7fe37c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe383af3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c109e40 0x7fe37c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe383af3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113b00 con 0x7fe37c103bb0 2024-09-21T02:03:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe383af3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113980 con 0x7fe37c1094f0 2024-09-21T02:03:13.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe383af3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113c80 con 0x7fe37c109e40 2024-09-21T02:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c103bb0 0x7fe37c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c103bb0 0x7fe37c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50440/0 (socket says 172.21.15.92:50440) 2024-09-21T02:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 -- 172.21.15.92:0/3697659447 learned_addr learned my addr 172.21.15.92:0/3697659447 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe382090640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c109e40 0x7fe37c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38108e640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c1094f0 0x7fe37c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 -- 172.21.15.92:0/3697659447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c109e40 msgr2=0x7fe37c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe37c109e40 0x7fe37c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 -- 172.21.15.92:0/3697659447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c1094f0 msgr2=0x7fe37c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c1094f0 0x7fe37c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 -- 172.21.15.92:0/3697659447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1a0340 con 0x7fe37c103bb0 2024-09-21T02:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe38188f640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c103bb0 0x7fe37c19a490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe37800c890 tx=0x7fe37800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.965+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378004220 con 0x7fe37c103bb0 2024-09-21T02:03:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.966+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3780043c0 con 0x7fe37c103bb0 2024-09-21T02:03:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.966+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378004a00 con 0x7fe37c103bb0 2024-09-21T02:03:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.966+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a0630 con 0x7fe37c103bb0 2024-09-21T02:03:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.966+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a6e50 con 0x7fe37c103bb0 2024-09-21T02:03:13.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.966+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe37c063b10 con 0x7fe37c103bb0 2024-09-21T02:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.968+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe3780079b0 con 0x7fe37c103bb0 2024-09-21T02:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe372ffd640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3680b6650 0x7fe3680b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe38108e640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3680b6650 0x7fe3680b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe3780e75a0 con 0x7fe37c103bb0 2024-09-21T02:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe372ffd640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] conn(0x7fe3680c0240 0x7fe3680c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 --> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3680c2e00 con 0x7fe3680c0240 2024-09-21T02:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe3780e79f0 con 0x7fe37c103bb0 2024-09-21T02:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe382090640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] conn(0x7fe3680c0240 0x7fe3680c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.969+0000 7fe38108e640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3680b6650 0x7fe3680b8b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005e10 tx=0x7fe36c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.970+0000 7fe382090640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] conn(0x7fe3680c0240 0x7fe3680c2730 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:13.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.970+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== osd.9 v2:172.21.15.92:6826/1884194979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3680c2e00 con 0x7fe3680c0240 2024-09-21T02:03:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 --> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe37c063fe0 con 0x7fe3680c0240 2024-09-21T02:03:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe372ffd640 1 -- 172.21.15.92:0/3697659447 <== osd.9 v2:172.21.15.92:6826/1884194979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe37c063fe0 con 0x7fe3680c0240 2024-09-21T02:03:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] conn(0x7fe3680c0240 msgr2=0x7fe3680c2730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] conn(0x7fe3680c0240 0x7fe3680c2730 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3680b6650 msgr2=0x7fe3680b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3680b6650 0x7fe3680b8b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005e10 tx=0x7fe36c005da0 comp rx=0 tx=0).stop 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c103bb0 msgr2=0x7fe37c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 --2- 172.21.15.92:0/3697659447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe37c103bb0 0x7fe37c19a490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe37800c890 tx=0x7fe37800cd50 comp rx=0 tx=0).stop 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe382090640 1 -- 172.21.15.92:0/3697659447 reap_dead start 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 shutdown_connections 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 >> 172.21.15.92:0/3697659447 conn(0x7fe37c0ff600 msgr2=0x7fe37c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.984+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 shutdown_connections 2024-09-21T02:03:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:13.985+0000 7fe383af3640 1 -- 172.21.15.92:0/3697659447 wait complete. 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 -- 172.21.15.92:0/1403147481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 msgr2=0x7f6bb410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 --2- 172.21.15.92:0/1403147481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb410f540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009900 tx=0x7f6bac02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 -- 172.21.15.92:0/1403147481 shutdown_connections 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 --2- 172.21.15.92:0/1403147481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb410f540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 --2- 172.21.15.92:0/1403147481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bb41083c0 0x7f6bb41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 --2- 172.21.15.92:0/1403147481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bb4102a80 0x7f6bb4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.145+0000 7f6bb924a640 1 -- 172.21.15.92:0/1403147481 >> 172.21.15.92:0/1403147481 conn(0x7f6bb4079740 msgr2=0x7f6bb4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.146+0000 7f6bb924a640 1 -- 172.21.15.92:0/1403147481 shutdown_connections 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.146+0000 7f6bb924a640 1 -- 172.21.15.92:0/1403147481 wait complete. 2024-09-21T02:03:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 Processor -- start 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 -- start start 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bb4102a80 0x7f6bb4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bb41083c0 0x7f6bb4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4112980 con 0x7f6bb4108d10 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4112800 con 0x7f6bb4102a80 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb924a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4112b00 con 0x7f6bb41083c0 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50456/0 (socket says 172.21.15.92:50456) 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb359e640 1 -- 172.21.15.92:0/328870660 learned_addr learned my addr 172.21.15.92:0/328870660 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb259c640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bb41083c0 0x7f6bb4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.147+0000 7f6bb2d9d640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bb4102a80 0x7f6bb4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb359e640 1 -- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bb41083c0 msgr2=0x7f6bb4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb359e640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bb41083c0 0x7f6bb4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb359e640 1 -- 172.21.15.92:0/328870660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bb4102a80 msgr2=0x7f6bb4199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb359e640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bb4102a80 0x7f6bb4199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb359e640 1 -- 172.21.15.92:0/328870660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb419f240 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb259c640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bb41083c0 0x7f6bb4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb2d9d640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bb4102a80 0x7f6bb4199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb359e640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb419ebd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004770 tx=0x7f6bac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03f030 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb419f4d0 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bac003aa0 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac035070 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.148+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb419fa00 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.149+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bb406bef0 con 0x7f6bb4108d10 2024-09-21T02:03:14.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.151+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6bac042030 con 0x7f6bb4108d10 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.151+0000 7f6b9bfff640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6b9c0b65d0 0x7f6b9c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.151+0000 7f6bb2d9d640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6b9c0b65d0 0x7f6b9c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.152+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6bac1080e0 con 0x7f6bb4108d10 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.152+0000 7f6b9bfff640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] conn(0x7f6b9c0c01c0 0x7f6b9c0c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.152+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 --> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b9c0c2d80 con 0x7f6b9c0c01c0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.152+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f6bac108530 con 0x7f6bb4108d10 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.152+0000 7f6bb2d9d640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6b9c0b65d0 0x7f6b9c0b8ae0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005fd0 tx=0x7f6ba40093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.152+0000 7f6bb259c640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] conn(0x7f6b9c0c01c0 0x7f6b9c0c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.153+0000 7f6bb259c640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] conn(0x7f6b9c0c01c0 0x7f6b9c0c26b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.153+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== osd.5 v2:172.21.15.136:6808/659398601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b9c0c2d80 con 0x7f6b9c0c01c0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.167+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 --> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bb406c2f0 con 0x7f6b9c0c01c0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6b9bfff640 1 -- 172.21.15.92:0/328870660 <== osd.5 v2:172.21.15.136:6808/659398601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bb406c2f0 con 0x7f6b9c0c01c0 2024-09-21T02:03:14.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] conn(0x7f6b9c0c01c0 msgr2=0x7f6b9c0c26b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] conn(0x7f6b9c0c01c0 0x7f6b9c0c26b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6b9c0b65d0 msgr2=0x7f6b9c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6b9c0b65d0 0x7f6b9c0b8ae0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005fd0 tx=0x7f6ba40093e0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 msgr2=0x7f6bb419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 --2- 172.21.15.92:0/328870660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bb4108d10 0x7f6bb419ebd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004770 tx=0x7f6bac004270 comp rx=0 tx=0).stop 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb359e640 1 -- 172.21.15.92:0/328870660 reap_dead start 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 shutdown_connections 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 >> 172.21.15.92:0/328870660 conn(0x7f6bb4079740 msgr2=0x7f6bb4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 shutdown_connections 2024-09-21T02:03:14.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.168+0000 7f6bb924a640 1 -- 172.21.15.92:0/328870660 wait complete. 2024-09-21T02:03:14.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 -- 172.21.15.92:0/2712809964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 msgr2=0x7fe3b0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 --2- 172.21.15.92:0/2712809964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b0104cd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009900 tx=0x7fe3a002a410 comp rx=0 tx=0).stop 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 -- 172.21.15.92:0/2712809964 shutdown_connections 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 --2- 172.21.15.92:0/2712809964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b010de70 0x7fe3b0110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 --2- 172.21.15.92:0/2712809964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b01052f0 0x7fe3b010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 --2- 172.21.15.92:0/2712809964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b0104cd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 -- 172.21.15.92:0/2712809964 >> 172.21.15.92:0/2712809964 conn(0x7fe3b0079770 msgr2=0x7fe3b0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 -- 172.21.15.92:0/2712809964 shutdown_connections 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.591+0000 7fe3b6170640 1 -- 172.21.15.92:0/2712809964 wait complete. 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.592+0000 7fe3b6170640 1 Processor -- start 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.592+0000 7fe3b6170640 1 -- start start 2024-09-21T02:03:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.592+0000 7fe3b6170640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b01027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b01052f0 0x7fe3b0102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b010de70 0x7fe3b0103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0113bd0 con 0x7fe3b01048c0 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0113a50 con 0x7fe3b010de70 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0113d50 con 0x7fe3b01052f0 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b01027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b01027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50470/0 (socket says 172.21.15.92:50470) 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 -- 172.21.15.92:0/3038937559 learned_addr learned my addr 172.21.15.92:0/3038937559 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3aeffd640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b01052f0 0x7fe3b0102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3affff640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b010de70 0x7fe3b0103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 -- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b01052f0 msgr2=0x7fe3b0102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b01052f0 0x7fe3b0102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 -- 172.21.15.92:0/3038937559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b010de70 msgr2=0x7fe3b0103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b010de70 0x7fe3b0103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 -- 172.21.15.92:0/3038937559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b01a88c0 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3affff640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b010de70 0x7fe3b0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3af7fe640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b01027d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0006010 tx=0x7fe3a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a003f030 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b01a8b50 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a0003800 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0035070 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.593+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b01a9030 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.594+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3b01011d0 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.596+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fe3a00039b0 con 0x7fe3b01048c0 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.596+0000 7fe3acff9640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3980b65a0 0x7fe3980b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3aeffd640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3980b65a0 0x7fe3980b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe3a0108580 con 0x7fe3b01048c0 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3acff9640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] conn(0x7fe3980c0190 0x7fe3980c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 --> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3980c2d50 con 0x7fe3980c0190 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe3a01089d0 con 0x7fe3b01048c0 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3aeffd640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3980b65a0 0x7fe3980b8ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0102470 tx=0x7fe3a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.597+0000 7fe3affff640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] conn(0x7fe3980c0190 0x7fe3980c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.598+0000 7fe3affff640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] conn(0x7fe3980c0190 0x7fe3980c2680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:14.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.598+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== osd.8 v2:172.21.15.136:6816/3496989963 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3980c2d50 con 0x7fe3980c0190 2024-09-21T02:03:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.614+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 --> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3b006b390 con 0x7fe3980c0190 2024-09-21T02:03:14.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3acff9640 1 -- 172.21.15.92:0/3038937559 <== osd.8 v2:172.21.15.136:6816/3496989963 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe3b006b390 con 0x7fe3980c0190 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] conn(0x7fe3980c0190 msgr2=0x7fe3980c2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] conn(0x7fe3980c0190 0x7fe3980c2680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3980b65a0 msgr2=0x7fe3980b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3980b65a0 0x7fe3980b8ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0102470 tx=0x7fe3a4008040 comp rx=0 tx=0).stop 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 msgr2=0x7fe3b01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 --2- 172.21.15.92:0/3038937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b01048c0 0x7fe3b01027d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0006010 tx=0x7fe3a00042f0 comp rx=0 tx=0).stop 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3affff640 1 -- 172.21.15.92:0/3038937559 reap_dead start 2024-09-21T02:03:14.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 shutdown_connections 2024-09-21T02:03:14.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 >> 172.21.15.92:0/3038937559 conn(0x7fe3b0079770 msgr2=0x7fe3b01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:14.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 shutdown_connections 2024-09-21T02:03:14.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:14.615+0000 7fe3b6170640 1 -- 172.21.15.92:0/3038937559 wait complete. 2024-09-21T02:03:14.687 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:14 smithi092 ceph-mon[31190]: pgmap v308: 97 pgs: 1 peering, 96 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:14.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:14 smithi136 ceph-mon[34754]: pgmap v308: 97 pgs: 1 peering, 96 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:14.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:14 smithi099 ceph-mon[34404]: pgmap v308: 97 pgs: 1 peering, 96 active+clean; 648 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:15.790 INFO:teuthology.orchestra.run.smithi092.stdout:176093659182 2024-09-21T02:03:15.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T02:03:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 -- 172.21.15.92:0/4078510731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018102a60 msgr2=0x7fa018102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/4078510731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018102a60 0x7fa018102e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa008009900 tx=0x7fa00802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 -- 172.21.15.92:0/4078510731 shutdown_connections 2024-09-21T02:03:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/4078510731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa018108cf0 0x7fa01810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:15.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/4078510731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0181083a0 0x7fa0181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/4078510731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018102a60 0x7fa018102e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 -- 172.21.15.92:0/4078510731 >> 172.21.15.92:0/4078510731 conn(0x7fa018079730 msgr2=0x7fa018079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 -- 172.21.15.92:0/4078510731 shutdown_connections 2024-09-21T02:03:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.955+0000 7fa01e2b6640 1 -- 172.21.15.92:0/4078510731 wait complete. 2024-09-21T02:03:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 Processor -- start 2024-09-21T02:03:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 -- start start 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018102a60 0x7fa018104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0181083a0 0x7fa018104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018108cf0 0x7fa018104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa018112960 con 0x7fa018108cf0 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181127e0 con 0x7fa018102a60 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01e2b6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa018112ae0 con 0x7fa0181083a0 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01c853640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018108cf0 0x7fa018104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01c853640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018108cf0 0x7fa018104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50488/0 (socket says 172.21.15.92:50488) 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa01c853640 1 -- 172.21.15.92:0/3664336239 learned_addr learned my addr 172.21.15.92:0/3664336239 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa0177fe640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0181083a0 0x7fa018104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.956+0000 7fa017fff640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018102a60 0x7fa018104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01c853640 1 -- 172.21.15.92:0/3664336239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0181083a0 msgr2=0x7fa018104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:15.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01c853640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0181083a0 0x7fa018104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01c853640 1 -- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018102a60 msgr2=0x7fa018104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01c853640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018102a60 0x7fa018104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01c853640 1 -- 172.21.15.92:0/3664336239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0181a8720 con 0x7fa018108cf0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01c853640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018108cf0 0x7fa018104cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa01000eaa0 tx=0x7fa01000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cdb0 con 0x7fa018108cf0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa010005020 con 0x7fa018108cf0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0181a8a10 con 0x7fa018108cf0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0100116d0 con 0x7fa018108cf0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.957+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0181a8f00 con 0x7fa018108cf0 2024-09-21T02:03:15.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.958+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa01806b1e0 con 0x7fa018108cf0 2024-09-21T02:03:15.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.960+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa0100051d0 con 0x7fa018108cf0 2024-09-21T02:03:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.961+0000 7fa0157fa640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa0040b65a0 0x7fa0040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.961+0000 7fa017fff640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa0040b65a0 0x7fa0040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.961+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa0100e7a30 con 0x7fa018108cf0 2024-09-21T02:03:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.962+0000 7fa0157fa640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] conn(0x7fa0040c0190 0x7fa0040c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:15.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.962+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 --> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa01001be50 con 0x7fa0040c0190 2024-09-21T02:03:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.962+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fa0100e7e20 con 0x7fa018108cf0 2024-09-21T02:03:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.962+0000 7fa017fff640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa0040b65a0 0x7fa0040b8ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa008031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.962+0000 7fa0177fe640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] conn(0x7fa0040c0190 0x7fa0040c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.962+0000 7fa0177fe640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] conn(0x7fa0040c0190 0x7fa0040c2680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:15.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.963+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== osd.7 v2:172.21.15.99:6816/1697733163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa01001be50 con 0x7fa0040c0190 2024-09-21T02:03:15.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.977+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 --> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa018103ce0 con 0x7fa0040c0190 2024-09-21T02:03:15.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa0157fa640 1 -- 172.21.15.92:0/3664336239 <== osd.7 v2:172.21.15.99:6816/1697733163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa018103ce0 con 0x7fa0040c0190 2024-09-21T02:03:15.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] conn(0x7fa0040c0190 msgr2=0x7fa0040c2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:15.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] conn(0x7fa0040c0190 0x7fa0040c2680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:15.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa0040b65a0 msgr2=0x7fa0040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa0040b65a0 0x7fa0040b8ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa008031400 comp rx=0 tx=0).stop 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018108cf0 msgr2=0x7fa018104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01e2b6640 1 --2- 172.21.15.92:0/3664336239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa018108cf0 0x7fa018104cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa01000eaa0 tx=0x7fa01000ef60 comp rx=0 tx=0).stop 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.978+0000 7fa01c853640 1 -- 172.21.15.92:0/3664336239 reap_dead start 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.979+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 shutdown_connections 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.979+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 >> 172.21.15.92:0/3664336239 conn(0x7fa018079730 msgr2=0x7fa018101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.979+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 shutdown_connections 2024-09-21T02:03:15.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:15.979+0000 7fa01e2b6640 1 -- 172.21.15.92:0/3664336239 wait complete. 2024-09-21T02:03:16.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 -- 172.21.15.92:0/707225364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c102a80 msgr2=0x7ff76c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 --2- 172.21.15.92:0/707225364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c102a80 0x7ff76c102e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009900 tx=0x7ff75c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 -- 172.21.15.92:0/707225364 shutdown_connections 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 --2- 172.21.15.92:0/707225364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c108d10 0x7ff76c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 --2- 172.21.15.92:0/707225364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c1083c0 0x7ff76c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 --2- 172.21.15.92:0/707225364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c102a80 0x7ff76c102e90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 -- 172.21.15.92:0/707225364 >> 172.21.15.92:0/707225364 conn(0x7ff76c079740 msgr2=0x7ff76c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 -- 172.21.15.92:0/707225364 shutdown_connections 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.120+0000 7ff770be0640 1 -- 172.21.15.92:0/707225364 wait complete. 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 Processor -- start 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 -- start start 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c102a80 0x7ff76c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c1083c0 0x7ff76c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c108d10 0x7ff76c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c112a60 con 0x7ff76c108d10 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1128e0 con 0x7ff76c1083c0 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff770be0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c112be0 con 0x7ff76c102a80 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c108d10 0x7ff76c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c108d10 0x7ff76c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50494/0 (socket says 172.21.15.92:50494) 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 -- 172.21.15.92:0/492357351 learned_addr learned my addr 172.21.15.92:0/492357351 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76a59c640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c102a80 0x7ff76c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff769d9b640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c1083c0 0x7ff76c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 -- 172.21.15.92:0/492357351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c102a80 msgr2=0x7ff76c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c102a80 0x7ff76c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 -- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c1083c0 msgr2=0x7ff76c19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c1083c0 0x7ff76c19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:16.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff76ad9d640 1 -- 172.21.15.92:0/492357351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c1a6680 con 0x7ff76c108d10 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.121+0000 7ff769d9b640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c1083c0 0x7ff76c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.122+0000 7ff76ad9d640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c108d10 0x7ff76c19be70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff76400b970 tx=0x7ff76400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.122+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764019070 con 0x7ff76c108d10 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.122+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff764014440 con 0x7ff76c108d10 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.122+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c1a6970 con 0x7ff76c108d10 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.122+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640075d0 con 0x7ff76c108d10 2024-09-21T02:03:16.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.122+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c1a6f30 con 0x7ff76c108d10 2024-09-21T02:03:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.123+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff76c063b50 con 0x7ff76c108d10 2024-09-21T02:03:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.125+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff76401c070 con 0x7ff76c108d10 2024-09-21T02:03:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.125+0000 7ff7637fe640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7580b65d0 0x7ff7580b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:16.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.125+0000 7ff76a59c640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7580b65d0 0x7ff7580b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.126+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff7640e74b0 con 0x7ff76c108d10 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.126+0000 7ff7637fe640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] conn(0x7ff7580c01c0 0x7ff7580c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.126+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 --> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7580c2d80 con 0x7ff7580c01c0 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.126+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff7640e7900 con 0x7ff76c108d10 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.126+0000 7ff76a59c640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7580b65d0 0x7ff7580b8ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff75c000c00 tx=0x7ff75c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.126+0000 7ff769d9b640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] conn(0x7ff7580c01c0 0x7ff7580c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:16.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.127+0000 7ff769d9b640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] conn(0x7ff7580c01c0 0x7ff7580c26b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:16.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.127+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== osd.4 v2:172.21.15.99:6808/1012266354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7580c2d80 con 0x7ff7580c01c0 2024-09-21T02:03:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.141+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 --> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff76c063f50 con 0x7ff7580c01c0 2024-09-21T02:03:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.141+0000 7ff7637fe640 1 -- 172.21.15.92:0/492357351 <== osd.4 v2:172.21.15.99:6808/1012266354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff76c063f50 con 0x7ff7580c01c0 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] conn(0x7ff7580c01c0 msgr2=0x7ff7580c26b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] conn(0x7ff7580c01c0 0x7ff7580c26b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7580b65d0 msgr2=0x7ff7580b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7580b65d0 0x7ff7580b8ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff75c000c00 tx=0x7ff75c031400 comp rx=0 tx=0).stop 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c108d10 msgr2=0x7ff76c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 --2- 172.21.15.92:0/492357351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c108d10 0x7ff76c19be70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff76400b970 tx=0x7ff76400be30 comp rx=0 tx=0).stop 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff76ad9d640 1 -- 172.21.15.92:0/492357351 reap_dead start 2024-09-21T02:03:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 shutdown_connections 2024-09-21T02:03:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 >> 172.21.15.92:0/492357351 conn(0x7ff76c079740 msgr2=0x7ff76c101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 shutdown_connections 2024-09-21T02:03:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:16.142+0000 7ff770be0640 1 -- 172.21.15.92:0/492357351 wait complete. 2024-09-21T02:03:16.776 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:16 smithi092 ceph-mon[31190]: pgmap v309: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:16 smithi136 ceph-mon[34754]: pgmap v309: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:16.803 INFO:teuthology.orchestra.run.smithi092.stdout:249108103200 2024-09-21T02:03:16.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T02:03:16.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:16 smithi099 ceph-mon[34404]: pgmap v309: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:17.509 INFO:teuthology.orchestra.run.smithi092.stdout:73014444096 2024-09-21T02:03:17.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T02:03:17.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 -- 172.21.15.92:0/719433348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b81083d0 msgr2=0x7ff1b81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:17.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 --2- 172.21.15.92:0/719433348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b81083d0 0x7ff1b81087e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009900 tx=0x7ff1a002a410 comp rx=0 tx=0).stop 2024-09-21T02:03:17.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 -- 172.21.15.92:0/719433348 shutdown_connections 2024-09-21T02:03:17.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 --2- 172.21.15.92:0/719433348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b8108d20 0x7ff1b810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 --2- 172.21.15.92:0/719433348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b81083d0 0x7ff1b81087e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 --2- 172.21.15.92:0/719433348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8102ae0 0x7ff1b8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 -- 172.21.15.92:0/719433348 >> 172.21.15.92:0/719433348 conn(0x7ff1b80797b0 msgr2=0x7ff1b8079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 -- 172.21.15.92:0/719433348 shutdown_connections 2024-09-21T02:03:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.668+0000 7ff1bdb20640 1 -- 172.21.15.92:0/719433348 wait complete. 2024-09-21T02:03:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 Processor -- start 2024-09-21T02:03:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 -- start start 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8102ae0 0x7ff1b819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b81083d0 0x7ff1b819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b8108d20 0x7ff1b81a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8112a00 con 0x7ff1b8108d20 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8112880 con 0x7ff1b81083d0 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.669+0000 7ff1bdb20640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8112b80 con 0x7ff1b8102ae0 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b8108d20 0x7ff1b81a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b8108d20 0x7ff1b81a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50516/0 (socket says 172.21.15.92:50516) 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 -- 172.21.15.92:0/2685671585 learned_addr learned my addr 172.21.15.92:0/2685671585 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b77fe640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8102ae0 0x7ff1b819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1affff640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b81083d0 0x7ff1b819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 -- 172.21.15.92:0/2685671585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8102ae0 msgr2=0x7ff1b819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8102ae0 0x7ff1b819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 -- 172.21.15.92:0/2685671585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b81083d0 msgr2=0x7ff1b819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b81083d0 0x7ff1b819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 -- 172.21.15.92:0/2685671585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b81a26c0 con 0x7ff1b8108d20 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b7fff640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b8108d20 0x7ff1b81a1f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000cad0 tx=0x7ff1b000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000ad60 con 0x7ff1b8108d20 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.670+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b0005020 con 0x7ff1b8108d20 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.671+0000 7ff1bdb20640 1 -- 172.21.15.92:0/2685671585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81a29b0 con 0x7ff1b8108d20 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.671+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000f6b0 con 0x7ff1b8108d20 2024-09-21T02:03:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.671+0000 7ff1bdb20640 1 -- 172.21.15.92:0/2685671585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b81a9110 con 0x7ff1b8108d20 2024-09-21T02:03:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.672+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1b8063b50 con 0x7ff1b8108d20 2024-09-21T02:03:17.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.674+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff1b00051d0 con 0x7ff1b8108d20 2024-09-21T02:03:17.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.675+0000 7ff1b57fa640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1a40b6620 0x7ff1a40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:17.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.675+0000 7ff1b77fe640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1a40b6620 0x7ff1a40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:17.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.675+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff1b00e7820 con 0x7ff1b8108d20 2024-09-21T02:03:17.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.676+0000 7ff1b57fa640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] conn(0x7ff1a40c0210 0x7ff1a40c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:17.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.676+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 --> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1b0016ec0 con 0x7ff1a40c0210 2024-09-21T02:03:17.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.676+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff1b00e7c10 con 0x7ff1b8108d20 2024-09-21T02:03:17.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.676+0000 7ff1b77fe640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1a40b6620 0x7ff1a40b8b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009990 tx=0x7ff1a8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:17.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.676+0000 7ff1affff640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] conn(0x7ff1a40c0210 0x7ff1a40c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:17.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.676+0000 7ff1affff640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] conn(0x7ff1a40c0210 0x7ff1a40c2700 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:17.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.677+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== osd.0 v2:172.21.15.92:6802/2450056429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1b0016ec0 con 0x7ff1a40c0210 2024-09-21T02:03:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 --> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1b8063fd0 con 0x7ff1a40c0210 2024-09-21T02:03:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1b57fa640 1 -- 172.21.15.92:0/2685671585 <== osd.0 v2:172.21.15.92:6802/2450056429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1b8063fd0 con 0x7ff1a40c0210 2024-09-21T02:03:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] conn(0x7ff1a40c0210 msgr2=0x7ff1a40c2700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] conn(0x7ff1a40c0210 0x7ff1a40c2700 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1a40b6620 msgr2=0x7ff1a40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1a40b6620 0x7ff1a40b8b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009990 tx=0x7ff1a8009110 comp rx=0 tx=0).stop 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b8108d20 msgr2=0x7ff1b81a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1ae7fc640 1 --2- 172.21.15.92:0/2685671585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b8108d20 0x7ff1b81a1f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000cad0 tx=0x7ff1b000ced0 comp rx=0 tx=0).stop 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.691+0000 7ff1b7fff640 1 -- 172.21.15.92:0/2685671585 reap_dead start 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.692+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 shutdown_connections 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.692+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 >> 172.21.15.92:0/2685671585 conn(0x7ff1b80797b0 msgr2=0x7ff1b806b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.692+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 shutdown_connections 2024-09-21T02:03:17.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.692+0000 7ff1ae7fc640 1 -- 172.21.15.92:0/2685671585 wait complete. 2024-09-21T02:03:18.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 -- 172.21.15.92:0/3965616781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 msgr2=0x7fd0701087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:18.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 --2- 172.21.15.92:0/3965616781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd0701087b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd060009900 tx=0x7fd06002a410 comp rx=0 tx=0).stop 2024-09-21T02:03:18.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 -- 172.21.15.92:0/3965616781 shutdown_connections 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 --2- 172.21.15.92:0/3965616781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd070108cf0 0x7fd07010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 --2- 172.21.15.92:0/3965616781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd0701087b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 --2- 172.21.15.92:0/3965616781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd070102a60 0x7fd070102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 -- 172.21.15.92:0/3965616781 >> 172.21.15.92:0/3965616781 conn(0x7fd070079730 msgr2=0x7fd070079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 -- 172.21.15.92:0/3965616781 shutdown_connections 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 -- 172.21.15.92:0/3965616781 wait complete. 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.922+0000 7fd077c28640 1 Processor -- start 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 -- start start 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd070102a60 0x7fd07019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd07019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd070108cf0 0x7fd07019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd070113a80 con 0x7fd0701083a0 2024-09-21T02:03:18.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd070113900 con 0x7fd070108cf0 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd077c28640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd070113c00 con 0x7fd070102a60 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd07019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd07019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50522/0 (socket says 172.21.15.92:50522) 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 -- 172.21.15.92:0/2620570576 learned_addr learned my addr 172.21.15.92:0/2620570576 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0759c4640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd070102a60 0x7fd07019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0761c5640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd070108cf0 0x7fd07019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:18.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 -- 172.21.15.92:0/2620570576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd070102a60 msgr2=0x7fd07019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd070102a60 0x7fd07019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 -- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd070108cf0 msgr2=0x7fd07019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd070108cf0 0x7fd07019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 -- 172.21.15.92:0/2620570576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0701a04a0 con 0x7fd0701083a0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.923+0000 7fd0751c3640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd07019aad0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd060032fd0 tx=0x7fd060004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.924+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06003f030 con 0x7fd0701083a0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.924+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0701a0730 con 0x7fd0701083a0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.924+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd060004490 con 0x7fd0701083a0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.924+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060035070 con 0x7fd0701083a0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.924+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701a6ef0 con 0x7fd0701083a0 2024-09-21T02:03:18.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.924+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd070063ad0 con 0x7fd0701083a0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.926+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd060042030 con 0x7fd0701083a0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd066ffd640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd05c0b65a0 0x7fd05c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd0759c4640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd05c0b65a0 0x7fd05c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd060107da0 con 0x7fd0701083a0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd066ffd640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] conn(0x7fd05c0c0190 0x7fd05c0c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 --> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd05c0c2d50 con 0x7fd05c0c0190 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd0601081f0 con 0x7fd0701083a0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd0759c4640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd05c0b65a0 0x7fd05c0b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0058c0 tx=0x7fd06c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.927+0000 7fd0761c5640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] conn(0x7fd05c0c0190 0x7fd05c0c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.928+0000 7fd0761c5640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] conn(0x7fd05c0c0190 0x7fd05c0c2680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.929+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== osd.10 v2:172.21.15.99:6824/3821192974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd05c0c2d50 con 0x7fd05c0c0190 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.942+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 --> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd070063ed0 con 0x7fd05c0c0190 2024-09-21T02:03:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd066ffd640 1 -- 172.21.15.92:0/2620570576 <== osd.10 v2:172.21.15.99:6824/3821192974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd070063ed0 con 0x7fd05c0c0190 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] conn(0x7fd05c0c0190 msgr2=0x7fd05c0c2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] conn(0x7fd05c0c0190 0x7fd05c0c2680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd05c0b65a0 msgr2=0x7fd05c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd05c0b65a0 0x7fd05c0b8ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0058c0 tx=0x7fd06c00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 msgr2=0x7fd07019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 --2- 172.21.15.92:0/2620570576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0701083a0 0x7fd07019aad0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd060032fd0 tx=0x7fd060004270 comp rx=0 tx=0).stop 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd0761c5640 1 -- 172.21.15.92:0/2620570576 reap_dead start 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 shutdown_connections 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 >> 172.21.15.92:0/2620570576 conn(0x7fd070079730 msgr2=0x7fd070105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 shutdown_connections 2024-09-21T02:03:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:17.943+0000 7fd077c28640 1 -- 172.21.15.92:0/2620570576 wait complete. 2024-09-21T02:03:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:18 smithi136 ceph-mon[34754]: pgmap v310: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:18 smithi092 ceph-mon[31190]: pgmap v310: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:18.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:18 smithi099 ceph-mon[34404]: pgmap v310: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:18.893 INFO:teuthology.orchestra.run.smithi092.stdout:154618822708 2024-09-21T02:03:18.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T02:03:19.301 INFO:teuthology.orchestra.run.smithi092.stdout:274877906971 2024-09-21T02:03:19.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T02:03:19.772 INFO:teuthology.orchestra.run.smithi092.stdout:128849018938 2024-09-21T02:03:19.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T02:03:20.452 INFO:teuthology.orchestra.run.smithi092.stdout:201863462954 2024-09-21T02:03:20.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T02:03:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:20 smithi136 ceph-mon[34754]: pgmap v311: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:20.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:20 smithi092 ceph-mon[31190]: pgmap v311: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:20.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:20 smithi099 ceph-mon[34404]: pgmap v311: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:20.853 INFO:teuthology.orchestra.run.smithi092.stdout:227633266725 2024-09-21T02:03:20.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T02:03:21.312 INFO:teuthology.orchestra.run.smithi092.stdout:34359738445 2024-09-21T02:03:21.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T02:03:21.428 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.429 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.436 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.450 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.454 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 -- 172.21.15.92:0/4189140951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc103b80 msgr2=0x7f1dfc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 --2- 172.21.15.92:0/4189140951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc103b80 0x7f1dfc103f90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009900 tx=0x7f1dec02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 -- 172.21.15.92:0/4189140951 shutdown_connections 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 --2- 172.21.15.92:0/4189140951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dfc109e10 0x7f1dfc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 --2- 172.21.15.92:0/4189140951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1094c0 0x7f1dfc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 --2- 172.21.15.92:0/4189140951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc103b80 0x7f1dfc103f90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 -- 172.21.15.92:0/4189140951 >> 172.21.15.92:0/4189140951 conn(0x7f1dfc0ff5d0 msgr2=0x7f1dfc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 -- 172.21.15.92:0/4189140951 shutdown_connections 2024-09-21T02:03:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.201+0000 7f1e02259640 1 -- 172.21.15.92:0/4189140951 wait complete. 2024-09-21T02:03:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 Processor -- start 2024-09-21T02:03:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 -- start start 2024-09-21T02:03:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dfc103b80 0x7f1dfc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1094c0 0x7f1dfc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 0x7f1dfc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc113ad0 con 0x7f1dfc109e10 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc113950 con 0x7f1dfc1094c0 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.202+0000 7f1e02259640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc113c50 con 0x7f1dfc103b80 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 0x7f1dfc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 0x7f1dfc19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50540/0 (socket says 172.21.15.92:50540) 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 -- 172.21.15.92:0/4152157569 learned_addr learned my addr 172.21.15.92:0/4152157569 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfb7fe640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dfc103b80 0x7f1dfc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfaffd640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1094c0 0x7f1dfc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 -- 172.21.15.92:0/4152157569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dfc103b80 msgr2=0x7f1dfc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dfc103b80 0x7f1dfc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 -- 172.21.15.92:0/4152157569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1094c0 msgr2=0x7f1dfc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1094c0 0x7f1dfc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 -- 172.21.15.92:0/4152157569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc1a0410 con 0x7f1dfc109e10 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1dfbfff640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 0x7f1dfc19fce0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1df400eaa0 tx=0x7f1df400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:22.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400cdb0 con 0x7f1dfc109e10 2024-09-21T02:03:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.203+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc1a0700 con 0x7f1dfc109e10 2024-09-21T02:03:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.204+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4005020 con 0x7f1dfc109e10 2024-09-21T02:03:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.204+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df40116d0 con 0x7f1dfc109e10 2024-09-21T02:03:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.204+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1a6e60 con 0x7f1dfc109e10 2024-09-21T02:03:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.205+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc103f90 con 0x7f1dfc109e10 2024-09-21T02:03:22.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.206+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f1df40051d0 con 0x7f1dfc109e10 2024-09-21T02:03:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.207+0000 7f1df8ff9640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1de40b6620 0x7f1de40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.207+0000 7f1dfb7fe640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1de40b6620 0x7f1de40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.207+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1df40e7670 con 0x7f1dfc109e10 2024-09-21T02:03:22.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.208+0000 7f1dfb7fe640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1de40b6620 0x7f1de40b8b30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1dec031440 tx=0x7f1dec009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:22.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.212+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1df4016030 con 0x7f1dfc109e10 2024-09-21T02:03:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.396+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1dfc10bbe0 con 0x7f1dfc109e10 2024-09-21T02:03:22.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.397+0000 7f1df8ff9640 1 -- 172.21.15.92:0/4152157569 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1df40b2100 con 0x7f1dfc109e10 2024-09-21T02:03:22.398 INFO:teuthology.orchestra.run.smithi092.stdout:55834574922 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1de40b6620 msgr2=0x7f1de40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1de40b6620 0x7f1de40b8b30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1dec031440 tx=0x7f1dec009560 comp rx=0 tx=0).stop 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 msgr2=0x7f1dfc19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 0x7f1dfc19fce0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1df400eaa0 tx=0x7f1df400ef60 comp rx=0 tx=0).stop 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 shutdown_connections 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1de40b6620 0x7f1de40b8b30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1dfc109e10 0x7f1dfc19fce0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1094c0 0x7f1dfc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 --2- 172.21.15.92:0/4152157569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dfc103b80 0x7f1dfc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 >> 172.21.15.92:0/4152157569 conn(0x7f1dfc0ff5d0 msgr2=0x7f1dfc1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 shutdown_connections 2024-09-21T02:03:22.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.399+0000 7f1e02259640 1 -- 172.21.15.92:0/4152157569 wait complete. 2024-09-21T02:03:22.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:22 smithi092 ceph-mon[31190]: pgmap v312: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 212 B/s wr, 10 op/s; 7 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:22.450 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:22 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4152157569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:03:22.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 -- 172.21.15.92:0/1958778514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e0102a60 msgr2=0x7f80e0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 --2- 172.21.15.92:0/1958778514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e0102a60 0x7f80e0102e70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009900 tx=0x7f80dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 -- 172.21.15.92:0/1958778514 shutdown_connections 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 --2- 172.21.15.92:0/1958778514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e0108cf0 0x7f80e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 --2- 172.21.15.92:0/1958778514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80e01083a0 0x7f80e01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 --2- 172.21.15.92:0/1958778514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e0102a60 0x7f80e0102e70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 -- 172.21.15.92:0/1958778514 >> 172.21.15.92:0/1958778514 conn(0x7f80e0079730 msgr2=0x7f80e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 -- 172.21.15.92:0/1958778514 shutdown_connections 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.450+0000 7f80e862e640 1 -- 172.21.15.92:0/1958778514 wait complete. 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 Processor -- start 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 -- start start 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80e0102a60 0x7f80e019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 0x7f80e019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e0108cf0 0x7f80e019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0112960 con 0x7f80e01083a0 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80e01127e0 con 0x7f80e0108cf0 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e862e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0112ae0 con 0x7f80e0102a60 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 0x7f80e019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 0x7f80e019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50560/0 (socket says 172.21.15.92:50560) 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 -- 172.21.15.92:0/4011542106 learned_addr learned my addr 172.21.15.92:0/4011542106 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e6bcb640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e0108cf0 0x7f80e019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e63ca640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80e0102a60 0x7f80e019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 -- 172.21.15.92:0/4011542106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80e0102a60 msgr2=0x7f80e019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80e0102a60 0x7f80e019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 -- 172.21.15.92:0/4011542106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e0108cf0 msgr2=0x7f80e019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e0108cf0 0x7f80e019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 -- 172.21.15.92:0/4011542106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e01a03d0 con 0x7f80e01083a0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.451+0000 7f80e5bc9640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 0x7f80e019aa00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80d000d740 tx=0x7f80d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.452+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0017070 con 0x7f80e01083a0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.452+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80d000de50 con 0x7f80e01083a0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.452+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e01a06c0 con 0x7f80e01083a0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.452+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0004ce0 con 0x7f80e01083a0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.452+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e01a6e20 con 0x7f80e01083a0 2024-09-21T02:03:22.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.453+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e0102e70 con 0x7f80e01083a0 2024-09-21T02:03:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.454+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f80d0004e80 con 0x7f80e01083a0 2024-09-21T02:03:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.455+0000 7f80d77fe640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80cc0b6620 0x7f80cc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.455+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f80d00e71c0 con 0x7f80e01083a0 2024-09-21T02:03:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.455+0000 7f80e63ca640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80cc0b6620 0x7f80cc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:22.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.456+0000 7f80e63ca640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80cc0b6620 0x7f80cc0b8b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80dc031440 tx=0x7f80dc005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:22.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.458+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80d00ad950 con 0x7f80e01083a0 2024-09-21T02:03:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.641+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f80e01a7150 con 0x7f80e01083a0 2024-09-21T02:03:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.641+0000 7f80d77fe640 1 -- 172.21.15.92:0/4011542106 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f80d00b1d30 con 0x7f80e01083a0 2024-09-21T02:03:22.642 INFO:teuthology.orchestra.run.smithi092.stdout:296352743446 2024-09-21T02:03:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.643+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80cc0b6620 msgr2=0x7f80cc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.643+0000 7f80e862e640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80cc0b6620 0x7f80cc0b8b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f80dc031440 tx=0x7f80dc005d50 comp rx=0 tx=0).stop 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.643+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 msgr2=0x7f80e019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.643+0000 7f80e862e640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 0x7f80e019aa00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80d000d740 tx=0x7f80d000dc00 comp rx=0 tx=0).stop 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 shutdown_connections 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80cc0b6620 0x7f80cc0b8b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80e0108cf0 0x7f80e019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80e01083a0 0x7f80e019aa00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 --2- 172.21.15.92:0/4011542106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80e0102a60 0x7f80e019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 >> 172.21.15.92:0/4011542106 conn(0x7f80e0079730 msgr2=0x7f80e0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 shutdown_connections 2024-09-21T02:03:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:22.644+0000 7f80e862e640 1 -- 172.21.15.92:0/4011542106 wait complete. 2024-09-21T02:03:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:22 smithi136 ceph-mon[34754]: pgmap v312: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 212 B/s wr, 10 op/s; 7 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:22 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4152157569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:03:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:22 smithi099 ceph-mon[34404]: pgmap v312: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 212 B/s wr, 10 op/s; 7 B/s, 2 keys/s, 1 objects/s recovering 2024-09-21T02:03:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:22 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4152157569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:03:23.570 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743444 got 296352743446 for osd.11 2024-09-21T02:03:23.570 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:23.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:23 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4011542106' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:03:23.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:23 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4011542106' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:03:23.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:23 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4011542106' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:03:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 -- 172.21.15.92:0/3622173322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e8103b60 msgr2=0x7fd0e8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3622173322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e8103b60 0x7fd0e8103f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009900 tx=0x7fd0d802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:24.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 -- 172.21.15.92:0/3622173322 shutdown_connections 2024-09-21T02:03:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3622173322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8109df0 0x7fd0e8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3622173322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e81094a0 0x7fd0e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3622173322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e8103b60 0x7fd0e8103f70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.216+0000 7fd0edf79640 1 -- 172.21.15.92:0/3622173322 >> 172.21.15.92:0/3622173322 conn(0x7fd0e807b850 msgr2=0x7fd0e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.217+0000 7fd0edf79640 1 -- 172.21.15.92:0/3622173322 shutdown_connections 2024-09-21T02:03:24.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.217+0000 7fd0edf79640 1 -- 172.21.15.92:0/3622173322 wait complete. 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.217+0000 7fd0edf79640 1 Processor -- start 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.217+0000 7fd0edf79640 1 -- start start 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e8103b60 0x7fd0e819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 0x7fd0e819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8109df0 0x7fd0e819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8113ab0 con 0x7fd0e81094a0 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8113930 con 0x7fd0e8109df0 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e8113c30 con 0x7fd0e8103b60 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 0x7fd0e819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 0x7fd0e819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58528/0 (socket says 172.21.15.92:58528) 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 -- 172.21.15.92:0/3708331854 learned_addr learned my addr 172.21.15.92:0/3708331854 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e77fe640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e8103b60 0x7fd0e819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e7fff640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8109df0 0x7fd0e819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 -- 172.21.15.92:0/3708331854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e8103b60 msgr2=0x7fd0e819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e8103b60 0x7fd0e819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 -- 172.21.15.92:0/3708331854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8109df0 msgr2=0x7fd0e819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8109df0 0x7fd0e819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 -- 172.21.15.92:0/3708331854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e81a0310 con 0x7fd0e81094a0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e6ffd640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 0x7fd0e819aa00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00d720 tx=0x7fd0dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc009c30 con 0x7fd0e81094a0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0dc00de30 con 0x7fd0e81094a0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.218+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e81a0600 con 0x7fd0e81094a0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.219+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc010460 con 0x7fd0e81094a0 2024-09-21T02:03:24.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.219+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e81a6e20 con 0x7fd0e81094a0 2024-09-21T02:03:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.220+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e8103f70 con 0x7fd0e81094a0 2024-09-21T02:03:24.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.222+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd0dc004ea0 con 0x7fd0e81094a0 2024-09-21T02:03:24.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.222+0000 7fd0e4ff9640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd0d00b6650 0x7fd0d00b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.223+0000 7fd0e77fe640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd0d00b6650 0x7fd0d00b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.223+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd0dc0e7560 con 0x7fd0e81094a0 2024-09-21T02:03:24.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.223+0000 7fd0e77fe640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd0d00b6650 0x7fd0d00b8b60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8031440 tx=0x7fd0d8009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:24.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.226+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0dc0b20e0 con 0x7fd0e81094a0 2024-09-21T02:03:24.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.412+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd0e810bbf0 con 0x7fd0e81094a0 2024-09-21T02:03:24.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.412+0000 7fd0e4ff9640 1 -- 172.21.15.92:0/3708331854 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0dc0b6f90 con 0x7fd0e81094a0 2024-09-21T02:03:24.413 INFO:teuthology.orchestra.run.smithi092.stdout:249108103202 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd0d00b6650 msgr2=0x7fd0d00b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd0d00b6650 0x7fd0d00b8b60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8031440 tx=0x7fd0d8009560 comp rx=0 tx=0).stop 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 msgr2=0x7fd0e819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 0x7fd0e819aa00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00d720 tx=0x7fd0dc00dbe0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 shutdown_connections 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd0d00b6650 0x7fd0d00b8b60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8109df0 0x7fd0e819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0e81094a0 0x7fd0e819aa00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 --2- 172.21.15.92:0/3708331854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e8103b60 0x7fd0e819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 >> 172.21.15.92:0/3708331854 conn(0x7fd0e807b850 msgr2=0x7fd0e8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 shutdown_connections 2024-09-21T02:03:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.415+0000 7fd0edf79640 1 -- 172.21.15.92:0/3708331854 wait complete. 2024-09-21T02:03:24.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:24 smithi092 ceph-mon[31190]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 192 B/s wr, 10 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:03:24.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:24 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3708331854' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:03:24.733 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:24 smithi099 ceph-mon[34404]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 192 B/s wr, 10 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:03:24.733 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:24 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3708331854' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:03:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 -- 172.21.15.92:0/671311922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 msgr2=0x7f511010ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 --2- 172.21.15.92:0/671311922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511010ed60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5108009ca0 tx=0x7f510802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 -- 172.21.15.92:0/671311922 shutdown_connections 2024-09-21T02:03:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 --2- 172.21.15.92:0/671311922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511010ed60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 --2- 172.21.15.92:0/671311922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 0x7f5110107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 --2- 172.21.15.92:0/671311922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51101022a0 0x7f51101026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 -- 172.21.15.92:0/671311922 >> 172.21.15.92:0/671311922 conn(0x7f5110079ec0 msgr2=0x7f511007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 -- 172.21.15.92:0/671311922 shutdown_connections 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.741+0000 7f5115161640 1 -- 172.21.15.92:0/671311922 wait complete. 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 Processor -- start 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 -- start start 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51101022a0 0x7f511019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 0x7f511019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5110112200 con 0x7f5110108530 2024-09-21T02:03:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5110112080 con 0x7f5110107be0 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f5115161640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5110112380 con 0x7f51101022a0 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f510f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f510f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58576/0 (socket says 172.21.15.92:58576) 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.742+0000 7f510f59e640 1 -- 172.21.15.92:0/3302190984 learned_addr learned my addr 172.21.15.92:0/3302190984 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510ed9d640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51101022a0 0x7f511019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510e59c640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 0x7f511019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510f59e640 1 -- 172.21.15.92:0/3302190984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51101022a0 msgr2=0x7f511019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510f59e640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51101022a0 0x7f511019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510f59e640 1 -- 172.21.15.92:0/3302190984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 msgr2=0x7f511019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510f59e640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 0x7f511019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510f59e640 1 -- 172.21.15.92:0/3302190984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101a6570 con 0x7f5110108530 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510e59c640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 0x7f511019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f510f59e640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511019bd60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5108005cc0 tx=0x7f510802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108003bf0 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51101a6800 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5108003d90 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510802fbc0 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.743+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51101a6d40 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.744+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51101026b0 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.746+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f510802fd60 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.747+0000 7f50f7fff640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50f80b65d0 0x7f50f80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.747+0000 7f510ed9d640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50f80b65d0 0x7f50f80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.747+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5108108950 con 0x7f5110108530 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.747+0000 7f510ed9d640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50f80b65d0 0x7f50f80b8ae0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51101044d0 tx=0x7f510000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.750+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51080cf060 con 0x7f5110108530 2024-09-21T02:03:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:24 smithi136 ceph-mon[34754]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 192 B/s wr, 10 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:03:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:24 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3708331854' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:03:24.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.934+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f51101a7070 con 0x7f5110108530 2024-09-21T02:03:24.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.935+0000 7f50f7fff640 1 -- 172.21.15.92:0/3302190984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f51080d3440 con 0x7f5110108530 2024-09-21T02:03:24.936 INFO:teuthology.orchestra.run.smithi092.stdout:201863462955 2024-09-21T02:03:24.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50f80b65d0 msgr2=0x7f50f80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50f80b65d0 0x7f50f80b8ae0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51101044d0 tx=0x7f510000b390 comp rx=0 tx=0).stop 2024-09-21T02:03:24.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 msgr2=0x7f511019bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:24.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511019bd60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5108005cc0 tx=0x7f510802f380 comp rx=0 tx=0).stop 2024-09-21T02:03:24.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 shutdown_connections 2024-09-21T02:03:24.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50f80b65d0 0x7f50f80b8ae0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5110108530 0x7f511019bd60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5110107be0 0x7f511019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 --2- 172.21.15.92:0/3302190984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51101022a0 0x7f511019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.937+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 >> 172.21.15.92:0/3302190984 conn(0x7f5110079ec0 msgr2=0x7f5110100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.938+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 shutdown_connections 2024-09-21T02:03:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.938+0000 7f5115161640 1 -- 172.21.15.92:0/3302190984 wait complete. 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 -- 172.21.15.92:0/862746703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10102a60 msgr2=0x7f3c10102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 --2- 172.21.15.92:0/862746703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10102a60 0x7f3c10102e70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009ca0 tx=0x7f3c0002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 -- 172.21.15.92:0/862746703 shutdown_connections 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 --2- 172.21.15.92:0/862746703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c10108cf0 0x7f3c1010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 --2- 172.21.15.92:0/862746703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 0x7f3c101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 --2- 172.21.15.92:0/862746703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10102a60 0x7f3c10102e70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 -- 172.21.15.92:0/862746703 >> 172.21.15.92:0/862746703 conn(0x7f3c10079730 msgr2=0x7f3c10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 -- 172.21.15.92:0/862746703 shutdown_connections 2024-09-21T02:03:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.468+0000 7f3c1722d640 1 -- 172.21.15.92:0/862746703 wait complete. 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 Processor -- start 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 -- start start 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c10102a60 0x7f3c1019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 0x7f3c1019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 0x7f3c1019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10113ae0 con 0x7f3c10108cf0 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10113960 con 0x7f3c10102a60 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c1722d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10113c60 con 0x7f3c101083a0 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 0x7f3c1019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 0x7f3c1019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58546/0 (socket says 172.21.15.92:58546) 2024-09-21T02:03:25.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 -- 172.21.15.92:0/3586616503 learned_addr learned my addr 172.21.15.92:0/3586616503 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stdout:154618822710 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c14fc9640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c10102a60 0x7f3c1019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c0ffff640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 0x7f3c1019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 -- 172.21.15.92:0/3586616503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 msgr2=0x7f3c1019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 0x7f3c1019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 -- 172.21.15.92:0/3586616503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c10102a60 msgr2=0x7f3c1019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.469+0000 7f3c157ca640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c10102a60 0x7f3c1019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c157ca640 1 -- 172.21.15.92:0/3586616503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c101a04a0 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c0ffff640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 0x7f3c1019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c157ca640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 0x7f3c1019fd70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009fd0 tx=0x7f3c0800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08011830 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08016070 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08010de0 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c101a0790 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.470+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a6ef0 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.471+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c10102e70 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.472+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3c080119d0 con 0x7f3c10108cf0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.473+0000 7f3c0dffb640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3bfc0b65a0 0x7f3bfc0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.473+0000 7f3c14fc9640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3bfc0b65a0 0x7f3bfc0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.473+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3c080e6af0 con 0x7f3c10108cf0 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.474+0000 7f3c14fc9640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3bfc0b65a0 0x7f3bfc0b8ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009a60 tx=0x7f3c000093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.477+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c080ad200 con 0x7f3c10108cf0 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.666+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3c1010a530 con 0x7f3c10108cf0 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.666+0000 7f3c0dffb640 1 -- 172.21.15.92:0/3586616503 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3c080b15e0 con 0x7f3c10108cf0 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.668+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3bfc0b65a0 msgr2=0x7f3bfc0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.668+0000 7f3c1722d640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3bfc0b65a0 0x7f3bfc0b8ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009a60 tx=0x7f3c000093e0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 msgr2=0x7f3c1019fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 0x7f3c1019fd70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009fd0 tx=0x7f3c0800b8e0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 shutdown_connections 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3bfc0b65a0 0x7f3bfc0b8ab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c10108cf0 0x7f3c1019fd70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c101083a0 0x7f3c1019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 --2- 172.21.15.92:0/3586616503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c10102a60 0x7f3c1019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 >> 172.21.15.92:0/3586616503 conn(0x7f3c10079730 msgr2=0x7f3c101013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:25.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 shutdown_connections 2024-09-21T02:03:25.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:24.669+0000 7f3c1722d640 1 -- 172.21.15.92:0/3586616503 wait complete. 2024-09-21T02:03:25.621 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3586616503' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:03:25.622 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:25 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3302190984' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 -- 172.21.15.92:0/2519429137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8102ae0 msgr2=0x7f53d8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 --2- 172.21.15.92:0/2519429137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8102ae0 0x7f53d8102ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009900 tx=0x7f53c802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 -- 172.21.15.92:0/2519429137 shutdown_connections 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 --2- 172.21.15.92:0/2519429137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8108d70 0x7f53d810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 --2- 172.21.15.92:0/2519429137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53d8108420 0x7f53d8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 --2- 172.21.15.92:0/2519429137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8102ae0 0x7f53d8102ef0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 -- 172.21.15.92:0/2519429137 >> 172.21.15.92:0/2519429137 conn(0x7f53d8079770 msgr2=0x7f53d8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 -- 172.21.15.92:0/2519429137 shutdown_connections 2024-09-21T02:03:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.636+0000 7f53dea19640 1 -- 172.21.15.92:0/2519429137 wait complete. 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 Processor -- start 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 -- start start 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53d8102ae0 0x7f53d819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8108420 0x7f53d819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 0x7f53d819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8112a70 con 0x7f53d8108d70 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81128f0 con 0x7f53d8108420 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dea19640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8112bf0 con 0x7f53d8102ae0 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dcfb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 0x7f53d819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dcfb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 0x7f53d819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58590/0 (socket says 172.21.15.92:58590) 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.637+0000 7f53dcfb6640 1 -- 172.21.15.92:0/594439752 learned_addr learned my addr 172.21.15.92:0/594439752 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53cffff640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8108420 0x7f53d819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dcfb6640 1 -- 172.21.15.92:0/594439752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53d8102ae0 msgr2=0x7f53d819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dcfb6640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53d8102ae0 0x7f53d819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dcfb6640 1 -- 172.21.15.92:0/594439752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8108420 msgr2=0x7f53d819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dcfb6640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8108420 0x7f53d819b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dcfb6640 1 -- 172.21.15.92:0/594439752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d81a65f0 con 0x7f53d8108d70 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dcfb6640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 0x7f53d819bde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f53d000eaa0 tx=0x7f53d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d000cdb0 con 0x7f53d8108d70 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d0005020 con 0x7f53d8108d70 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d81a68e0 con 0x7f53d8108d70 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00116d0 con 0x7f53d8108d70 2024-09-21T02:03:25.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.638+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d81a6e20 con 0x7f53d8108d70 2024-09-21T02:03:25.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.639+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d8103d60 con 0x7f53d8108d70 2024-09-21T02:03:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.641+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f53d00051d0 con 0x7f53d8108d70 2024-09-21T02:03:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.642+0000 7f53d5ffb640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53c40b6650 0x7f53c40b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.642+0000 7f53d7fff640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53c40b6650 0x7f53c40b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.642+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f53d00e7a90 con 0x7f53d8108d70 2024-09-21T02:03:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.643+0000 7f53d7fff640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53c40b6650 0x7f53c40b8b60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f53c802a930 tx=0x7f53c8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.645+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53d0016030 con 0x7f53d8108d70 2024-09-21T02:03:25.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3586616503' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:03:25.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:25 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3302190984' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:03:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 -- 172.21.15.92:0/1354076435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da0105300 msgr2=0x7f2da010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 --2- 172.21.15.92:0/1354076435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da0105300 0x7f2da010d850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d8802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 -- 172.21.15.92:0/1354076435 shutdown_connections 2024-09-21T02:03:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 --2- 172.21.15.92:0/1354076435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2da010de10 0x7f2da0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 --2- 172.21.15.92:0/1354076435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da0105300 0x7f2da010d850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 --2- 172.21.15.92:0/1354076435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da01049b0 0x7f2da0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 -- 172.21.15.92:0/1354076435 >> 172.21.15.92:0/1354076435 conn(0x7f2da0079730 msgr2=0x7f2da0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 -- 172.21.15.92:0/1354076435 shutdown_connections 2024-09-21T02:03:25.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.820+0000 7f2da5316640 1 -- 172.21.15.92:0/1354076435 wait complete. 2024-09-21T02:03:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.821+0000 7f2da5316640 1 Processor -- start 2024-09-21T02:03:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.821+0000 7f2da5316640 1 -- start start 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 0x7f2da019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2da0105300 0x7f2da019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da010de10 0x7f2da019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0113a70 con 0x7f2da01049b0 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01138f0 con 0x7f2da010de10 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0113bf0 con 0x7f2da0105300 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 0x7f2da019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 0x7f2da019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58602/0 (socket says 172.21.15.92:58602) 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 -- 172.21.15.92:0/975675375 learned_addr learned my addr 172.21.15.92:0/975675375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9e7fc640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2da0105300 0x7f2da019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9f7fe640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da010de10 0x7f2da019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 -- 172.21.15.92:0/975675375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2da0105300 msgr2=0x7f2da019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2da0105300 0x7f2da019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 -- 172.21.15.92:0/975675375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da010de10 msgr2=0x7f2da019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da010de10 0x7f2da019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 -- 172.21.15.92:0/975675375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da01a03d0 con 0x7f2da01049b0 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9f7fe640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da010de10 0x7f2da019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d9effd640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 0x7f2da019a460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c970 tx=0x7f2d9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90004220 con 0x7f2da01049b0 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d900043c0 con 0x7f2da01049b0 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.822+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da01a06c0 con 0x7f2da01049b0 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.823+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000f450 con 0x7f2da01049b0 2024-09-21T02:03:25.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.823+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01a6e20 con 0x7f2da01049b0 2024-09-21T02:03:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.824+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da006b560 con 0x7f2da01049b0 2024-09-21T02:03:25.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.825+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2d90003610 con 0x7f2da01049b0 2024-09-21T02:03:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.826+0000 7f2d97fff640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2d8c0b65a0 0x7f2d8c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.826+0000 7f2d9e7fc640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2d8c0b65a0 0x7f2d8c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.826+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2d900e7340 con 0x7f2da01049b0 2024-09-21T02:03:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.827+0000 7f2d9e7fc640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2d8c0b65a0 0x7f2d8c0b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009230 tx=0x7f2d88031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:25.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3586616503' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:03:25.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:25 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3302190984' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:03:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.831+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2d90014030 con 0x7f2da01049b0 2024-09-21T02:03:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.837+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f53d8102ef0 con 0x7f53d8108d70 2024-09-21T02:03:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.837+0000 7f53d5ffb640 1 -- 172.21.15.92:0/594439752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f53d00ae220 con 0x7f53d8108d70 2024-09-21T02:03:25.838 INFO:teuthology.orchestra.run.smithi092.stdout:274877906973 2024-09-21T02:03:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53c40b6650 msgr2=0x7f53c40b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53c40b6650 0x7f53c40b8b60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f53c802a930 tx=0x7f53c8031400 comp rx=0 tx=0).stop 2024-09-21T02:03:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 msgr2=0x7f53d819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:25.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 0x7f53d819bde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f53d000eaa0 tx=0x7f53d000ef60 comp rx=0 tx=0).stop 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 shutdown_connections 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53c40b6650 0x7f53c40b8b60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53d8108d70 0x7f53d819bde0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8108420 0x7f53d819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 --2- 172.21.15.92:0/594439752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53d8102ae0 0x7f53d819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 >> 172.21.15.92:0/594439752 conn(0x7f53d8079770 msgr2=0x7f53d81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 shutdown_connections 2024-09-21T02:03:25.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:25.840+0000 7f53dea19640 1 -- 172.21.15.92:0/594439752 wait complete. 2024-09-21T02:03:26.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.024+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2da0108980 con 0x7f2da01049b0 2024-09-21T02:03:26.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.024+0000 7f2d97fff640 1 -- 172.21.15.92:0/975675375 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2d900b6030 con 0x7f2da01049b0 2024-09-21T02:03:26.026 INFO:teuthology.orchestra.run.smithi092.stdout:128849018940 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2d8c0b65a0 msgr2=0x7f2d8c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2d8c0b65a0 0x7f2d8c0b8ab0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009230 tx=0x7f2d88031400 comp rx=0 tx=0).stop 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 msgr2=0x7f2da019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 0x7f2da019a460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c970 tx=0x7f2d9000ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 shutdown_connections 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2d8c0b65a0 0x7f2d8c0b8ab0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2da010de10 0x7f2da019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2da0105300 0x7f2da019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 --2- 172.21.15.92:0/975675375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da01049b0 0x7f2da019a460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.027+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 >> 172.21.15.92:0/975675375 conn(0x7f2da0079730 msgr2=0x7f2da0100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:26.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.028+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 shutdown_connections 2024-09-21T02:03:26.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.028+0000 7f2da5316640 1 -- 172.21.15.92:0/975675375 wait complete. 2024-09-21T02:03:26.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:26 smithi092 ceph-mon[31190]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:03:26.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:26 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/594439752' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:03:26.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:26 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/975675375' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:03:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:26 smithi136 ceph-mon[34754]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:03:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:26 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/594439752' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:03:26.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:26 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/975675375' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 -- 172.21.15.92:0/1906926196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 msgr2=0x7f13bc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/1906926196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc108830 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009900 tx=0x7f13a402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 -- 172.21.15.92:0/1906926196 shutdown_connections 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/1906926196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 0x7f13bc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/1906926196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc108830 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/1906926196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13bc102ae0 0x7f13bc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 -- 172.21.15.92:0/1906926196 >> 172.21.15.92:0/1906926196 conn(0x7f13bc079770 msgr2=0x7f13bc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 -- 172.21.15.92:0/1906926196 shutdown_connections 2024-09-21T02:03:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 -- 172.21.15.92:0/1906926196 wait complete. 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 Processor -- start 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.820+0000 7f13c1bcd640 1 -- start start 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13c1bcd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13bc102ae0 0x7f13bc1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13c1bcd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13c1bcd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 0x7f13bc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13c1bcd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112a50 con 0x7f13bc108420 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13c1bcd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1128d0 con 0x7f13bc102ae0 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13c1bcd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112bd0 con 0x7f13bc108d70 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58628/0 (socket says 172.21.15.92:58628) 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 -- 172.21.15.92:0/3007138921 learned_addr learned my addr 172.21.15.92:0/3007138921 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13bb7fe640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13bc102ae0 0x7f13bc1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13bbfff640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 0x7f13bc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 -- 172.21.15.92:0/3007138921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 msgr2=0x7f13bc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 0x7f13bc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 -- 172.21.15.92:0/3007138921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13bc102ae0 msgr2=0x7f13bc1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13bc102ae0 0x7f13bc1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 -- 172.21.15.92:0/3007138921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc19f350 con 0x7f13bc108420 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13bbfff640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 0x7f13bc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.821+0000 7f13b3fff640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc199a40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13a402a930 tx=0x7f13a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.822+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a403f030 con 0x7f13bc108420 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.822+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4003aa0 con 0x7f13bc108420 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.822+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc19f5e0 con 0x7f13bc108420 2024-09-21T02:03:26.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.822+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4035070 con 0x7f13bc108420 2024-09-21T02:03:26.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.822+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc19fac0 con 0x7f13bc108420 2024-09-21T02:03:26.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.823+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc103d60 con 0x7f13bc108420 2024-09-21T02:03:26.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.824+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f13a4042030 con 0x7f13bc108420 2024-09-21T02:03:26.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.825+0000 7f13b97fa640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f13a80b65d0 0x7f13a80b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:26.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.825+0000 7f13bb7fe640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f13a80b65d0 0x7f13a80b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:26.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.825+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f13a41080f0 con 0x7f13bc108420 2024-09-21T02:03:26.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.825+0000 7f13bb7fe640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f13a80b65d0 0x7f13a80b8ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00abe0 tx=0x7f13ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:26.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:26.828+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13a40ce800 con 0x7f13bc108420 2024-09-21T02:03:26.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:26 smithi099 ceph-mon[34404]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:03:26.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:26 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/594439752' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:03:26.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:26 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/975675375' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:03:26.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018938 got 128849018940 for osd.4 2024-09-21T02:03:26.966 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:26.978 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:27.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.022+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f13bc102ef0 con 0x7f13bc108420 2024-09-21T02:03:27.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.023+0000 7f13b97fa640 1 -- 172.21.15.92:0/3007138921 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f13a40d2be0 con 0x7f13bc108420 2024-09-21T02:03:27.024 INFO:teuthology.orchestra.run.smithi092.stdout:227633266727 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f13a80b65d0 msgr2=0x7f13a80b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f13a80b65d0 0x7f13a80b8ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00abe0 tx=0x7f13ac00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 msgr2=0x7f13bc199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc199a40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13a402a930 tx=0x7f13a4004270 comp rx=0 tx=0).stop 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 shutdown_connections 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f13a80b65d0 0x7f13a80b8ae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13bc108d70 0x7f13bc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc108420 0x7f13bc199a40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 --2- 172.21.15.92:0/3007138921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13bc102ae0 0x7f13bc1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:27.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 >> 172.21.15.92:0/3007138921 conn(0x7f13bc079770 msgr2=0x7f13bc106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:27.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 shutdown_connections 2024-09-21T02:03:27.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:27.025+0000 7f13c1bcd640 1 -- 172.21.15.92:0/3007138921 wait complete. 2024-09-21T02:03:27.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:27 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3007138921' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:03:27.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:27 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:03:27.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:27 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3007138921' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:03:27.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:27 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:03:27.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:27 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3007138921' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:03:27.789 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:27 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:03:28.125 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462954 got 201863462955 for osd.7 2024-09-21T02:03:28.125 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:28.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822708 got 154618822710 for osd.5 2024-09-21T02:03:28.546 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:28.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:28 smithi092 ceph-mon[31190]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T02:03:28.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:28 smithi136 ceph-mon[34754]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T02:03:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:28 smithi099 ceph-mon[34404]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T02:03:29.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574922 for osd.1 2024-09-21T02:03:29.064 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:29.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 -- 172.21.15.92:0/722511898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c1094c0 msgr2=0x7f4b6c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:29.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 --2- 172.21.15.92:0/722511898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c1094c0 0x7f4b6c1098d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009900 tx=0x7f4b6002a410 comp rx=0 tx=0).stop 2024-09-21T02:03:29.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 -- 172.21.15.92:0/722511898 shutdown_connections 2024-09-21T02:03:29.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 --2- 172.21.15.92:0/722511898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c109e10 0x7f4b6c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 --2- 172.21.15.92:0/722511898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c1094c0 0x7f4b6c1098d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 --2- 172.21.15.92:0/722511898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6c103b80 0x7f4b6c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 -- 172.21.15.92:0/722511898 >> 172.21.15.92:0/722511898 conn(0x7f4b6c0ff610 msgr2=0x7f4b6c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 -- 172.21.15.92:0/722511898 shutdown_connections 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.599+0000 7f4b71325640 1 -- 172.21.15.92:0/722511898 wait complete. 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 Processor -- start 2024-09-21T02:03:29.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 -- start start 2024-09-21T02:03:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c103b80 0x7f4b6c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6c1094c0 0x7f4b6c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 0x7f4b6c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c113af0 con 0x7f4b6c109e10 2024-09-21T02:03:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c113970 con 0x7f4b6c1094c0 2024-09-21T02:03:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b71325640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c113c70 con 0x7f4b6c103b80 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 0x7f4b6c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 0x7f4b6c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58644/0 (socket says 172.21.15.92:58644) 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b6b7fe640 1 -- 172.21.15.92:0/2076239471 learned_addr learned my addr 172.21.15.92:0/2076239471 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b6affd640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c103b80 0x7f4b6c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.600+0000 7f4b6a7fc640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6c1094c0 0x7f4b6c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6b7fe640 1 -- 172.21.15.92:0/2076239471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c103b80 msgr2=0x7f4b6c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6b7fe640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c103b80 0x7f4b6c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6b7fe640 1 -- 172.21.15.92:0/2076239471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6c1094c0 msgr2=0x7f4b6c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6b7fe640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6c1094c0 0x7f4b6c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6b7fe640 1 -- 172.21.15.92:0/2076239471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1a0470 con 0x7f4b6c109e10 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6affd640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c103b80 0x7f4b6c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:29.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.601+0000 7f4b6b7fe640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 0x7f4b6c19fd40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009fd0 tx=0x7f4b6400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:29.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.604+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64011830 con 0x7f4b6c109e10 2024-09-21T02:03:29.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.604+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64016070 con 0x7f4b6c109e10 2024-09-21T02:03:29.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.604+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64010de0 con 0x7f4b6c109e10 2024-09-21T02:03:29.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.604+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1a0760 con 0x7f4b6c109e10 2024-09-21T02:03:29.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.604+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a6ec0 con 0x7f4b6c109e10 2024-09-21T02:03:29.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.605+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c103f90 con 0x7f4b6c109e10 2024-09-21T02:03:29.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.606+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4b640119d0 con 0x7f4b6c109e10 2024-09-21T02:03:29.607 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:29.607 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:29.607 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:29.607 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:29 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:29.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.607+0000 7f4b53fff640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4b540b6620 0x7f4b540b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:29.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.607+0000 7f4b6affd640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4b540b6620 0x7f4b540b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:29.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.607+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4b640e6fd0 con 0x7f4b6c109e10 2024-09-21T02:03:29.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.608+0000 7f4b6affd640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4b540b6620 0x7f4b540b8b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009fd0 tx=0x7f4b5c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.610+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b640ad780 con 0x7f4b6c109e10 2024-09-21T02:03:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.795+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4b6c19b630 con 0x7f4b6c109e10 2024-09-21T02:03:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.796+0000 7f4b53fff640 1 -- 172.21.15.92:0/2076239471 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4b6c19b630 con 0x7f4b6c109e10 2024-09-21T02:03:29.796 INFO:teuthology.orchestra.run.smithi092.stdout:107374182465 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4b540b6620 msgr2=0x7f4b540b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4b540b6620 0x7f4b540b8b30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009fd0 tx=0x7f4b5c009680 comp rx=0 tx=0).stop 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 msgr2=0x7f4b6c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 0x7f4b6c19fd40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009fd0 tx=0x7f4b6400b8e0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 shutdown_connections 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4b540b6620 0x7f4b540b8b30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c109e10 0x7f4b6c19fd40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6c1094c0 0x7f4b6c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 --2- 172.21.15.92:0/2076239471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b6c103b80 0x7f4b6c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 >> 172.21.15.92:0/2076239471 conn(0x7f4b6c0ff610 msgr2=0x7f4b6c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.798+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 shutdown_connections 2024-09-21T02:03:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:29.799+0000 7f4b71325640 1 -- 172.21.15.92:0/2076239471 wait complete. 2024-09-21T02:03:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:29 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:29 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:30.132 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906971 got 274877906973 for osd.10 2024-09-21T02:03:30.132 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:30.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103202 for osd.9 2024-09-21T02:03:30.534 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:30 smithi092 ceph-mon[31190]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T02:03:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:30 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2076239471' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:03:31.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:30 smithi136 ceph-mon[34754]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T02:03:31.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:30 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2076239471' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:03:31.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:30 smithi099 ceph-mon[34404]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T02:03:31.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:30 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2076239471' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:03:31.113 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182465 for osd.3 2024-09-21T02:03:31.114 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 -- 172.21.15.92:0/2041380327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec102030 msgr2=0x7f6cec102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2041380327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec102030 0x7f6cec102440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009900 tx=0x7f6cdc02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 -- 172.21.15.92:0/2041380327 shutdown_connections 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2041380327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cec106bd0 0x7f6cec110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2041380327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cec106280 0x7f6cec106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2041380327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec102030 0x7f6cec102440 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 -- 172.21.15.92:0/2041380327 >> 172.21.15.92:0/2041380327 conn(0x7f6cec077c60 msgr2=0x7f6cec0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:31.597 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 -- 172.21.15.92:0/2041380327 shutdown_connections 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.596+0000 7f6cf2710640 1 -- 172.21.15.92:0/2041380327 wait complete. 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.597+0000 7f6cf2710640 1 Processor -- start 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.597+0000 7f6cf2710640 1 -- start start 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf2710640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cec102030 0x7f6cec074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf2710640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 0x7f6cec06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf2710640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cec106bd0 0x7f6cec06f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf2710640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec06d3d0 con 0x7f6cec106280 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf2710640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec06d250 con 0x7f6cec106bd0 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf2710640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec06d550 con 0x7f6cec102030 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 0x7f6cec06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 0x7f6cec06ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58664/0 (socket says 172.21.15.92:58664) 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 -- 172.21.15.92:0/2564582192 learned_addr learned my addr 172.21.15.92:0/2564582192 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cebfff640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cec102030 0x7f6cec074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6cf0cad640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cec106bd0 0x7f6cec06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 -- 172.21.15.92:0/2564582192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cec102030 msgr2=0x7f6cec074c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cec102030 0x7f6cec074c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 -- 172.21.15.92:0/2564582192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cec106bd0 msgr2=0x7f6cec06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cec106bd0 0x7f6cec06f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.598+0000 7f6ceb7fe640 1 -- 172.21.15.92:0/2564582192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec06faf0 con 0x7f6cec106280 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.599+0000 7f6ceb7fe640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 0x7f6cec06ef40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000c970 tx=0x7f6ce000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.599+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0004220 con 0x7f6cec106280 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.599+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec102eb0 con 0x7f6cec106280 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.599+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce00043c0 con 0x7f6cec106280 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.599+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0004a00 con 0x7f6cec106280 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.599+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec1033c0 con 0x7f6cec106280 2024-09-21T02:03:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.600+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec102440 con 0x7f6cec106280 2024-09-21T02:03:31.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.602+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6ce00079b0 con 0x7f6cec106280 2024-09-21T02:03:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.603+0000 7f6ce97fa640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cd40b65d0 0x7f6cd40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.603+0000 7f6cebfff640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cd40b65d0 0x7f6cd40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.603+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6ce00e71c0 con 0x7f6cec106280 2024-09-21T02:03:31.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.604+0000 7f6cebfff640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cd40b65d0 0x7f6cd40b8ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc004770 tx=0x7f6cdc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.607+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ce0014030 con 0x7f6cec106280 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 -- 172.21.15.92:0/4062861125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7184105300 msgr2=0x7f718410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 --2- 172.21.15.92:0/4062861125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7184105300 0x7f718410d850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7174009900 tx=0x7f717402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 -- 172.21.15.92:0/4062861125 shutdown_connections 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 --2- 172.21.15.92:0/4062861125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718410de10 0x7f7184110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 --2- 172.21.15.92:0/4062861125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7184105300 0x7f718410d850 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 --2- 172.21.15.92:0/4062861125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71841049b0 0x7f7184104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 -- 172.21.15.92:0/4062861125 >> 172.21.15.92:0/4062861125 conn(0x7f7184079730 msgr2=0x7f7184079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 -- 172.21.15.92:0/4062861125 shutdown_connections 2024-09-21T02:03:31.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.789+0000 7f718c7b5640 1 -- 172.21.15.92:0/4062861125 wait complete. 2024-09-21T02:03:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.790+0000 7f718c7b5640 1 Processor -- start 2024-09-21T02:03:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.790+0000 7f718c7b5640 1 -- start start 2024-09-21T02:03:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718c7b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 0x7f71841026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718c7b5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7184105300 0x7f7184102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718c7b5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f718410de10 0x7f7184103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718c7b5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7184113a70 con 0x7f71841049b0 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718c7b5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71841138f0 con 0x7f7184105300 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718c7b5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7184113bf0 con 0x7f718410de10 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 0x7f71841026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 0x7f71841026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58692/0 (socket says 172.21.15.92:58692) 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 -- 172.21.15.92:0/2362367127 learned_addr learned my addr 172.21.15.92:0/2362367127 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f7189d50640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7184105300 0x7f7184102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718ad52640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f718410de10 0x7f7184103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 -- 172.21.15.92:0/2362367127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f718410de10 msgr2=0x7f7184103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f718410de10 0x7f7184103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 -- 172.21.15.92:0/2362367127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7184105300 msgr2=0x7f7184102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7184105300 0x7f7184102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 -- 172.21.15.92:0/2362367127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71841a8760 con 0x7f71841049b0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718ad52640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f718410de10 0x7f7184103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f7189d50640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7184105300 0x7f7184102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f718a551640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 0x7f71841026f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f718000c970 tx=0x7f718000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.791+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180007bb0 con 0x7f71841049b0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.792+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180007d50 con 0x7f71841049b0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.792+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180018780 con 0x7f71841049b0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.792+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71841a8a50 con 0x7f71841049b0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.792+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71841a8f00 con 0x7f71841049b0 2024-09-21T02:03:31.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.793+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7184104dc0 con 0x7f71841049b0 2024-09-21T02:03:31.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.794+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f718000fa20 con 0x7f71841049b0 2024-09-21T02:03:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.795+0000 7f717b7fe640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71700b6620 0x7f71700b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.796+0000 7f7189d50640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71700b6620 0x7f71700b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:31.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.796+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f71800e7600 con 0x7f71841049b0 2024-09-21T02:03:31.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.796+0000 7f7189d50640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71700b6620 0x7f71700b8b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7174009230 tx=0x7f7174000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.799+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7180014030 con 0x7f71841049b0 2024-09-21T02:03:31.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.801+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6cec10bc60 con 0x7f6cec106280 2024-09-21T02:03:31.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.801+0000 7f6ce97fa640 1 -- 172.21.15.92:0/2564582192 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6ce00b1cb0 con 0x7f6cec106280 2024-09-21T02:03:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:176093659186 2024-09-21T02:03:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.803+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cd40b65d0 msgr2=0x7f6cd40b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.803+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cd40b65d0 0x7f6cd40b8ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc004770 tx=0x7f6cdc0058d0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.803+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 msgr2=0x7f6cec06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.803+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 0x7f6cec06ef40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000c970 tx=0x7f6ce000ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 shutdown_connections 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cd40b65d0 0x7f6cd40b8ae0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cec106bd0 0x7f6cec06f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cec106280 0x7f6cec06ef40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 --2- 172.21.15.92:0/2564582192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cec102030 0x7f6cec074c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 >> 172.21.15.92:0/2564582192 conn(0x7f6cec077c60 msgr2=0x7f6cec107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 shutdown_connections 2024-09-21T02:03:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.804+0000 7f6cf2710640 1 -- 172.21.15.92:0/2564582192 wait complete. 2024-09-21T02:03:31.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.984+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f71841039f0 con 0x7f71841049b0 2024-09-21T02:03:31.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.985+0000 7f717b7fe640 1 -- 172.21.15.92:0/2362367127 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71841039f0 con 0x7f71841049b0 2024-09-21T02:03:31.985 INFO:teuthology.orchestra.run.smithi092.stdout:73014444101 2024-09-21T02:03:31.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71700b6620 msgr2=0x7f71700b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71700b6620 0x7f71700b8b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7174009230 tx=0x7f7174000bc0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 msgr2=0x7f71841026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 0x7f71841026f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f718000c970 tx=0x7f718000ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 shutdown_connections 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71700b6620 0x7f71700b8b30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f718410de10 0x7f7184103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7184105300 0x7f7184102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 --2- 172.21.15.92:0/2362367127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71841049b0 0x7f71841026f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 >> 172.21.15.92:0/2362367127 conn(0x7f7184079730 msgr2=0x7f7184100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 shutdown_connections 2024-09-21T02:03:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:31.987+0000 7f718c7b5640 1 -- 172.21.15.92:0/2362367127 wait complete. 2024-09-21T02:03:32.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 -- 172.21.15.92:0/2113207664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 msgr2=0x7f3988103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 --2- 172.21.15.92:0/2113207664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f3988103680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3984009900 tx=0x7f398402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 -- 172.21.15.92:0/2113207664 shutdown_connections 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 --2- 172.21.15.92:0/2113207664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3988105b90 0x7f398810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 --2- 172.21.15.92:0/2113207664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 0x7f3988105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 --2- 172.21.15.92:0/2113207664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f3988103680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 -- 172.21.15.92:0/2113207664 >> 172.21.15.92:0/2113207664 conn(0x7f3988079ec0 msgr2=0x7f398807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 -- 172.21.15.92:0/2113207664 shutdown_connections 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.351+0000 7f3990407640 1 -- 172.21.15.92:0/2113207664 wait complete. 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 Processor -- start 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 -- start start 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f398819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 0x7f398819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3988105b90 0x7f398819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113c10 con 0x7f3988103270 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113a90 con 0x7f3988105240 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.352+0000 7f3990407640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113d90 con 0x7f3988105b90 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f398819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f398819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58700/0 (socket says 172.21.15.92:58700) 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 -- 172.21.15.92:0/3291010666 learned_addr learned my addr 172.21.15.92:0/3291010666 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e9a4640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3988105b90 0x7f398819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398d9a2640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 0x7f398819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 -- 172.21.15.92:0/3291010666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3988105b90 msgr2=0x7f398819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3988105b90 0x7f398819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 -- 172.21.15.92:0/3291010666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 msgr2=0x7f398819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 0x7f398819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 -- 172.21.15.92:0/3291010666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39881a05b0 con 0x7f3988103270 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398d9a2640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 0x7f398819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.353+0000 7f398e1a3640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f398819a640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3984006010 tx=0x7f39840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.354+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984035070 con 0x7f3988103270 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.354+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3984031d10 con 0x7f3988103270 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.354+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984003af0 con 0x7f3988103270 2024-09-21T02:03:32.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.354+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39881a0840 con 0x7f3988103270 2024-09-21T02:03:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.354+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398810ffa0 con 0x7f3988103270 2024-09-21T02:03:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.355+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398806b910 con 0x7f3988103270 2024-09-21T02:03:32.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.356+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f3984003c90 con 0x7f3988103270 2024-09-21T02:03:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.357+0000 7f397b7fe640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39740b6620 0x7f39740b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.357+0000 7f398d9a2640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39740b6620 0x7f39740b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.357+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f398410c1e0 con 0x7f3988103270 2024-09-21T02:03:32.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.358+0000 7f398d9a2640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39740b6620 0x7f39740b8b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f398807a8f0 tx=0x7f397c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.361+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39840d28f0 con 0x7f3988103270 2024-09-21T02:03:32.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.541+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f398810ab10 con 0x7f3988103270 2024-09-21T02:03:32.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.542+0000 7f397b7fe640 1 -- 172.21.15.92:0/3291010666 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39840d6cd0 con 0x7f3988103270 2024-09-21T02:03:32.542 INFO:teuthology.orchestra.run.smithi092.stdout:34359738447 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39740b6620 msgr2=0x7f39740b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39740b6620 0x7f39740b8b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f398807a8f0 tx=0x7f397c008040 comp rx=0 tx=0).stop 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 msgr2=0x7f398819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f398819a640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3984006010 tx=0x7f39840042f0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 shutdown_connections 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39740b6620 0x7f39740b8b30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3988105b90 0x7f398819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3988105240 0x7f398819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 --2- 172.21.15.92:0/3291010666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988103270 0x7f398819a640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 >> 172.21.15.92:0/3291010666 conn(0x7f3988079ec0 msgr2=0x7f3988107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 shutdown_connections 2024-09-21T02:03:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:32.544+0000 7f3990407640 1 -- 172.21.15.92:0/3291010666 wait complete. 2024-09-21T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:32 smithi092 ceph-mon[31190]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:32 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2564582192' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:32 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2362367127' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:03:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:32 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3291010666' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:03:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:32 smithi136 ceph-mon[34754]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:32 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2564582192' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:03:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:32 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2362367127' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:03:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:32 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3291010666' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:03:33.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:32 smithi099 ceph-mon[34404]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:33.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:32 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2564582192' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:03:33.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:32 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2362367127' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:03:33.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:32 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3291010666' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:03:33.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659182 got 176093659186 for osd.6 2024-09-21T02:03:33.268 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:33.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266725 got 227633266727 for osd.8 2024-09-21T02:03:33.646 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:34.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444096 got 73014444101 for osd.2 2024-09-21T02:03:34.128 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:34.529 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-09-21T02:03:34.529 DEBUG:teuthology.parallel:result is None 2024-09-21T02:03:34.529 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T02:03:34.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:03:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:34 smithi092 ceph-mon[31190]: pgmap v318: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:35.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:34 smithi136 ceph-mon[34754]: pgmap v318: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:35.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:34 smithi099 ceph-mon[34404]: pgmap v318: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:35.291 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 -- 172.21.15.92:0/861355604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a501094a0 msgr2=0x7f2a501098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 --2- 172.21.15.92:0/861355604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a501094a0 0x7f2a501098b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009900 tx=0x7f2a4402a410 comp rx=0 tx=0).stop 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 -- 172.21.15.92:0/861355604 shutdown_connections 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 --2- 172.21.15.92:0/861355604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a50109df0 0x7f2a50110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 --2- 172.21.15.92:0/861355604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a501094a0 0x7f2a501098b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 --2- 172.21.15.92:0/861355604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50103b60 0x7f2a50103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 -- 172.21.15.92:0/861355604 >> 172.21.15.92:0/861355604 conn(0x7f2a5007b850 msgr2=0x7f2a5007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 -- 172.21.15.92:0/861355604 shutdown_connections 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.774+0000 7f2a55de8640 1 -- 172.21.15.92:0/861355604 wait complete. 2024-09-21T02:03:35.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 Processor -- start 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 -- start start 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 0x7f2a5019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a501094a0 0x7f2a5019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50109df0 0x7f2a5019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50113ad0 con 0x7f2a50103b60 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50113950 con 0x7f2a501094a0 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a55de8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50113c50 con 0x7f2a50109df0 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 0x7f2a5019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 0x7f2a5019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44358/0 (socket says 172.21.15.92:44358) 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 -- 172.21.15.92:0/276733838 learned_addr learned my addr 172.21.15.92:0/276733838 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4ffff640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50109df0 0x7f2a5019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 -- 172.21.15.92:0/276733838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50109df0 msgr2=0x7f2a5019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50109df0 0x7f2a5019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 -- 172.21.15.92:0/276733838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a501094a0 msgr2=0x7f2a5019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4effd640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a501094a0 0x7f2a5019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a501094a0 0x7f2a5019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4f7fe640 1 -- 172.21.15.92:0/276733838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a501a04b0 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.775+0000 7f2a4ffff640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50109df0 0x7f2a5019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.776+0000 7f2a4f7fe640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 0x7f2a5019a540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000b4a0 tx=0x7f2a4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.776+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40014070 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.776+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a501a07a0 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.776+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a40010ac0 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.776+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40017c30 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.776+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a501a6f00 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.777+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a50103f70 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.778+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2a40017070 con 0x7f2a50103b60 2024-09-21T02:03:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.779+0000 7f2a4cff9640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a380b65a0 0x7f2a380b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.779+0000 7f2a4effd640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a380b65a0 0x7f2a380b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.779+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2a400e6b70 con 0x7f2a50103b60 2024-09-21T02:03:35.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.780+0000 7f2a4effd640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a380b65a0 0x7f2a380b8ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a44006010 tx=0x7f2a44000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.782+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a400ad280 con 0x7f2a50103b60 2024-09-21T02:03:35.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.983+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a50063800 con 0x7f2a380b65a0 2024-09-21T02:03:35.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.988+0000 7f2a4cff9640 1 -- 172.21.15.92:0/276733838 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258756 (secure 0 0 0) 0x7f2a50063800 con 0x7f2a380b65a0 2024-09-21T02:03:35.994 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:35.996 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:03:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a380b65a0 msgr2=0x7f2a380b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a380b65a0 0x7f2a380b8ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a44006010 tx=0x7f2a44000bc0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 msgr2=0x7f2a5019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 0x7f2a5019a540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000b4a0 tx=0x7f2a4000b960 comp rx=0 tx=0).stop 2024-09-21T02:03:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 shutdown_connections 2024-09-21T02:03:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a380b65a0 0x7f2a380b8ab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a50109df0 0x7f2a5019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a501094a0 0x7f2a5019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 --2- 172.21.15.92:0/276733838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a50103b60 0x7f2a5019a540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.992+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 >> 172.21.15.92:0/276733838 conn(0x7f2a5007b850 msgr2=0x7f2a50107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.993+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 shutdown_connections 2024-09-21T02:03:36.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:35.993+0000 7f2a55de8640 1 -- 172.21.15.92:0/276733838 wait complete. 2024-09-21T02:03:36.683 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-21T02:03:35.038748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665036,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":644,"num_read_kb":428,"num_write":374,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2951908,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407260,"kb_used_data":12180,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534564,"statfs":{"total":1151940427776,"available":1151523393536,"internally_reserved":0,"allocated":12472320,"data_stored":6857448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003752"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422387+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:03:12.422387+0000","last_peered":"2024-09-21T02:03:12.422387+0000","last_clean":"2024-09-21T02:03:12.422387+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:03:12.422387+0000","last_undegraded":"2024-09-21T02:03:12.422387+0000","last_fullsized":"2024-09-21T02:03:12.422387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420981+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:03:12.420981+0000","last_peered":"2024-09-21T02:03:12.420981+0000","last_clean":"2024-09-21T02:03:12.420981+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:03:12.420981+0000","last_undegraded":"2024-09-21T02:03:12.420981+0000","last_fullsized":"2024-09-21T02:03:12.420981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420779+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:03:12.420779+0000","last_peered":"2024-09-21T02:03:12.420779+0000","last_clean":"2024-09-21T02:03:12.420779+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:03:12.420779+0000","last_undegraded":"2024-09-21T02:03:12.420779+0000","last_fullsized":"2024-09-21T02:03:12.420779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418940+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:03:12.418940+0000","last_peered":"2024-09-21T02:03:12.418940+0000","last_clean":"2024-09-21T02:03:12.418940+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:03:12.418940+0000","last_undegraded":"2024-09-21T02:03:12.418940+0000","last_fullsized":"2024-09-21T02:03:12.418940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424156+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:03:12.424156+0000","last_peered":"2024-09-21T02:03:12.424156+0000","last_clean":"2024-09-21T02:03:12.424156+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:03:12.424156+0000","last_undegraded":"2024-09-21T02:03:12.424156+0000","last_fullsized":"2024-09-21T02:03:12.424156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424797+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:03:12.424797+0000","last_peered":"2024-09-21T02:03:12.424797+0000","last_clean":"2024-09-21T02:03:12.424797+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:03:12.424797+0000","last_undegraded":"2024-09-21T02:03:12.424797+0000","last_fullsized":"2024-09-21T02:03:12.424797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421051+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:03:12.421051+0000","last_peered":"2024-09-21T02:03:12.421051+0000","last_clean":"2024-09-21T02:03:12.421051+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:03:12.421051+0000","last_undegraded":"2024-09-21T02:03:12.421051+0000","last_fullsized":"2024-09-21T02:03:12.421051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424102+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:03:12.424102+0000","last_peered":"2024-09-21T02:03:12.424102+0000","last_clean":"2024-09-21T02:03:12.424102+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:03:12.424102+0000","last_undegraded":"2024-09-21T02:03:12.424102+0000","last_fullsized":"2024-09-21T02:03:12.424102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.428700+0000","last_change":"2024-09-21T02:03:12.428541+0000","last_active":"2024-09-21T02:03:12.428700+0000","last_peered":"2024-09-21T02:03:12.428700+0000","last_clean":"2024-09-21T02:03:12.428700+0000","last_became_active":"2024-09-21T02:03:12.428177+0000","last_became_peered":"2024-09-21T02:03:12.428177+0000","last_unstale":"2024-09-21T02:03:12.428700+0000","last_undegraded":"2024-09-21T02:03:12.428700+0000","last_fullsized":"2024-09-21T02:03:12.428700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:42.955294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425109+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:03:12.425109+0000","last_peered":"2024-09-21T02:03:12.425109+0000","last_clean":"2024-09-21T02:03:12.425109+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:03:12.425109+0000","last_undegraded":"2024-09-21T02:03:12.425109+0000","last_fullsized":"2024-09-21T02:03:12.425109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421620+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:03:12.421620+0000","last_peered":"2024-09-21T02:03:12.421620+0000","last_clean":"2024-09-21T02:03:12.421620+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:03:12.421620+0000","last_undegraded":"2024-09-21T02:03:12.421620+0000","last_fullsized":"2024-09-21T02:03:12.421620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426556+0000","last_change":"2024-09-21T02:03:12.426083+0000","last_active":"2024-09-21T02:03:12.426556+0000","last_peered":"2024-09-21T02:03:12.426556+0000","last_clean":"2024-09-21T02:03:12.426556+0000","last_became_active":"2024-09-21T02:03:12.425716+0000","last_became_peered":"2024-09-21T02:03:12.425716+0000","last_unstale":"2024-09-21T02:03:12.426556+0000","last_undegraded":"2024-09-21T02:03:12.426556+0000","last_fullsized":"2024-09-21T02:03:12.426556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641654+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:03:12.641654+0000","last_peered":"2024-09-21T02:03:12.641654+0000","last_clean":"2024-09-21T02:03:12.641654+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:03:12.641654+0000","last_undegraded":"2024-09-21T02:03:12.641654+0000","last_fullsized":"2024-09-21T02:03:12.641654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420658+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:03:12.420658+0000","last_peered":"2024-09-21T02:03:12.420658+0000","last_clean":"2024-09-21T02:03:12.420658+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:03:12.420658+0000","last_undegraded":"2024-09-21T02:03:12.420658+0000","last_fullsized":"2024-09-21T02:03:12.420658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420538+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:03:12.420538+0000","last_peered":"2024-09-21T02:03:12.420538+0000","last_clean":"2024-09-21T02:03:12.420538+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:03:12.420538+0000","last_undegraded":"2024-09-21T02:03:12.420538+0000","last_fullsized":"2024-09-21T02:03:12.420538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422026+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:03:12.422026+0000","last_peered":"2024-09-21T02:03:12.422026+0000","last_clean":"2024-09-21T02:03:12.422026+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:03:12.422026+0000","last_undegraded":"2024-09-21T02:03:12.422026+0000","last_fullsized":"2024-09-21T02:03:12.422026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422433+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:03:12.422433+0000","last_peered":"2024-09-21T02:03:12.422433+0000","last_clean":"2024-09-21T02:03:12.422433+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:03:12.422433+0000","last_undegraded":"2024-09-21T02:03:12.422433+0000","last_fullsized":"2024-09-21T02:03:12.422433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417841+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:03:12.417841+0000","last_peered":"2024-09-21T02:03:12.417841+0000","last_clean":"2024-09-21T02:03:12.417841+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:03:12.417841+0000","last_undegraded":"2024-09-21T02:03:12.417841+0000","last_fullsized":"2024-09-21T02:03:12.417841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.427193+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:03:12.427193+0000","last_peered":"2024-09-21T02:03:12.427193+0000","last_clean":"2024-09-21T02:03:12.427193+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:03:12.427193+0000","last_undegraded":"2024-09-21T02:03:12.427193+0000","last_fullsized":"2024-09-21T02:03:12.427193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421802+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:03:12.421802+0000","last_peered":"2024-09-21T02:03:12.421802+0000","last_clean":"2024-09-21T02:03:12.421802+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:03:12.421802+0000","last_undegraded":"2024-09-21T02:03:12.421802+0000","last_fullsized":"2024-09-21T02:03:12.421802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641652+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:03:12.641652+0000","last_peered":"2024-09-21T02:03:12.641652+0000","last_clean":"2024-09-21T02:03:12.641652+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:03:12.641652+0000","last_undegraded":"2024-09-21T02:03:12.641652+0000","last_fullsized":"2024-09-21T02:03:12.641652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422446+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:03:12.422446+0000","last_peered":"2024-09-21T02:03:12.422446+0000","last_clean":"2024-09-21T02:03:12.422446+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:03:12.422446+0000","last_undegraded":"2024-09-21T02:03:12.422446+0000","last_fullsized":"2024-09-21T02:03:12.422446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424763+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:03:12.424763+0000","last_peered":"2024-09-21T02:03:12.424763+0000","last_clean":"2024-09-21T02:03:12.424763+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:03:12.424763+0000","last_undegraded":"2024-09-21T02:03:12.424763+0000","last_fullsized":"2024-09-21T02:03:12.424763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426737+0000","last_change":"2024-09-21T02:03:12.426598+0000","last_active":"2024-09-21T02:03:12.426737+0000","last_peered":"2024-09-21T02:03:12.426737+0000","last_clean":"2024-09-21T02:03:12.426737+0000","last_became_active":"2024-09-21T02:03:12.425552+0000","last_became_peered":"2024-09-21T02:03:12.425552+0000","last_unstale":"2024-09-21T02:03:12.426737+0000","last_undegraded":"2024-09-21T02:03:12.426737+0000","last_fullsized":"2024-09-21T02:03:12.426737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:40.991213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424329+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:03:12.424329+0000","last_peered":"2024-09-21T02:03:12.424329+0000","last_clean":"2024-09-21T02:03:12.424329+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:03:12.424329+0000","last_undegraded":"2024-09-21T02:03:12.424329+0000","last_fullsized":"2024-09-21T02:03:12.424329+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.427106+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:03:12.427106+0000","last_peered":"2024-09-21T02:03:12.427106+0000","last_clean":"2024-09-21T02:03:12.427106+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:03:12.427106+0000","last_undegraded":"2024-09-21T02:03:12.427106+0000","last_fullsized":"2024-09-21T02:03:12.427106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.428809+0000","last_change":"2024-09-21T02:03:12.428701+0000","last_active":"2024-09-21T02:03:12.428809+0000","last_peered":"2024-09-21T02:03:12.428809+0000","last_clean":"2024-09-21T02:03:12.428809+0000","last_became_active":"2024-09-21T02:03:12.428283+0000","last_became_peered":"2024-09-21T02:03:12.428283+0000","last_unstale":"2024-09-21T02:03:12.428809+0000","last_undegraded":"2024-09-21T02:03:12.428809+0000","last_fullsized":"2024-09-21T02:03:12.428809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:43.972970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641662+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:03:12.641662+0000","last_peered":"2024-09-21T02:03:12.641662+0000","last_clean":"2024-09-21T02:03:12.641662+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:03:12.641662+0000","last_undegraded":"2024-09-21T02:03:12.641662+0000","last_fullsized":"2024-09-21T02:03:12.641662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420821+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:03:12.420821+0000","last_peered":"2024-09-21T02:03:12.420821+0000","last_clean":"2024-09-21T02:03:12.420821+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:03:12.420821+0000","last_undegraded":"2024-09-21T02:03:12.420821+0000","last_fullsized":"2024-09-21T02:03:12.420821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426403+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:03:12.426403+0000","last_peered":"2024-09-21T02:03:12.426403+0000","last_clean":"2024-09-21T02:03:12.426403+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:03:12.426403+0000","last_undegraded":"2024-09-21T02:03:12.426403+0000","last_fullsized":"2024-09-21T02:03:12.426403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.428902+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:03:12.428902+0000","last_peered":"2024-09-21T02:03:12.428902+0000","last_clean":"2024-09-21T02:03:12.428902+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:03:12.428902+0000","last_undegraded":"2024-09-21T02:03:12.428902+0000","last_fullsized":"2024-09-21T02:03:12.428902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421333+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:03:12.421333+0000","last_peered":"2024-09-21T02:03:12.421333+0000","last_clean":"2024-09-21T02:03:12.421333+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:03:12.421333+0000","last_undegraded":"2024-09-21T02:03:12.421333+0000","last_fullsized":"2024-09-21T02:03:12.421333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424008+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:03:12.424008+0000","last_peered":"2024-09-21T02:03:12.424008+0000","last_clean":"2024-09-21T02:03:12.424008+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:03:12.424008+0000","last_undegraded":"2024-09-21T02:03:12.424008+0000","last_fullsized":"2024-09-21T02:03:12.424008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:31.385402+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:03:31.385402+0000","last_peered":"2024-09-21T02:03:31.385402+0000","last_clean":"2024-09-21T02:03:31.385402+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:03:31.385402+0000","last_undegraded":"2024-09-21T02:03:31.385402+0000","last_fullsized":"2024-09-21T02:03:31.385402+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418986+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:03:12.418986+0000","last_peered":"2024-09-21T02:03:12.418986+0000","last_clean":"2024-09-21T02:03:12.418986+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:03:12.418986+0000","last_undegraded":"2024-09-21T02:03:12.418986+0000","last_fullsized":"2024-09-21T02:03:12.418986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424950+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:03:12.424950+0000","last_peered":"2024-09-21T02:03:12.424950+0000","last_clean":"2024-09-21T02:03:12.424950+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:03:12.424950+0000","last_undegraded":"2024-09-21T02:03:12.424950+0000","last_fullsized":"2024-09-21T02:03:12.424950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641291+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:03:12.641291+0000","last_peered":"2024-09-21T02:03:12.641291+0000","last_clean":"2024-09-21T02:03:12.641291+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:03:12.641291+0000","last_undegraded":"2024-09-21T02:03:12.641291+0000","last_fullsized":"2024-09-21T02:03:12.641291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424224+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:03:12.424224+0000","last_peered":"2024-09-21T02:03:12.424224+0000","last_clean":"2024-09-21T02:03:12.424224+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:03:12.424224+0000","last_undegraded":"2024-09-21T02:03:12.424224+0000","last_fullsized":"2024-09-21T02:03:12.424224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419546+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:03:12.419546+0000","last_peered":"2024-09-21T02:03:12.419546+0000","last_clean":"2024-09-21T02:03:12.419546+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:03:12.419546+0000","last_undegraded":"2024-09-21T02:03:12.419546+0000","last_fullsized":"2024-09-21T02:03:12.419546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417576+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:03:12.417576+0000","last_peered":"2024-09-21T02:03:12.417576+0000","last_clean":"2024-09-21T02:03:12.417576+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:03:12.417576+0000","last_undegraded":"2024-09-21T02:03:12.417576+0000","last_fullsized":"2024-09-21T02:03:12.417576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:33.387995+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:03:33.387995+0000","last_peered":"2024-09-21T02:03:33.387995+0000","last_clean":"2024-09-21T02:03:33.387995+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:03:33.387995+0000","last_undegraded":"2024-09-21T02:03:33.387995+0000","last_fullsized":"2024-09-21T02:03:33.387995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423727+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:03:12.423727+0000","last_peered":"2024-09-21T02:03:12.423727+0000","last_clean":"2024-09-21T02:03:12.423727+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:03:12.423727+0000","last_undegraded":"2024-09-21T02:03:12.423727+0000","last_fullsized":"2024-09-21T02:03:12.423727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421592+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:03:12.421592+0000","last_peered":"2024-09-21T02:03:12.421592+0000","last_clean":"2024-09-21T02:03:12.421592+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:03:12.421592+0000","last_undegraded":"2024-09-21T02:03:12.421592+0000","last_fullsized":"2024-09-21T02:03:12.421592+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423973+0000","last_change":"2024-09-21T02:03:12.423870+0000","last_active":"2024-09-21T02:03:12.423973+0000","last_peered":"2024-09-21T02:03:12.423973+0000","last_clean":"2024-09-21T02:03:12.423973+0000","last_became_active":"2024-09-21T02:03:12.423441+0000","last_became_peered":"2024-09-21T02:03:12.423441+0000","last_unstale":"2024-09-21T02:03:12.423973+0000","last_undegraded":"2024-09-21T02:03:12.423973+0000","last_fullsized":"2024-09-21T02:03:12.423973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:59.071213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419481+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:03:12.419481+0000","last_peered":"2024-09-21T02:03:12.419481+0000","last_clean":"2024-09-21T02:03:12.419481+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:03:12.419481+0000","last_undegraded":"2024-09-21T02:03:12.419481+0000","last_fullsized":"2024-09-21T02:03:12.419481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422669+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:03:12.422669+0000","last_peered":"2024-09-21T02:03:12.422669+0000","last_clean":"2024-09-21T02:03:12.422669+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:03:12.422669+0000","last_undegraded":"2024-09-21T02:03:12.422669+0000","last_fullsized":"2024-09-21T02:03:12.422669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'32","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:33.387994+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:03:33.387994+0000","last_peered":"2024-09-21T02:03:33.387994+0000","last_clean":"2024-09-21T02:03:33.387994+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:03:33.387994+0000","last_undegraded":"2024-09-21T02:03:33.387994+0000","last_fullsized":"2024-09-21T02:03:33.387994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66448,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":122,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424624+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:03:12.424624+0000","last_peered":"2024-09-21T02:03:12.424624+0000","last_clean":"2024-09-21T02:03:12.424624+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:03:12.424624+0000","last_undegraded":"2024-09-21T02:03:12.424624+0000","last_fullsized":"2024-09-21T02:03:12.424624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442392+0000","last_change":"2024-09-21T02:03:12.441759+0000","last_active":"2024-09-21T02:03:12.442392+0000","last_peered":"2024-09-21T02:03:12.442392+0000","last_clean":"2024-09-21T02:03:12.442392+0000","last_became_active":"2024-09-21T02:03:12.426827+0000","last_became_peered":"2024-09-21T02:03:12.426827+0000","last_unstale":"2024-09-21T02:03:12.442392+0000","last_undegraded":"2024-09-21T02:03:12.442392+0000","last_fullsized":"2024-09-21T02:03:12.442392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:33.917276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641411+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:03:12.641411+0000","last_peered":"2024-09-21T02:03:12.641411+0000","last_clean":"2024-09-21T02:03:12.641411+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:03:12.641411+0000","last_undegraded":"2024-09-21T02:03:12.641411+0000","last_fullsized":"2024-09-21T02:03:12.641411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418936+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:03:12.418936+0000","last_peered":"2024-09-21T02:03:12.418936+0000","last_clean":"2024-09-21T02:03:12.418936+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:03:12.418936+0000","last_undegraded":"2024-09-21T02:03:12.418936+0000","last_fullsized":"2024-09-21T02:03:12.418936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.643129+0000","last_change":"2024-09-21T02:03:12.643022+0000","last_active":"2024-09-21T02:03:12.643129+0000","last_peered":"2024-09-21T02:03:12.643129+0000","last_clean":"2024-09-21T02:03:12.643129+0000","last_became_active":"2024-09-21T02:03:12.642501+0000","last_became_peered":"2024-09-21T02:03:12.642501+0000","last_unstale":"2024-09-21T02:03:12.643129+0000","last_undegraded":"2024-09-21T02:03:12.643129+0000","last_fullsized":"2024-09-21T02:03:12.643129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:29.600445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:33.403689+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:03:33.403689+0000","last_peered":"2024-09-21T02:03:33.403689+0000","last_clean":"2024-09-21T02:03:33.403689+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:03:33.403689+0000","last_undegraded":"2024-09-21T02:03:33.403689+0000","last_fullsized":"2024-09-21T02:03:33.403689+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421158+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:03:12.421158+0000","last_peered":"2024-09-21T02:03:12.421158+0000","last_clean":"2024-09-21T02:03:12.421158+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:03:12.421158+0000","last_undegraded":"2024-09-21T02:03:12.421158+0000","last_fullsized":"2024-09-21T02:03:12.421158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423964+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:03:12.423964+0000","last_peered":"2024-09-21T02:03:12.423964+0000","last_clean":"2024-09-21T02:03:12.423964+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:03:12.423964+0000","last_undegraded":"2024-09-21T02:03:12.423964+0000","last_fullsized":"2024-09-21T02:03:12.423964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418934+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:03:12.418934+0000","last_peered":"2024-09-21T02:03:12.418934+0000","last_clean":"2024-09-21T02:03:12.418934+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:03:12.418934+0000","last_undegraded":"2024-09-21T02:03:12.418934+0000","last_fullsized":"2024-09-21T02:03:12.418934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425082+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:03:12.425082+0000","last_peered":"2024-09-21T02:03:12.425082+0000","last_clean":"2024-09-21T02:03:12.425082+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:03:12.425082+0000","last_undegraded":"2024-09-21T02:03:12.425082+0000","last_fullsized":"2024-09-21T02:03:12.425082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422136+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:03:12.422136+0000","last_peered":"2024-09-21T02:03:12.422136+0000","last_clean":"2024-09-21T02:03:12.422136+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:03:12.422136+0000","last_undegraded":"2024-09-21T02:03:12.422136+0000","last_fullsized":"2024-09-21T02:03:12.422136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641780+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:03:12.641780+0000","last_peered":"2024-09-21T02:03:12.641780+0000","last_clean":"2024-09-21T02:03:12.641780+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:03:12.641780+0000","last_undegraded":"2024-09-21T02:03:12.641780+0000","last_fullsized":"2024-09-21T02:03:12.641780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422063+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:03:12.422063+0000","last_peered":"2024-09-21T02:03:12.422063+0000","last_clean":"2024-09-21T02:03:12.422063+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:03:12.422063+0000","last_undegraded":"2024-09-21T02:03:12.422063+0000","last_fullsized":"2024-09-21T02:03:12.422063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419713+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:03:12.419713+0000","last_peered":"2024-09-21T02:03:12.419713+0000","last_clean":"2024-09-21T02:03:12.419713+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:03:12.419713+0000","last_undegraded":"2024-09-21T02:03:12.419713+0000","last_fullsized":"2024-09-21T02:03:12.419713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421761+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:03:12.421761+0000","last_peered":"2024-09-21T02:03:12.421761+0000","last_clean":"2024-09-21T02:03:12.421761+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:03:12.421761+0000","last_undegraded":"2024-09-21T02:03:12.421761+0000","last_fullsized":"2024-09-21T02:03:12.421761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":85,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:32.387046+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:03:32.387046+0000","last_peered":"2024-09-21T02:03:32.387046+0000","last_clean":"2024-09-21T02:03:32.387046+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:03:32.387046+0000","last_undegraded":"2024-09-21T02:03:32.387046+0000","last_fullsized":"2024-09-21T02:03:32.387046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421906+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:03:12.421906+0000","last_peered":"2024-09-21T02:03:12.421906+0000","last_clean":"2024-09-21T02:03:12.421906+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:03:12.421906+0000","last_undegraded":"2024-09-21T02:03:12.421906+0000","last_fullsized":"2024-09-21T02:03:12.421906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422111+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:03:12.422111+0000","last_peered":"2024-09-21T02:03:12.422111+0000","last_clean":"2024-09-21T02:03:12.422111+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:03:12.422111+0000","last_undegraded":"2024-09-21T02:03:12.422111+0000","last_fullsized":"2024-09-21T02:03:12.422111+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419020+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:03:12.419020+0000","last_peered":"2024-09-21T02:03:12.419020+0000","last_clean":"2024-09-21T02:03:12.419020+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:03:12.419020+0000","last_undegraded":"2024-09-21T02:03:12.419020+0000","last_fullsized":"2024-09-21T02:03:12.419020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424850+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:03:12.424850+0000","last_peered":"2024-09-21T02:03:12.424850+0000","last_clean":"2024-09-21T02:03:12.424850+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:03:12.424850+0000","last_undegraded":"2024-09-21T02:03:12.424850+0000","last_fullsized":"2024-09-21T02:03:12.424850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421843+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:03:12.421843+0000","last_peered":"2024-09-21T02:03:12.421843+0000","last_clean":"2024-09-21T02:03:12.421843+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:03:12.421843+0000","last_undegraded":"2024-09-21T02:03:12.421843+0000","last_fullsized":"2024-09-21T02:03:12.421843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641320+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:03:12.641320+0000","last_peered":"2024-09-21T02:03:12.641320+0000","last_clean":"2024-09-21T02:03:12.641320+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:03:12.641320+0000","last_undegraded":"2024-09-21T02:03:12.641320+0000","last_fullsized":"2024-09-21T02:03:12.641320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:32.401262+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:03:32.401262+0000","last_peered":"2024-09-21T02:03:32.401262+0000","last_clean":"2024-09-21T02:03:32.401262+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:03:32.401262+0000","last_undegraded":"2024-09-21T02:03:32.401262+0000","last_fullsized":"2024-09-21T02:03:32.401262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419308+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:03:12.419308+0000","last_peered":"2024-09-21T02:03:12.419308+0000","last_clean":"2024-09-21T02:03:12.419308+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:03:12.419308+0000","last_undegraded":"2024-09-21T02:03:12.419308+0000","last_fullsized":"2024-09-21T02:03:12.419308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:31.385645+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:03:31.385645+0000","last_peered":"2024-09-21T02:03:31.385645+0000","last_clean":"2024-09-21T02:03:31.385645+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:03:31.385645+0000","last_undegraded":"2024-09-21T02:03:31.385645+0000","last_fullsized":"2024-09-21T02:03:31.385645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425206+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:03:12.425206+0000","last_peered":"2024-09-21T02:03:12.425206+0000","last_clean":"2024-09-21T02:03:12.425206+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:03:12.425206+0000","last_undegraded":"2024-09-21T02:03:12.425206+0000","last_fullsized":"2024-09-21T02:03:12.425206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417918+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:03:12.417918+0000","last_peered":"2024-09-21T02:03:12.417918+0000","last_clean":"2024-09-21T02:03:12.417918+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:03:12.417918+0000","last_undegraded":"2024-09-21T02:03:12.417918+0000","last_fullsized":"2024-09-21T02:03:12.417918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424809+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:03:12.424809+0000","last_peered":"2024-09-21T02:03:12.424809+0000","last_clean":"2024-09-21T02:03:12.424809+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:03:12.424809+0000","last_undegraded":"2024-09-21T02:03:12.424809+0000","last_fullsized":"2024-09-21T02:03:12.424809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425092+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:03:12.425092+0000","last_peered":"2024-09-21T02:03:12.425092+0000","last_clean":"2024-09-21T02:03:12.425092+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:03:12.425092+0000","last_undegraded":"2024-09-21T02:03:12.425092+0000","last_fullsized":"2024-09-21T02:03:12.425092+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641740+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:03:12.641740+0000","last_peered":"2024-09-21T02:03:12.641740+0000","last_clean":"2024-09-21T02:03:12.641740+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:03:12.641740+0000","last_undegraded":"2024-09-21T02:03:12.641740+0000","last_fullsized":"2024-09-21T02:03:12.641740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417868+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:03:12.417868+0000","last_peered":"2024-09-21T02:03:12.417868+0000","last_clean":"2024-09-21T02:03:12.417868+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:03:12.417868+0000","last_undegraded":"2024-09-21T02:03:12.417868+0000","last_fullsized":"2024-09-21T02:03:12.417868+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424450+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:03:12.424450+0000","last_peered":"2024-09-21T02:03:12.424450+0000","last_clean":"2024-09-21T02:03:12.424450+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:03:12.424450+0000","last_undegraded":"2024-09-21T02:03:12.424450+0000","last_fullsized":"2024-09-21T02:03:12.424450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419259+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:03:12.419259+0000","last_peered":"2024-09-21T02:03:12.419259+0000","last_clean":"2024-09-21T02:03:12.419259+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:03:12.419259+0000","last_undegraded":"2024-09-21T02:03:12.419259+0000","last_fullsized":"2024-09-21T02:03:12.419259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421030+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:03:12.421030+0000","last_peered":"2024-09-21T02:03:12.421030+0000","last_clean":"2024-09-21T02:03:12.421030+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:03:12.421030+0000","last_undegraded":"2024-09-21T02:03:12.421030+0000","last_fullsized":"2024-09-21T02:03:12.421030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422234+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:03:12.422234+0000","last_peered":"2024-09-21T02:03:12.422234+0000","last_clean":"2024-09-21T02:03:12.422234+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:03:12.422234+0000","last_undegraded":"2024-09-21T02:03:12.422234+0000","last_fullsized":"2024-09-21T02:03:12.422234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423901+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:03:12.423901+0000","last_peered":"2024-09-21T02:03:12.423901+0000","last_clean":"2024-09-21T02:03:12.423901+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:03:12.423901+0000","last_undegraded":"2024-09-21T02:03:12.423901+0000","last_fullsized":"2024-09-21T02:03:12.423901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424628+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:03:12.424628+0000","last_peered":"2024-09-21T02:03:12.424628+0000","last_clean":"2024-09-21T02:03:12.424628+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:03:12.424628+0000","last_undegraded":"2024-09-21T02:03:12.424628+0000","last_fullsized":"2024-09-21T02:03:12.424628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421779+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:03:12.421779+0000","last_peered":"2024-09-21T02:03:12.421779+0000","last_clean":"2024-09-21T02:03:12.421779+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:03:12.421779+0000","last_undegraded":"2024-09-21T02:03:12.421779+0000","last_fullsized":"2024-09-21T02:03:12.421779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426547+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:03:12.426547+0000","last_peered":"2024-09-21T02:03:12.426547+0000","last_clean":"2024-09-21T02:03:12.426547+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:03:12.426547+0000","last_undegraded":"2024-09-21T02:03:12.426547+0000","last_fullsized":"2024-09-21T02:03:12.426547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424421+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:03:12.424421+0000","last_peered":"2024-09-21T02:03:12.424421+0000","last_clean":"2024-09-21T02:03:12.424421+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:03:12.424421+0000","last_undegraded":"2024-09-21T02:03:12.424421+0000","last_fullsized":"2024-09-21T02:03:12.424421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.446014+0000","last_change":"2024-09-21T02:03:12.445516+0000","last_active":"2024-09-21T02:03:12.446014+0000","last_peered":"2024-09-21T02:03:12.446014+0000","last_clean":"2024-09-21T02:03:12.446014+0000","last_became_active":"2024-09-21T02:03:12.430149+0000","last_became_peered":"2024-09-21T02:03:12.430149+0000","last_unstale":"2024-09-21T02:03:12.446014+0000","last_undegraded":"2024-09-21T02:03:12.446014+0000","last_fullsized":"2024-09-21T02:03:12.446014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:23.573018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421274+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:03:12.421274+0000","last_peered":"2024-09-21T02:03:12.421274+0000","last_clean":"2024-09-21T02:03:12.421274+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:03:12.421274+0000","last_undegraded":"2024-09-21T02:03:12.421274+0000","last_fullsized":"2024-09-21T02:03:12.421274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.643058+0000","last_change":"2024-09-21T02:03:12.642935+0000","last_active":"2024-09-21T02:03:12.643058+0000","last_peered":"2024-09-21T02:03:12.643058+0000","last_clean":"2024-09-21T02:03:12.643058+0000","last_became_active":"2024-09-21T02:03:12.642520+0000","last_became_peered":"2024-09-21T02:03:12.642520+0000","last_unstale":"2024-09-21T02:03:12.643058+0000","last_undegraded":"2024-09-21T02:03:12.643058+0000","last_fullsized":"2024-09-21T02:03:12.643058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:32.147679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424736+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:03:12.424736+0000","last_peered":"2024-09-21T02:03:12.424736+0000","last_clean":"2024-09-21T02:03:12.424736+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:03:12.424736+0000","last_undegraded":"2024-09-21T02:03:12.424736+0000","last_fullsized":"2024-09-21T02:03:12.424736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422588+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:03:12.422588+0000","last_peered":"2024-09-21T02:03:12.422588+0000","last_clean":"2024-09-21T02:03:12.422588+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:03:12.422588+0000","last_undegraded":"2024-09-21T02:03:12.422588+0000","last_fullsized":"2024-09-21T02:03:12.422588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.441500+0000","last_change":"2024-09-21T02:03:12.441361+0000","last_active":"2024-09-21T02:03:12.441500+0000","last_peered":"2024-09-21T02:03:12.441500+0000","last_clean":"2024-09-21T02:03:12.441500+0000","last_became_active":"2024-09-21T02:03:12.428877+0000","last_became_peered":"2024-09-21T02:03:12.428877+0000","last_unstale":"2024-09-21T02:03:12.441500+0000","last_undegraded":"2024-09-21T02:03:12.441500+0000","last_fullsized":"2024-09-21T02:03:12.441500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.464134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421013+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:03:12.421013+0000","last_peered":"2024-09-21T02:03:12.421013+0000","last_clean":"2024-09-21T02:03:12.421013+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:03:12.421013+0000","last_undegraded":"2024-09-21T02:03:12.421013+0000","last_fullsized":"2024-09-21T02:03:12.421013+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424866+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:03:12.424866+0000","last_peered":"2024-09-21T02:03:12.424866+0000","last_clean":"2024-09-21T02:03:12.424866+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:03:12.424866+0000","last_undegraded":"2024-09-21T02:03:12.424866+0000","last_fullsized":"2024-09-21T02:03:12.424866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424636+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:03:12.424636+0000","last_peered":"2024-09-21T02:03:12.424636+0000","last_clean":"2024-09-21T02:03:12.424636+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:03:12.424636+0000","last_undegraded":"2024-09-21T02:03:12.424636+0000","last_fullsized":"2024-09-21T02:03:12.424636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":88,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:31.385033+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:03:31.385033+0000","last_peered":"2024-09-21T02:03:31.385033+0000","last_clean":"2024-09-21T02:03:31.385033+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:03:31.385033+0000","last_undegraded":"2024-09-21T02:03:31.385033+0000","last_fullsized":"2024-09-21T02:03:31.385033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74434,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":364,"num_write":249,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":68,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":860160,"data_stored":406846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1560576,"data_stored":1064255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":876544,"data_stored":406317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":872448,"data_stored":406695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1470464,"data_stored":999995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":860160,"data_stored":406425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1482752,"data_stored":998891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":876544,"data_stored":405859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":474329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":929792,"data_stored":473786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":880640,"data_stored":407764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:03:36.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:03:36.951 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:36 smithi136 ceph-mon[34754]: pgmap v319: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:36 smithi136 ceph-mon[34754]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:03:37.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:36 smithi092 ceph-mon[31190]: pgmap v319: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:37.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:36 smithi092 ceph-mon[31190]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:03:37.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:36 smithi099 ceph-mon[34404]: pgmap v319: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:37.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:36 smithi099 ceph-mon[34404]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 -- 172.21.15.92:0/3858084194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c108d00 msgr2=0x7fa20c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 --2- 172.21.15.92:0/3858084194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c108d00 0x7fa20c10f530 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa204009ca0 tx=0x7fa20402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 -- 172.21.15.92:0/3858084194 shutdown_connections 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 --2- 172.21.15.92:0/3858084194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c108d00 0x7fa20c10f530 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 --2- 172.21.15.92:0/3858084194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c1083b0 0x7fa20c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 --2- 172.21.15.92:0/3858084194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa20c102a70 0x7fa20c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 -- 172.21.15.92:0/3858084194 >> 172.21.15.92:0/3858084194 conn(0x7fa20c079710 msgr2=0x7fa20c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 -- 172.21.15.92:0/3858084194 shutdown_connections 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 -- 172.21.15.92:0/3858084194 wait complete. 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.469+0000 7fa213088640 1 Processor -- start 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 -- start start 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c102a70 0x7fa20c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 0x7fa20c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa20c108d00 0x7fa20c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1129c0 con 0x7fa20c1083b0 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c112840 con 0x7fa20c102a70 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa213088640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c112b40 con 0x7fa20c108d00 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 0x7fa20c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 0x7fa20c19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44388/0 (socket says 172.21.15.92:44388) 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 -- 172.21.15.92:0/4191296323 learned_addr learned my addr 172.21.15.92:0/4191296323 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa210e24640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c102a70 0x7fa20c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 -- 172.21.15.92:0/4191296323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa20c108d00 msgr2=0x7fa20c19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa20c108d00 0x7fa20c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 -- 172.21.15.92:0/4191296323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c102a70 msgr2=0x7fa20c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c102a70 0x7fa20c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 -- 172.21.15.92:0/4191296323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c1a6510 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa210e24640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c102a70 0x7fa20c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.470+0000 7fa20bfff640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 0x7fa20c19b850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c8c0 tx=0x7fa1f800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.471+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8007bb0 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.471+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1f8007d50 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.471+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c1a6800 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.471+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80187c0 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.471+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1a6d40 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.472+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c103cf0 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.473+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa1f800fa20 con 0x7fa20c1083b0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.474+0000 7fa209ffb640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa1fc0b65d0 0x7fa1fc0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.474+0000 7fa210e24640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa1fc0b65d0 0x7fa1fc0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.474+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa1f80e75c0 con 0x7fa20c1083b0 2024-09-21T02:03:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.475+0000 7fa210e24640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa1fc0b65d0 0x7fa1fc0b8ae0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa20c06c6e0 tx=0x7fa200005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.479+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1f8014030 con 0x7fa20c1083b0 2024-09-21T02:03:37.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.655+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa20c063800 con 0x7fa1fc0b65d0 2024-09-21T02:03:37.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.659+0000 7fa209ffb640 1 -- 172.21.15.92:0/4191296323 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258756 (secure 0 0 0) 0x7fa20c063800 con 0x7fa1fc0b65d0 2024-09-21T02:03:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:37.663 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa1fc0b65d0 msgr2=0x7fa1fc0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa1fc0b65d0 0x7fa1fc0b8ae0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa20c06c6e0 tx=0x7fa200005e70 comp rx=0 tx=0).stop 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 msgr2=0x7fa20c19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 0x7fa20c19b850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c8c0 tx=0x7fa1f800cd80 comp rx=0 tx=0).stop 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 shutdown_connections 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa1fc0b65d0 0x7fa1fc0b8ae0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa20c108d00 0x7fa20c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20c1083b0 0x7fa20c19b850 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 --2- 172.21.15.92:0/4191296323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa20c102a70 0x7fa20c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 >> 172.21.15.92:0/4191296323 conn(0x7fa20c079710 msgr2=0x7fa20c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:37.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 shutdown_connections 2024-09-21T02:03:37.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:37.662+0000 7fa213088640 1 -- 172.21.15.92:0/4191296323 wait complete. 2024-09-21T02:03:38.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:03:38.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:03:38.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:03:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:03:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:03:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:03:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:03:38.385 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-21T02:03:37.039321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665036,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":644,"num_read_kb":428,"num_write":374,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2951908,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407260,"kb_used_data":12180,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534564,"statfs":{"total":1151940427776,"available":1151523393536,"internally_reserved":0,"allocated":12472320,"data_stored":6857448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003701"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422387+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:03:12.422387+0000","last_peered":"2024-09-21T02:03:12.422387+0000","last_clean":"2024-09-21T02:03:12.422387+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:03:12.422387+0000","last_undegraded":"2024-09-21T02:03:12.422387+0000","last_fullsized":"2024-09-21T02:03:12.422387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420981+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:03:12.420981+0000","last_peered":"2024-09-21T02:03:12.420981+0000","last_clean":"2024-09-21T02:03:12.420981+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:03:12.420981+0000","last_undegraded":"2024-09-21T02:03:12.420981+0000","last_fullsized":"2024-09-21T02:03:12.420981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420779+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:03:12.420779+0000","last_peered":"2024-09-21T02:03:12.420779+0000","last_clean":"2024-09-21T02:03:12.420779+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:03:12.420779+0000","last_undegraded":"2024-09-21T02:03:12.420779+0000","last_fullsized":"2024-09-21T02:03:12.420779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418940+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:03:12.418940+0000","last_peered":"2024-09-21T02:03:12.418940+0000","last_clean":"2024-09-21T02:03:12.418940+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:03:12.418940+0000","last_undegraded":"2024-09-21T02:03:12.418940+0000","last_fullsized":"2024-09-21T02:03:12.418940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424156+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:03:12.424156+0000","last_peered":"2024-09-21T02:03:12.424156+0000","last_clean":"2024-09-21T02:03:12.424156+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:03:12.424156+0000","last_undegraded":"2024-09-21T02:03:12.424156+0000","last_fullsized":"2024-09-21T02:03:12.424156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424797+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:03:12.424797+0000","last_peered":"2024-09-21T02:03:12.424797+0000","last_clean":"2024-09-21T02:03:12.424797+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:03:12.424797+0000","last_undegraded":"2024-09-21T02:03:12.424797+0000","last_fullsized":"2024-09-21T02:03:12.424797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421051+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:03:12.421051+0000","last_peered":"2024-09-21T02:03:12.421051+0000","last_clean":"2024-09-21T02:03:12.421051+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:03:12.421051+0000","last_undegraded":"2024-09-21T02:03:12.421051+0000","last_fullsized":"2024-09-21T02:03:12.421051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424102+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:03:12.424102+0000","last_peered":"2024-09-21T02:03:12.424102+0000","last_clean":"2024-09-21T02:03:12.424102+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:03:12.424102+0000","last_undegraded":"2024-09-21T02:03:12.424102+0000","last_fullsized":"2024-09-21T02:03:12.424102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.428700+0000","last_change":"2024-09-21T02:03:12.428541+0000","last_active":"2024-09-21T02:03:12.428700+0000","last_peered":"2024-09-21T02:03:12.428700+0000","last_clean":"2024-09-21T02:03:12.428700+0000","last_became_active":"2024-09-21T02:03:12.428177+0000","last_became_peered":"2024-09-21T02:03:12.428177+0000","last_unstale":"2024-09-21T02:03:12.428700+0000","last_undegraded":"2024-09-21T02:03:12.428700+0000","last_fullsized":"2024-09-21T02:03:12.428700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:42.955294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425109+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:03:12.425109+0000","last_peered":"2024-09-21T02:03:12.425109+0000","last_clean":"2024-09-21T02:03:12.425109+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:03:12.425109+0000","last_undegraded":"2024-09-21T02:03:12.425109+0000","last_fullsized":"2024-09-21T02:03:12.425109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421620+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:03:12.421620+0000","last_peered":"2024-09-21T02:03:12.421620+0000","last_clean":"2024-09-21T02:03:12.421620+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:03:12.421620+0000","last_undegraded":"2024-09-21T02:03:12.421620+0000","last_fullsized":"2024-09-21T02:03:12.421620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426556+0000","last_change":"2024-09-21T02:03:12.426083+0000","last_active":"2024-09-21T02:03:12.426556+0000","last_peered":"2024-09-21T02:03:12.426556+0000","last_clean":"2024-09-21T02:03:12.426556+0000","last_became_active":"2024-09-21T02:03:12.425716+0000","last_became_peered":"2024-09-21T02:03:12.425716+0000","last_unstale":"2024-09-21T02:03:12.426556+0000","last_undegraded":"2024-09-21T02:03:12.426556+0000","last_fullsized":"2024-09-21T02:03:12.426556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641654+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:03:12.641654+0000","last_peered":"2024-09-21T02:03:12.641654+0000","last_clean":"2024-09-21T02:03:12.641654+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:03:12.641654+0000","last_undegraded":"2024-09-21T02:03:12.641654+0000","last_fullsized":"2024-09-21T02:03:12.641654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420658+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:03:12.420658+0000","last_peered":"2024-09-21T02:03:12.420658+0000","last_clean":"2024-09-21T02:03:12.420658+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:03:12.420658+0000","last_undegraded":"2024-09-21T02:03:12.420658+0000","last_fullsized":"2024-09-21T02:03:12.420658+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420538+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:03:12.420538+0000","last_peered":"2024-09-21T02:03:12.420538+0000","last_clean":"2024-09-21T02:03:12.420538+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:03:12.420538+0000","last_undegraded":"2024-09-21T02:03:12.420538+0000","last_fullsized":"2024-09-21T02:03:12.420538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422026+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:03:12.422026+0000","last_peered":"2024-09-21T02:03:12.422026+0000","last_clean":"2024-09-21T02:03:12.422026+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:03:12.422026+0000","last_undegraded":"2024-09-21T02:03:12.422026+0000","last_fullsized":"2024-09-21T02:03:12.422026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422433+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:03:12.422433+0000","last_peered":"2024-09-21T02:03:12.422433+0000","last_clean":"2024-09-21T02:03:12.422433+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:03:12.422433+0000","last_undegraded":"2024-09-21T02:03:12.422433+0000","last_fullsized":"2024-09-21T02:03:12.422433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417841+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:03:12.417841+0000","last_peered":"2024-09-21T02:03:12.417841+0000","last_clean":"2024-09-21T02:03:12.417841+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:03:12.417841+0000","last_undegraded":"2024-09-21T02:03:12.417841+0000","last_fullsized":"2024-09-21T02:03:12.417841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.427193+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:03:12.427193+0000","last_peered":"2024-09-21T02:03:12.427193+0000","last_clean":"2024-09-21T02:03:12.427193+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:03:12.427193+0000","last_undegraded":"2024-09-21T02:03:12.427193+0000","last_fullsized":"2024-09-21T02:03:12.427193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421802+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:03:12.421802+0000","last_peered":"2024-09-21T02:03:12.421802+0000","last_clean":"2024-09-21T02:03:12.421802+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:03:12.421802+0000","last_undegraded":"2024-09-21T02:03:12.421802+0000","last_fullsized":"2024-09-21T02:03:12.421802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641652+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:03:12.641652+0000","last_peered":"2024-09-21T02:03:12.641652+0000","last_clean":"2024-09-21T02:03:12.641652+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:03:12.641652+0000","last_undegraded":"2024-09-21T02:03:12.641652+0000","last_fullsized":"2024-09-21T02:03:12.641652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422446+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:03:12.422446+0000","last_peered":"2024-09-21T02:03:12.422446+0000","last_clean":"2024-09-21T02:03:12.422446+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:03:12.422446+0000","last_undegraded":"2024-09-21T02:03:12.422446+0000","last_fullsized":"2024-09-21T02:03:12.422446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424763+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:03:12.424763+0000","last_peered":"2024-09-21T02:03:12.424763+0000","last_clean":"2024-09-21T02:03:12.424763+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:03:12.424763+0000","last_undegraded":"2024-09-21T02:03:12.424763+0000","last_fullsized":"2024-09-21T02:03:12.424763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426737+0000","last_change":"2024-09-21T02:03:12.426598+0000","last_active":"2024-09-21T02:03:12.426737+0000","last_peered":"2024-09-21T02:03:12.426737+0000","last_clean":"2024-09-21T02:03:12.426737+0000","last_became_active":"2024-09-21T02:03:12.425552+0000","last_became_peered":"2024-09-21T02:03:12.425552+0000","last_unstale":"2024-09-21T02:03:12.426737+0000","last_undegraded":"2024-09-21T02:03:12.426737+0000","last_fullsized":"2024-09-21T02:03:12.426737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:40.991213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424329+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:03:12.424329+0000","last_peered":"2024-09-21T02:03:12.424329+0000","last_clean":"2024-09-21T02:03:12.424329+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:03:12.424329+0000","last_undegraded":"2024-09-21T02:03:12.424329+0000","last_fullsized":"2024-09-21T02:03:12.424329+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.427106+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:03:12.427106+0000","last_peered":"2024-09-21T02:03:12.427106+0000","last_clean":"2024-09-21T02:03:12.427106+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:03:12.427106+0000","last_undegraded":"2024-09-21T02:03:12.427106+0000","last_fullsized":"2024-09-21T02:03:12.427106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.428809+0000","last_change":"2024-09-21T02:03:12.428701+0000","last_active":"2024-09-21T02:03:12.428809+0000","last_peered":"2024-09-21T02:03:12.428809+0000","last_clean":"2024-09-21T02:03:12.428809+0000","last_became_active":"2024-09-21T02:03:12.428283+0000","last_became_peered":"2024-09-21T02:03:12.428283+0000","last_unstale":"2024-09-21T02:03:12.428809+0000","last_undegraded":"2024-09-21T02:03:12.428809+0000","last_fullsized":"2024-09-21T02:03:12.428809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:43.972970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641662+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:03:12.641662+0000","last_peered":"2024-09-21T02:03:12.641662+0000","last_clean":"2024-09-21T02:03:12.641662+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:03:12.641662+0000","last_undegraded":"2024-09-21T02:03:12.641662+0000","last_fullsized":"2024-09-21T02:03:12.641662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.420821+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:03:12.420821+0000","last_peered":"2024-09-21T02:03:12.420821+0000","last_clean":"2024-09-21T02:03:12.420821+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:03:12.420821+0000","last_undegraded":"2024-09-21T02:03:12.420821+0000","last_fullsized":"2024-09-21T02:03:12.420821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426403+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:03:12.426403+0000","last_peered":"2024-09-21T02:03:12.426403+0000","last_clean":"2024-09-21T02:03:12.426403+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:03:12.426403+0000","last_undegraded":"2024-09-21T02:03:12.426403+0000","last_fullsized":"2024-09-21T02:03:12.426403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.428902+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:03:12.428902+0000","last_peered":"2024-09-21T02:03:12.428902+0000","last_clean":"2024-09-21T02:03:12.428902+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:03:12.428902+0000","last_undegraded":"2024-09-21T02:03:12.428902+0000","last_fullsized":"2024-09-21T02:03:12.428902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421333+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:03:12.421333+0000","last_peered":"2024-09-21T02:03:12.421333+0000","last_clean":"2024-09-21T02:03:12.421333+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:03:12.421333+0000","last_undegraded":"2024-09-21T02:03:12.421333+0000","last_fullsized":"2024-09-21T02:03:12.421333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424008+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:03:12.424008+0000","last_peered":"2024-09-21T02:03:12.424008+0000","last_clean":"2024-09-21T02:03:12.424008+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:03:12.424008+0000","last_undegraded":"2024-09-21T02:03:12.424008+0000","last_fullsized":"2024-09-21T02:03:12.424008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:31.385402+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:03:31.385402+0000","last_peered":"2024-09-21T02:03:31.385402+0000","last_clean":"2024-09-21T02:03:31.385402+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:03:31.385402+0000","last_undegraded":"2024-09-21T02:03:31.385402+0000","last_fullsized":"2024-09-21T02:03:31.385402+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418986+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:03:12.418986+0000","last_peered":"2024-09-21T02:03:12.418986+0000","last_clean":"2024-09-21T02:03:12.418986+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:03:12.418986+0000","last_undegraded":"2024-09-21T02:03:12.418986+0000","last_fullsized":"2024-09-21T02:03:12.418986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424950+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:03:12.424950+0000","last_peered":"2024-09-21T02:03:12.424950+0000","last_clean":"2024-09-21T02:03:12.424950+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:03:12.424950+0000","last_undegraded":"2024-09-21T02:03:12.424950+0000","last_fullsized":"2024-09-21T02:03:12.424950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641291+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:03:12.641291+0000","last_peered":"2024-09-21T02:03:12.641291+0000","last_clean":"2024-09-21T02:03:12.641291+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:03:12.641291+0000","last_undegraded":"2024-09-21T02:03:12.641291+0000","last_fullsized":"2024-09-21T02:03:12.641291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424224+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:03:12.424224+0000","last_peered":"2024-09-21T02:03:12.424224+0000","last_clean":"2024-09-21T02:03:12.424224+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:03:12.424224+0000","last_undegraded":"2024-09-21T02:03:12.424224+0000","last_fullsized":"2024-09-21T02:03:12.424224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419546+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:03:12.419546+0000","last_peered":"2024-09-21T02:03:12.419546+0000","last_clean":"2024-09-21T02:03:12.419546+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:03:12.419546+0000","last_undegraded":"2024-09-21T02:03:12.419546+0000","last_fullsized":"2024-09-21T02:03:12.419546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417576+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:03:12.417576+0000","last_peered":"2024-09-21T02:03:12.417576+0000","last_clean":"2024-09-21T02:03:12.417576+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:03:12.417576+0000","last_undegraded":"2024-09-21T02:03:12.417576+0000","last_fullsized":"2024-09-21T02:03:12.417576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:33.387995+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:03:33.387995+0000","last_peered":"2024-09-21T02:03:33.387995+0000","last_clean":"2024-09-21T02:03:33.387995+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:03:33.387995+0000","last_undegraded":"2024-09-21T02:03:33.387995+0000","last_fullsized":"2024-09-21T02:03:33.387995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423727+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:03:12.423727+0000","last_peered":"2024-09-21T02:03:12.423727+0000","last_clean":"2024-09-21T02:03:12.423727+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:03:12.423727+0000","last_undegraded":"2024-09-21T02:03:12.423727+0000","last_fullsized":"2024-09-21T02:03:12.423727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421592+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:03:12.421592+0000","last_peered":"2024-09-21T02:03:12.421592+0000","last_clean":"2024-09-21T02:03:12.421592+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:03:12.421592+0000","last_undegraded":"2024-09-21T02:03:12.421592+0000","last_fullsized":"2024-09-21T02:03:12.421592+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423973+0000","last_change":"2024-09-21T02:03:12.423870+0000","last_active":"2024-09-21T02:03:12.423973+0000","last_peered":"2024-09-21T02:03:12.423973+0000","last_clean":"2024-09-21T02:03:12.423973+0000","last_became_active":"2024-09-21T02:03:12.423441+0000","last_became_peered":"2024-09-21T02:03:12.423441+0000","last_unstale":"2024-09-21T02:03:12.423973+0000","last_undegraded":"2024-09-21T02:03:12.423973+0000","last_fullsized":"2024-09-21T02:03:12.423973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:59.071213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419481+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:03:12.419481+0000","last_peered":"2024-09-21T02:03:12.419481+0000","last_clean":"2024-09-21T02:03:12.419481+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:03:12.419481+0000","last_undegraded":"2024-09-21T02:03:12.419481+0000","last_fullsized":"2024-09-21T02:03:12.419481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422669+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:03:12.422669+0000","last_peered":"2024-09-21T02:03:12.422669+0000","last_clean":"2024-09-21T02:03:12.422669+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:03:12.422669+0000","last_undegraded":"2024-09-21T02:03:12.422669+0000","last_fullsized":"2024-09-21T02:03:12.422669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'32","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:33.387994+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:03:33.387994+0000","last_peered":"2024-09-21T02:03:33.387994+0000","last_clean":"2024-09-21T02:03:33.387994+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:03:33.387994+0000","last_undegraded":"2024-09-21T02:03:33.387994+0000","last_fullsized":"2024-09-21T02:03:33.387994+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66448,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":122,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424624+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:03:12.424624+0000","last_peered":"2024-09-21T02:03:12.424624+0000","last_clean":"2024-09-21T02:03:12.424624+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:03:12.424624+0000","last_undegraded":"2024-09-21T02:03:12.424624+0000","last_fullsized":"2024-09-21T02:03:12.424624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442392+0000","last_change":"2024-09-21T02:03:12.441759+0000","last_active":"2024-09-21T02:03:12.442392+0000","last_peered":"2024-09-21T02:03:12.442392+0000","last_clean":"2024-09-21T02:03:12.442392+0000","last_became_active":"2024-09-21T02:03:12.426827+0000","last_became_peered":"2024-09-21T02:03:12.426827+0000","last_unstale":"2024-09-21T02:03:12.442392+0000","last_undegraded":"2024-09-21T02:03:12.442392+0000","last_fullsized":"2024-09-21T02:03:12.442392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:33.917276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641411+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:03:12.641411+0000","last_peered":"2024-09-21T02:03:12.641411+0000","last_clean":"2024-09-21T02:03:12.641411+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:03:12.641411+0000","last_undegraded":"2024-09-21T02:03:12.641411+0000","last_fullsized":"2024-09-21T02:03:12.641411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418936+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:03:12.418936+0000","last_peered":"2024-09-21T02:03:12.418936+0000","last_clean":"2024-09-21T02:03:12.418936+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:03:12.418936+0000","last_undegraded":"2024-09-21T02:03:12.418936+0000","last_fullsized":"2024-09-21T02:03:12.418936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.643129+0000","last_change":"2024-09-21T02:03:12.643022+0000","last_active":"2024-09-21T02:03:12.643129+0000","last_peered":"2024-09-21T02:03:12.643129+0000","last_clean":"2024-09-21T02:03:12.643129+0000","last_became_active":"2024-09-21T02:03:12.642501+0000","last_became_peered":"2024-09-21T02:03:12.642501+0000","last_unstale":"2024-09-21T02:03:12.643129+0000","last_undegraded":"2024-09-21T02:03:12.643129+0000","last_fullsized":"2024-09-21T02:03:12.643129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:29.600445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:33.403689+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:03:33.403689+0000","last_peered":"2024-09-21T02:03:33.403689+0000","last_clean":"2024-09-21T02:03:33.403689+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:03:33.403689+0000","last_undegraded":"2024-09-21T02:03:33.403689+0000","last_fullsized":"2024-09-21T02:03:33.403689+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421158+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:03:12.421158+0000","last_peered":"2024-09-21T02:03:12.421158+0000","last_clean":"2024-09-21T02:03:12.421158+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:03:12.421158+0000","last_undegraded":"2024-09-21T02:03:12.421158+0000","last_fullsized":"2024-09-21T02:03:12.421158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423964+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:03:12.423964+0000","last_peered":"2024-09-21T02:03:12.423964+0000","last_clean":"2024-09-21T02:03:12.423964+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:03:12.423964+0000","last_undegraded":"2024-09-21T02:03:12.423964+0000","last_fullsized":"2024-09-21T02:03:12.423964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418934+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:03:12.418934+0000","last_peered":"2024-09-21T02:03:12.418934+0000","last_clean":"2024-09-21T02:03:12.418934+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:03:12.418934+0000","last_undegraded":"2024-09-21T02:03:12.418934+0000","last_fullsized":"2024-09-21T02:03:12.418934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425082+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:03:12.425082+0000","last_peered":"2024-09-21T02:03:12.425082+0000","last_clean":"2024-09-21T02:03:12.425082+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:03:12.425082+0000","last_undegraded":"2024-09-21T02:03:12.425082+0000","last_fullsized":"2024-09-21T02:03:12.425082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422136+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:03:12.422136+0000","last_peered":"2024-09-21T02:03:12.422136+0000","last_clean":"2024-09-21T02:03:12.422136+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:03:12.422136+0000","last_undegraded":"2024-09-21T02:03:12.422136+0000","last_fullsized":"2024-09-21T02:03:12.422136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641780+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:03:12.641780+0000","last_peered":"2024-09-21T02:03:12.641780+0000","last_clean":"2024-09-21T02:03:12.641780+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:03:12.641780+0000","last_undegraded":"2024-09-21T02:03:12.641780+0000","last_fullsized":"2024-09-21T02:03:12.641780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422063+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:03:12.422063+0000","last_peered":"2024-09-21T02:03:12.422063+0000","last_clean":"2024-09-21T02:03:12.422063+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:03:12.422063+0000","last_undegraded":"2024-09-21T02:03:12.422063+0000","last_fullsized":"2024-09-21T02:03:12.422063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419713+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:03:12.419713+0000","last_peered":"2024-09-21T02:03:12.419713+0000","last_clean":"2024-09-21T02:03:12.419713+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:03:12.419713+0000","last_undegraded":"2024-09-21T02:03:12.419713+0000","last_fullsized":"2024-09-21T02:03:12.419713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421761+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:03:12.421761+0000","last_peered":"2024-09-21T02:03:12.421761+0000","last_clean":"2024-09-21T02:03:12.421761+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:03:12.421761+0000","last_undegraded":"2024-09-21T02:03:12.421761+0000","last_fullsized":"2024-09-21T02:03:12.421761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":85,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:32.387046+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:03:32.387046+0000","last_peered":"2024-09-21T02:03:32.387046+0000","last_clean":"2024-09-21T02:03:32.387046+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:03:32.387046+0000","last_undegraded":"2024-09-21T02:03:32.387046+0000","last_fullsized":"2024-09-21T02:03:32.387046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421906+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:03:12.421906+0000","last_peered":"2024-09-21T02:03:12.421906+0000","last_clean":"2024-09-21T02:03:12.421906+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:03:12.421906+0000","last_undegraded":"2024-09-21T02:03:12.421906+0000","last_fullsized":"2024-09-21T02:03:12.421906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422111+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:03:12.422111+0000","last_peered":"2024-09-21T02:03:12.422111+0000","last_clean":"2024-09-21T02:03:12.422111+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:03:12.422111+0000","last_undegraded":"2024-09-21T02:03:12.422111+0000","last_fullsized":"2024-09-21T02:03:12.422111+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419020+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:03:12.419020+0000","last_peered":"2024-09-21T02:03:12.419020+0000","last_clean":"2024-09-21T02:03:12.419020+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:03:12.419020+0000","last_undegraded":"2024-09-21T02:03:12.419020+0000","last_fullsized":"2024-09-21T02:03:12.419020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424850+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:03:12.424850+0000","last_peered":"2024-09-21T02:03:12.424850+0000","last_clean":"2024-09-21T02:03:12.424850+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:03:12.424850+0000","last_undegraded":"2024-09-21T02:03:12.424850+0000","last_fullsized":"2024-09-21T02:03:12.424850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421843+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:03:12.421843+0000","last_peered":"2024-09-21T02:03:12.421843+0000","last_clean":"2024-09-21T02:03:12.421843+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:03:12.421843+0000","last_undegraded":"2024-09-21T02:03:12.421843+0000","last_fullsized":"2024-09-21T02:03:12.421843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641320+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:03:12.641320+0000","last_peered":"2024-09-21T02:03:12.641320+0000","last_clean":"2024-09-21T02:03:12.641320+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:03:12.641320+0000","last_undegraded":"2024-09-21T02:03:12.641320+0000","last_fullsized":"2024-09-21T02:03:12.641320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:32.401262+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:03:32.401262+0000","last_peered":"2024-09-21T02:03:32.401262+0000","last_clean":"2024-09-21T02:03:32.401262+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:03:32.401262+0000","last_undegraded":"2024-09-21T02:03:32.401262+0000","last_fullsized":"2024-09-21T02:03:32.401262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419308+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:03:12.419308+0000","last_peered":"2024-09-21T02:03:12.419308+0000","last_clean":"2024-09-21T02:03:12.419308+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:03:12.419308+0000","last_undegraded":"2024-09-21T02:03:12.419308+0000","last_fullsized":"2024-09-21T02:03:12.419308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:31.385645+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:03:31.385645+0000","last_peered":"2024-09-21T02:03:31.385645+0000","last_clean":"2024-09-21T02:03:31.385645+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:03:31.385645+0000","last_undegraded":"2024-09-21T02:03:31.385645+0000","last_fullsized":"2024-09-21T02:03:31.385645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425206+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:03:12.425206+0000","last_peered":"2024-09-21T02:03:12.425206+0000","last_clean":"2024-09-21T02:03:12.425206+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:03:12.425206+0000","last_undegraded":"2024-09-21T02:03:12.425206+0000","last_fullsized":"2024-09-21T02:03:12.425206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417918+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:03:12.417918+0000","last_peered":"2024-09-21T02:03:12.417918+0000","last_clean":"2024-09-21T02:03:12.417918+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:03:12.417918+0000","last_undegraded":"2024-09-21T02:03:12.417918+0000","last_fullsized":"2024-09-21T02:03:12.417918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424809+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:03:12.424809+0000","last_peered":"2024-09-21T02:03:12.424809+0000","last_clean":"2024-09-21T02:03:12.424809+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:03:12.424809+0000","last_undegraded":"2024-09-21T02:03:12.424809+0000","last_fullsized":"2024-09-21T02:03:12.424809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425092+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:03:12.425092+0000","last_peered":"2024-09-21T02:03:12.425092+0000","last_clean":"2024-09-21T02:03:12.425092+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:03:12.425092+0000","last_undegraded":"2024-09-21T02:03:12.425092+0000","last_fullsized":"2024-09-21T02:03:12.425092+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641740+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:03:12.641740+0000","last_peered":"2024-09-21T02:03:12.641740+0000","last_clean":"2024-09-21T02:03:12.641740+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:03:12.641740+0000","last_undegraded":"2024-09-21T02:03:12.641740+0000","last_fullsized":"2024-09-21T02:03:12.641740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417868+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:03:12.417868+0000","last_peered":"2024-09-21T02:03:12.417868+0000","last_clean":"2024-09-21T02:03:12.417868+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:03:12.417868+0000","last_undegraded":"2024-09-21T02:03:12.417868+0000","last_fullsized":"2024-09-21T02:03:12.417868+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424450+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:03:12.424450+0000","last_peered":"2024-09-21T02:03:12.424450+0000","last_clean":"2024-09-21T02:03:12.424450+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:03:12.424450+0000","last_undegraded":"2024-09-21T02:03:12.424450+0000","last_fullsized":"2024-09-21T02:03:12.424450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419259+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:03:12.419259+0000","last_peered":"2024-09-21T02:03:12.419259+0000","last_clean":"2024-09-21T02:03:12.419259+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:03:12.419259+0000","last_undegraded":"2024-09-21T02:03:12.419259+0000","last_fullsized":"2024-09-21T02:03:12.419259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421030+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:03:12.421030+0000","last_peered":"2024-09-21T02:03:12.421030+0000","last_clean":"2024-09-21T02:03:12.421030+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:03:12.421030+0000","last_undegraded":"2024-09-21T02:03:12.421030+0000","last_fullsized":"2024-09-21T02:03:12.421030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422234+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:03:12.422234+0000","last_peered":"2024-09-21T02:03:12.422234+0000","last_clean":"2024-09-21T02:03:12.422234+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:03:12.422234+0000","last_undegraded":"2024-09-21T02:03:12.422234+0000","last_fullsized":"2024-09-21T02:03:12.422234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423901+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:03:12.423901+0000","last_peered":"2024-09-21T02:03:12.423901+0000","last_clean":"2024-09-21T02:03:12.423901+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:03:12.423901+0000","last_undegraded":"2024-09-21T02:03:12.423901+0000","last_fullsized":"2024-09-21T02:03:12.423901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424628+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:03:12.424628+0000","last_peered":"2024-09-21T02:03:12.424628+0000","last_clean":"2024-09-21T02:03:12.424628+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:03:12.424628+0000","last_undegraded":"2024-09-21T02:03:12.424628+0000","last_fullsized":"2024-09-21T02:03:12.424628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421779+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:03:12.421779+0000","last_peered":"2024-09-21T02:03:12.421779+0000","last_clean":"2024-09-21T02:03:12.421779+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:03:12.421779+0000","last_undegraded":"2024-09-21T02:03:12.421779+0000","last_fullsized":"2024-09-21T02:03:12.421779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426547+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:03:12.426547+0000","last_peered":"2024-09-21T02:03:12.426547+0000","last_clean":"2024-09-21T02:03:12.426547+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:03:12.426547+0000","last_undegraded":"2024-09-21T02:03:12.426547+0000","last_fullsized":"2024-09-21T02:03:12.426547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424421+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:03:12.424421+0000","last_peered":"2024-09-21T02:03:12.424421+0000","last_clean":"2024-09-21T02:03:12.424421+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:03:12.424421+0000","last_undegraded":"2024-09-21T02:03:12.424421+0000","last_fullsized":"2024-09-21T02:03:12.424421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.446014+0000","last_change":"2024-09-21T02:03:12.445516+0000","last_active":"2024-09-21T02:03:12.446014+0000","last_peered":"2024-09-21T02:03:12.446014+0000","last_clean":"2024-09-21T02:03:12.446014+0000","last_became_active":"2024-09-21T02:03:12.430149+0000","last_became_peered":"2024-09-21T02:03:12.430149+0000","last_unstale":"2024-09-21T02:03:12.446014+0000","last_undegraded":"2024-09-21T02:03:12.446014+0000","last_fullsized":"2024-09-21T02:03:12.446014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:23.573018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421274+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:03:12.421274+0000","last_peered":"2024-09-21T02:03:12.421274+0000","last_clean":"2024-09-21T02:03:12.421274+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:03:12.421274+0000","last_undegraded":"2024-09-21T02:03:12.421274+0000","last_fullsized":"2024-09-21T02:03:12.421274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.643058+0000","last_change":"2024-09-21T02:03:12.642935+0000","last_active":"2024-09-21T02:03:12.643058+0000","last_peered":"2024-09-21T02:03:12.643058+0000","last_clean":"2024-09-21T02:03:12.643058+0000","last_became_active":"2024-09-21T02:03:12.642520+0000","last_became_peered":"2024-09-21T02:03:12.642520+0000","last_unstale":"2024-09-21T02:03:12.643058+0000","last_undegraded":"2024-09-21T02:03:12.643058+0000","last_fullsized":"2024-09-21T02:03:12.643058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:32.147679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424736+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:03:12.424736+0000","last_peered":"2024-09-21T02:03:12.424736+0000","last_clean":"2024-09-21T02:03:12.424736+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:03:12.424736+0000","last_undegraded":"2024-09-21T02:03:12.424736+0000","last_fullsized":"2024-09-21T02:03:12.424736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422588+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:03:12.422588+0000","last_peered":"2024-09-21T02:03:12.422588+0000","last_clean":"2024-09-21T02:03:12.422588+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:03:12.422588+0000","last_undegraded":"2024-09-21T02:03:12.422588+0000","last_fullsized":"2024-09-21T02:03:12.422588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.441500+0000","last_change":"2024-09-21T02:03:12.441361+0000","last_active":"2024-09-21T02:03:12.441500+0000","last_peered":"2024-09-21T02:03:12.441500+0000","last_clean":"2024-09-21T02:03:12.441500+0000","last_became_active":"2024-09-21T02:03:12.428877+0000","last_became_peered":"2024-09-21T02:03:12.428877+0000","last_unstale":"2024-09-21T02:03:12.441500+0000","last_undegraded":"2024-09-21T02:03:12.441500+0000","last_fullsized":"2024-09-21T02:03:12.441500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.464134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421013+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:03:12.421013+0000","last_peered":"2024-09-21T02:03:12.421013+0000","last_clean":"2024-09-21T02:03:12.421013+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:03:12.421013+0000","last_undegraded":"2024-09-21T02:03:12.421013+0000","last_fullsized":"2024-09-21T02:03:12.421013+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424866+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:03:12.424866+0000","last_peered":"2024-09-21T02:03:12.424866+0000","last_clean":"2024-09-21T02:03:12.424866+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:03:12.424866+0000","last_undegraded":"2024-09-21T02:03:12.424866+0000","last_fullsized":"2024-09-21T02:03:12.424866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424636+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:03:12.424636+0000","last_peered":"2024-09-21T02:03:12.424636+0000","last_clean":"2024-09-21T02:03:12.424636+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:03:12.424636+0000","last_undegraded":"2024-09-21T02:03:12.424636+0000","last_fullsized":"2024-09-21T02:03:12.424636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":88,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:31.385033+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:03:31.385033+0000","last_peered":"2024-09-21T02:03:31.385033+0000","last_clean":"2024-09-21T02:03:31.385033+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:03:31.385033+0000","last_undegraded":"2024-09-21T02:03:31.385033+0000","last_fullsized":"2024-09-21T02:03:31.385033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74434,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":364,"num_write":249,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":68,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":860160,"data_stored":406846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1560576,"data_stored":1064255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":876544,"data_stored":406317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":872448,"data_stored":406695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1470464,"data_stored":999995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":860160,"data_stored":406425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1482752,"data_stored":998891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":876544,"data_stored":405859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":474329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":929792,"data_stored":473786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":880640,"data_stored":407764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:03:38.389 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T02:03:38.389 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T02:03:38.389 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T02:03:38.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T02:03:38.632 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:39.012 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:38 smithi092 ceph-mon[31190]: pgmap v320: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:39.012 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:38 smithi092 ceph-mon[31190]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:03:39.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:38 smithi136 ceph-mon[34754]: pgmap v320: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:39.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:38 smithi136 ceph-mon[34754]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:03:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:38 smithi099 ceph-mon[34404]: pgmap v320: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:38 smithi099 ceph-mon[34404]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 -- 172.21.15.92:0/86511366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 msgr2=0x7fd950103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 --2- 172.21.15.92:0/86511366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd950103f70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd940009ca0 tx=0x7fd94002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 -- 172.21.15.92:0/86511366 shutdown_connections 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 --2- 172.21.15.92:0/86511366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd950109df0 0x7fd950110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 --2- 172.21.15.92:0/86511366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9501094a0 0x7fd9501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 --2- 172.21.15.92:0/86511366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd950103f70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 -- 172.21.15.92:0/86511366 >> 172.21.15.92:0/86511366 conn(0x7fd95007b850 msgr2=0x7fd95007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:39.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 -- 172.21.15.92:0/86511366 shutdown_connections 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.098+0000 7fd958626640 1 -- 172.21.15.92:0/86511366 wait complete. 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 Processor -- start 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 -- start start 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd95019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9501094a0 0x7fd95019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd950109df0 0x7fd95019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd950113b60 con 0x7fd950103b60 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501139e0 con 0x7fd950109df0 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd958626640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd950113ce0 con 0x7fd9501094a0 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.099+0000 7fd9563c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd95019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd95019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44408/0 (socket says 172.21.15.92:44408) 2024-09-21T02:03:39.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 -- 172.21.15.92:0/3917777087 learned_addr learned my addr 172.21.15.92:0/3917777087 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd955bc1640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9501094a0 0x7fd95019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 -- 172.21.15.92:0/3917777087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9501094a0 msgr2=0x7fd95019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9501094a0 0x7fd95019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd956bc3640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd950109df0 0x7fd95019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 -- 172.21.15.92:0/3917777087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd950109df0 msgr2=0x7fd95019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd950109df0 0x7fd95019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 -- 172.21.15.92:0/3917777087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501a04e0 con 0x7fd950103b60 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd956bc3640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd950109df0 0x7fd95019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd9563c2640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd95019a570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd94002aeb0 tx=0x7fd940003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940003e20 con 0x7fd950103b60 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501a0770 con 0x7fd950103b60 2024-09-21T02:03:39.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9400044f0 con 0x7fd950103b60 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94002fdd0 con 0x7fd950103b60 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.100+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a6f30 con 0x7fd950103b60 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.101+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950103f70 con 0x7fd950103b60 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.103+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd940004040 con 0x7fd950103b60 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.104+0000 7fd94f7fe640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9380b65a0 0x7fd9380b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.104+0000 7fd955bc1640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9380b65a0 0x7fd9380b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.104+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd940108c30 con 0x7fd950103b60 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.104+0000 7fd955bc1640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9380b65a0 0x7fd9380b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9501091a0 tx=0x7fd94400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:39.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.107+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9400cf340 con 0x7fd950103b60 2024-09-21T02:03:39.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.326+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd95002e640 con 0x7fd950103b60 2024-09-21T02:03:39.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.326+0000 7fd94f7fe640 1 -- 172.21.15.92:0/3917777087 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd9400d3720 con 0x7fd950103b60 2024-09-21T02:03:39.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:39.327 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T02:03:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9380b65a0 msgr2=0x7fd9380b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9380b65a0 0x7fd9380b8ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9501091a0 tx=0x7fd94400a4c0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 msgr2=0x7fd95019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd95019a570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd94002aeb0 tx=0x7fd940003970 comp rx=0 tx=0).stop 2024-09-21T02:03:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 shutdown_connections 2024-09-21T02:03:39.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9380b65a0 0x7fd9380b8ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd950109df0 0x7fd95019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9501094a0 0x7fd95019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 --2- 172.21.15.92:0/3917777087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd950103b60 0x7fd95019a570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 >> 172.21.15.92:0/3917777087 conn(0x7fd95007b850 msgr2=0x7fd950107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 shutdown_connections 2024-09-21T02:03:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:39.328+0000 7fd958626640 1 -- 172.21.15.92:0/3917777087 wait complete. 2024-09-21T02:03:39.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:39 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3917777087' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:03:39.951 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T02:03:39.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:03:40.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:39 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3917777087' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:03:40.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:39 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3917777087' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:03:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 -- 172.21.15.92:0/603932493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c1083c0 msgr2=0x7f753c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 --2- 172.21.15.92:0/603932493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c1083c0 0x7f753c1087d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f752c009900 tx=0x7f752c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 -- 172.21.15.92:0/603932493 shutdown_connections 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 --2- 172.21.15.92:0/603932493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f753c108d10 0x7f753c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 --2- 172.21.15.92:0/603932493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c1083c0 0x7f753c1087d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 --2- 172.21.15.92:0/603932493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c102a80 0x7f753c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 -- 172.21.15.92:0/603932493 >> 172.21.15.92:0/603932493 conn(0x7f753c079740 msgr2=0x7f753c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 -- 172.21.15.92:0/603932493 shutdown_connections 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 -- 172.21.15.92:0/603932493 wait complete. 2024-09-21T02:03:40.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 Processor -- start 2024-09-21T02:03:40.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.643+0000 7f7543315640 1 -- start start 2024-09-21T02:03:40.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 0x7f753c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f753c1083c0 0x7f753c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c108d10 0x7f753c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f753c112980 con 0x7f753c102a80 2024-09-21T02:03:40.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f753c112800 con 0x7f753c108d10 2024-09-21T02:03:40.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f753c112b00 con 0x7f753c1083c0 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 0x7f753c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 0x7f753c19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44436/0 (socket says 172.21.15.92:44436) 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 -- 172.21.15.92:0/3485870238 learned_addr learned my addr 172.21.15.92:0/3485870238 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75408b0640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f753c1083c0 0x7f753c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75418b2640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c108d10 0x7f753c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 -- 172.21.15.92:0/3485870238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f753c1083c0 msgr2=0x7f753c19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f753c1083c0 0x7f753c19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 -- 172.21.15.92:0/3485870238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c108d10 msgr2=0x7f753c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c108d10 0x7f753c19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 -- 172.21.15.92:0/3485870238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c1990c0 con 0x7f753c102a80 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75418b2640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c108d10 0x7f753c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75410b1640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 0x7f753c19b2d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f753800cce0 tx=0x7f753800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538004b20 con 0x7f753c102a80 2024-09-21T02:03:40.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.644+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c1993b0 con 0x7f753c102a80 2024-09-21T02:03:40.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.645+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7538004cc0 con 0x7f753c102a80 2024-09-21T02:03:40.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.645+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753800f6b0 con 0x7f753c102a80 2024-09-21T02:03:40.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.645+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1998c0 con 0x7f753c102a80 2024-09-21T02:03:40.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.645+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c103d00 con 0x7f753c102a80 2024-09-21T02:03:40.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.647+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f753800f850 con 0x7f753c102a80 2024-09-21T02:03:40.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.648+0000 7f75327fc640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75280b65d0 0x7f75280b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:40.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.648+0000 7f75408b0640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75280b65d0 0x7f75280b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:40.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.648+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f75380e78e0 con 0x7f753c102a80 2024-09-21T02:03:40.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.648+0000 7f75408b0640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75280b65d0 0x7f75280b8ae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f752c000c00 tx=0x7f752c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:40.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.651+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7538014030 con 0x7f753c102a80 2024-09-21T02:03:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.865+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753c102e90 con 0x7f753c102a80 2024-09-21T02:03:40.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.866+0000 7f75327fc640 1 -- 172.21.15.92:0/3485870238 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11963 (secure 0 0 0) 0x7f75380adff0 con 0x7f753c102a80 2024-09-21T02:03:40.867 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:40.867 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"btime":"2024-09-21T02:02:43:311503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:02:42.638635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[14799,24610,24623,24619,24637]},"id":1}]} 2024-09-21T02:03:40.867 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:03:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75280b65d0 msgr2=0x7f75280b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75280b65d0 0x7f75280b8ae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f752c000c00 tx=0x7f752c031400 comp rx=0 tx=0).stop 2024-09-21T02:03:40.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 msgr2=0x7f753c19b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 0x7f753c19b2d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f753800cce0 tx=0x7f753800a620 comp rx=0 tx=0).stop 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 shutdown_connections 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75280b65d0 0x7f75280b8ae0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f753c108d10 0x7f753c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f753c1083c0 0x7f753c19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 --2- 172.21.15.92:0/3485870238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753c102a80 0x7f753c19b2d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.868+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 >> 172.21.15.92:0/3485870238 conn(0x7f753c079740 msgr2=0x7f753c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.869+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 shutdown_connections 2024-09-21T02:03:40.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:40.869+0000 7f7543315640 1 -- 172.21.15.92:0/3485870238 wait complete. 2024-09-21T02:03:40.879 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:40 smithi092 ceph-mon[31190]: pgmap v321: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:03:41.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:40 smithi136 ceph-mon[34754]: pgmap v321: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:03:41.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:40 smithi099 ceph-mon[34404]: pgmap v321: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:03:41.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:03:41.799 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:41.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:41 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3485870238' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:42.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:41 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3485870238' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:42.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:41 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3485870238' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:42.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 -- 172.21.15.92:0/2543815800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f24108cf0 msgr2=0x7f9f2410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/2543815800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f24108cf0 0x7f9f2410f520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009ca0 tx=0x7f9f1c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 -- 172.21.15.92:0/2543815800 shutdown_connections 2024-09-21T02:03:42.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/2543815800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f24108cf0 0x7f9f2410f520 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/2543815800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f241083a0 0x7f9f241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/2543815800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f24102a60 0x7f9f24102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 -- 172.21.15.92:0/2543815800 >> 172.21.15.92:0/2543815800 conn(0x7f9f24079730 msgr2=0x7f9f24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 -- 172.21.15.92:0/2543815800 shutdown_connections 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.238+0000 7f9f2b70d640 1 -- 172.21.15.92:0/2543815800 wait complete. 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 Processor -- start 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 -- start start 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f24102a60 0x7f9f24109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 0x7f9f24109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f24108cf0 0x7f9f2410a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24112960 con 0x7f9f241083a0 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241127e0 con 0x7f9f24102a60 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f2b70d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24112ae0 con 0x7f9f24108cf0 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f28ca8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 0x7f9f24109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f28ca8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 0x7f9f24109b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44452/0 (socket says 172.21.15.92:44452) 2024-09-21T02:03:42.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f28ca8640 1 -- 172.21.15.92:0/1878143994 learned_addr learned my addr 172.21.15.92:0/1878143994 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.239+0000 7f9f294a9640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f24102a60 0x7f9f24109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f29caa640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f24108cf0 0x7f9f2410a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f28ca8640 1 -- 172.21.15.92:0/1878143994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f24108cf0 msgr2=0x7f9f2410a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f28ca8640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f24108cf0 0x7f9f2410a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f28ca8640 1 -- 172.21.15.92:0/1878143994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f24102a60 msgr2=0x7f9f24109610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f28ca8640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f24102a60 0x7f9f24109610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f28ca8640 1 -- 172.21.15.92:0/1878143994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a8610 con 0x7f9f241083a0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f29caa640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f24108cf0 0x7f9f2410a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f28ca8640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 0x7f9f24109b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000d740 tx=0x7f9f2000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20017070 con 0x7f9f241083a0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a8900 con 0x7f9f241083a0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f2000de50 con 0x7f9f241083a0 2024-09-21T02:03:42.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20004ce0 con 0x7f9f241083a0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.240+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a8e40 con 0x7f9f241083a0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.241+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24102e70 con 0x7f9f241083a0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.244+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9f20004e80 con 0x7f9f241083a0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.245+0000 7f9f1a7fc640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9f100b65a0 0x7f9f100b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.245+0000 7f9f294a9640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9f100b65a0 0x7f9f100b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.245+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9f200e7330 con 0x7f9f241083a0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.245+0000 7f9f294a9640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9f100b65a0 0x7f9f100b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005e10 tx=0x7f9f14005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:42.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.248+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f200ada40 con 0x7f9f241083a0 2024-09-21T02:03:42.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.466+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f2410a910 con 0x7f9f241083a0 2024-09-21T02:03:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.467+0000 7f9f1a7fc640 1 -- 172.21.15.92:0/1878143994 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11963 (secure 0 0 0) 0x7f9f200b1e20 con 0x7f9f241083a0 2024-09-21T02:03:42.468 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:42.468 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"btime":"2024-09-21T02:02:43:311503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:02:42.638635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[14799,24610,24623,24619,24637]},"id":1}]} 2024-09-21T02:03:42.468 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9f100b65a0 msgr2=0x7f9f100b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9f100b65a0 0x7f9f100b8ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005e10 tx=0x7f9f14005da0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 msgr2=0x7f9f24109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 0x7f9f24109b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000d740 tx=0x7f9f2000dc00 comp rx=0 tx=0).stop 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 shutdown_connections 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9f100b65a0 0x7f9f100b8ab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f24108cf0 0x7f9f2410a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f241083a0 0x7f9f24109b50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 --2- 172.21.15.92:0/1878143994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f24102a60 0x7f9f24109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 >> 172.21.15.92:0/1878143994 conn(0x7f9f24079730 msgr2=0x7f9f2406b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 shutdown_connections 2024-09-21T02:03:42.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:42.469+0000 7f9f2b70d640 1 -- 172.21.15.92:0/1878143994 wait complete. 2024-09-21T02:03:43.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:42 smithi136 ceph-mon[34754]: pgmap v322: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:43.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:42 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1878143994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:43.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:42 smithi092 ceph-mon[31190]: pgmap v322: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:43.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:42 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1878143994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:43.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:42 smithi099 ceph-mon[34404]: pgmap v322: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:43.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:42 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1878143994' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:43.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:03:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 -- 172.21.15.92:0/1944300893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102a60 msgr2=0x7fd8ac102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/1944300893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102a60 0x7fd8ac102e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009900 tx=0x7fd8a002a410 comp rx=0 tx=0).stop 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 -- 172.21.15.92:0/1944300893 shutdown_connections 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/1944300893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/1944300893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ac1083a0 0x7fd8ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/1944300893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac102a60 0x7fd8ac102e70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 -- 172.21.15.92:0/1944300893 >> 172.21.15.92:0/1944300893 conn(0x7fd8ac079730 msgr2=0x7fd8ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 -- 172.21.15.92:0/1944300893 shutdown_connections 2024-09-21T02:03:43.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 -- 172.21.15.92:0/1944300893 wait complete. 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 Processor -- start 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.863+0000 7fd8b2e91640 1 -- start start 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ac102a60 0x7fd8ac104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac1083a0 0x7fd8ac104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac1129c0 con 0x7fd8ac108cf0 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112840 con 0x7fd8ac1083a0 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112b40 con 0x7fd8ac102a60 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34736/0 (socket says 172.21.15.92:34736) 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 -- 172.21.15.92:0/2980057221 learned_addr learned my addr 172.21.15.92:0/2980057221 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b0c2d640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ac102a60 0x7fd8ac104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:43.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8abfff640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac1083a0 0x7fd8ac104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 -- 172.21.15.92:0/2980057221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ac102a60 msgr2=0x7fd8ac104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ac102a60 0x7fd8ac104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 -- 172.21.15.92:0/2980057221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac1083a0 msgr2=0x7fd8ac104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac1083a0 0x7fd8ac104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 -- 172.21.15.92:0/2980057221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac1a87a0 con 0x7fd8ac108cf0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8abfff640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac1083a0 0x7fd8ac104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b142e640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac104cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b970 tx=0x7fd8a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4019070 con 0x7fd8ac108cf0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a4014440 con 0x7fd8ac108cf0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40075d0 con 0x7fd8ac108cf0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.864+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac1a8a90 con 0x7fd8ac108cf0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.865+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1a9050 con 0x7fd8ac108cf0 2024-09-21T02:03:43.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.866+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac102e70 con 0x7fd8ac108cf0 2024-09-21T02:03:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.867+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fd8a401c070 con 0x7fd8ac108cf0 2024-09-21T02:03:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.868+0000 7fd8a9ffb640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd89c0b6620 0x7fd89c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.868+0000 7fd8b0c2d640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd89c0b6620 0x7fd89c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.868+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd8a40e69e0 con 0x7fd8ac108cf0 2024-09-21T02:03:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.868+0000 7fd8b0c2d640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd89c0b6620 0x7fd89c0b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c00 tx=0x7fd8a0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:43.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:43.872+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd8a40ad0f0 con 0x7fd8ac108cf0 2024-09-21T02:03:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.054+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8ac1a9480 con 0x7fd8ac108cf0 2024-09-21T02:03:44.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.056+0000 7fd8a9ffb640 1 -- 172.21.15.92:0/2980057221 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22797 (secure 0 0 0) 0x7fd8a40b14d0 con 0x7fd8ac108cf0 2024-09-21T02:03:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:44.057 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:03:12.414418+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:44.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd89c0b6620 msgr2=0x7fd89c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd89c0b6620 0x7fd89c0b8b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c00 tx=0x7fd8a0031400 comp rx=0 tx=0).stop 2024-09-21T02:03:44.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 msgr2=0x7fd8ac104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac104cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400b970 tx=0x7fd8a400be30 comp rx=0 tx=0).stop 2024-09-21T02:03:44.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 shutdown_connections 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd89c0b6620 0x7fd89c0b8b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8ac108cf0 0x7fd8ac104cd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8ac1083a0 0x7fd8ac104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 --2- 172.21.15.92:0/2980057221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8ac102a60 0x7fd8ac104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.058+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 >> 172.21.15.92:0/2980057221 conn(0x7fd8ac079730 msgr2=0x7fd8ac101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.059+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 shutdown_connections 2024-09-21T02:03:44.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:44.059+0000 7fd8b2e91640 1 -- 172.21.15.92:0/2980057221 wait complete. 2024-09-21T02:03:44.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:45.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:44 smithi136 ceph-mon[34754]: pgmap v323: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:45.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2980057221' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:45.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:44 smithi092 ceph-mon[31190]: pgmap v323: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:45.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2980057221' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:45.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:44 smithi099 ceph-mon[34404]: pgmap v323: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:45.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2980057221' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 -- 172.21.15.92:0/1226498869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f924410ddf0 msgr2=0x7f9244110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 --2- 172.21.15.92:0/1226498869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f924410ddf0 0x7f9244110300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9234009ca0 tx=0x7f923402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 -- 172.21.15.92:0/1226498869 shutdown_connections 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 --2- 172.21.15.92:0/1226498869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f924410ddf0 0x7f9244110300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 --2- 172.21.15.92:0/1226498869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9244105270 0x7f924410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 --2- 172.21.15.92:0/1226498869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9244104840 0x7f9244104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 -- 172.21.15.92:0/1226498869 >> 172.21.15.92:0/1226498869 conn(0x7f9244079730 msgr2=0x7f9244079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 -- 172.21.15.92:0/1226498869 shutdown_connections 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.394+0000 7f924ae21640 1 -- 172.21.15.92:0/1226498869 wait complete. 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 Processor -- start 2024-09-21T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 -- start start 2024-09-21T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 0x7f924419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9244105270 0x7f924419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f924410ddf0 0x7f924419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9244113a70 con 0x7f9244104840 2024-09-21T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92441138f0 con 0x7f9244105270 2024-09-21T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f924ae21640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9244113bf0 con 0x7f924410ddf0 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f9248bbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 0x7f924419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f9248bbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 0x7f924419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34766/0 (socket says 172.21.15.92:34766) 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f9248bbd640 1 -- 172.21.15.92:0/605893085 learned_addr learned my addr 172.21.15.92:0/605893085 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.395+0000 7f9243fff640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9244105270 0x7f924419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f92493be640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f924410ddf0 0x7f924419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9248bbd640 1 -- 172.21.15.92:0/605893085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f924410ddf0 msgr2=0x7f924419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9248bbd640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f924410ddf0 0x7f924419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9248bbd640 1 -- 172.21.15.92:0/605893085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9244105270 msgr2=0x7f924419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9248bbd640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9244105270 0x7f924419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9248bbd640 1 -- 172.21.15.92:0/605893085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441a03d0 con 0x7f9244104840 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f92493be640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f924410ddf0 0x7f924419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9248bbd640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 0x7f924419a460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c970 tx=0x7f923c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c004220 con 0x7f9244104840 2024-09-21T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92441a06c0 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f923c0043c0 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c00f450 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.396+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441a6e20 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.397+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244104c50 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.400+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f923c003610 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.401+0000 7f9241ffb640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92300b65a0 0x7f92300b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.401+0000 7f9243fff640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92300b65a0 0x7f92300b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.401+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f923c0e7670 con 0x7f9244104840 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.401+0000 7f9243fff640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92300b65a0 0x7f92300b8ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f924410faf0 tx=0x7f922c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.404+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f923c014030 con 0x7f9244104840 2024-09-21T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.619+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f924419b590 con 0x7f9244104840 2024-09-21T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.620+0000 7f9241ffb640 1 -- 172.21.15.92:0/605893085 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11963 (secure 0 0 0) 0x7f923c0add80 con 0x7f9244104840 2024-09-21T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"btime":"2024-09-21T02:02:43:311503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:02:42.638635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[14799,24610,24623,24619,24637]},"id":1}]} 2024-09-21T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92300b65a0 msgr2=0x7f92300b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92300b65a0 0x7f92300b8ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f924410faf0 tx=0x7f922c00b390 comp rx=0 tx=0).stop 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 msgr2=0x7f924419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 0x7f924419a460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f923c00c970 tx=0x7f923c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 shutdown_connections 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92300b65a0 0x7f92300b8ab0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f924410ddf0 0x7f924419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9244105270 0x7f924419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 --2- 172.21.15.92:0/605893085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9244104840 0x7f924419a460 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 >> 172.21.15.92:0/605893085 conn(0x7f9244079730 msgr2=0x7f92440ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 shutdown_connections 2024-09-21T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:45.622+0000 7f924ae21640 1 -- 172.21.15.92:0/605893085 wait complete. 2024-09-21T02:03:46.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:45 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/605893085' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:46.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:45 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/605893085' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:46.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:45 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/605893085' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:03:46.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T02:02:31.545262+0000', 'modified': '2024-09-21T02:02:42.638635+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24637, 'mds_1': 24623, 'mds_2': 24619, 'mds_3': 24610, 'mds_4': 14799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14793': {'gid': 14793, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6839/4068035941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 4068035941}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 4068035941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14799': {'gid': 14799, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6841/2220276047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6840', 'nonce': 2220276047}, {'type': 'v1', 'addr': '172.21.15.92:6841', 'nonce': 2220276047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/3451484092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 3451484092}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 3451484092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24572': {'gid': 24572, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/2004123702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2004123702}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2004123702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24574': {'gid': 24574, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/2879720066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 2879720066}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 2879720066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24593': {'gid': 24593, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6837/3685475120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3685475120}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3685475120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6835/2933531534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 2933531534}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2933531534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24619': {'gid': 24619, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.136:6837/3734933147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 3734933147}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 3734933147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24623': {'gid': 24623, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.99:6839/2904178656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 2904178656}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 2904178656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.136:6839/419135468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 419135468}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 419135468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24637, 'qdb_cluster': [14799, 24610, 24623, 24619, 24637]} 2024-09-21T02:03:46.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T02:03:46.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T02:03:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:46.962 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:46 smithi092 ceph-mon[31190]: pgmap v324: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:46.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 -- 172.21.15.92:0/705403068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c10de50 msgr2=0x7f946c1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 --2- 172.21.15.92:0/705403068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c10de50 0x7f946c1102f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9464009ca0 tx=0x7f946402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 -- 172.21.15.92:0/705403068 shutdown_connections 2024-09-21T02:03:46.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 --2- 172.21.15.92:0/705403068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c10de50 0x7f946c1102f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 --2- 172.21.15.92:0/705403068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c1052d0 0x7f946c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 --2- 172.21.15.92:0/705403068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f946c1048a0 0x7f946c104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 -- 172.21.15.92:0/705403068 >> 172.21.15.92:0/705403068 conn(0x7f946c079730 msgr2=0x7f946c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 -- 172.21.15.92:0/705403068 shutdown_connections 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.960+0000 7f9471b91640 1 -- 172.21.15.92:0/705403068 wait complete. 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 Processor -- start 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 -- start start 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f946c1048a0 0x7f946c1027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 0x7f946c102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c10de50 0x7f946c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113a60 con 0x7f946c1052d0 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1138e0 con 0x7f946c10de50 2024-09-21T02:03:46.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f9471b91640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113be0 con 0x7f946c1048a0 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f946affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 0x7f946c102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f946affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 0x7f946c102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34780/0 (socket says 172.21.15.92:34780) 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.961+0000 7f946affd640 1 -- 172.21.15.92:0/4213994898 learned_addr learned my addr 172.21.15.92:0/4213994898 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946b7fe640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f946c1048a0 0x7f946c1027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946bfff640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c10de50 0x7f946c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946affd640 1 -- 172.21.15.92:0/4213994898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f946c1048a0 msgr2=0x7f946c1027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946affd640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f946c1048a0 0x7f946c1027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946affd640 1 -- 172.21.15.92:0/4213994898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c10de50 msgr2=0x7f946c103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946affd640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c10de50 0x7f946c103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946affd640 1 -- 172.21.15.92:0/4213994898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c1a86d0 con 0x7f946c1052d0 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f946affd640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 0x7f946c102cf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f945400c970 tx=0x7f945400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f9468ff9640 1 -- 172.21.15.92:0/4213994898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454007bb0 con 0x7f946c1052d0 2024-09-21T02:03:46.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c1a89c0 con 0x7f946c1052d0 2024-09-21T02:03:46.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f9468ff9640 1 -- 172.21.15.92:0/4213994898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454007d50 con 0x7f946c1052d0 2024-09-21T02:03:46.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f9468ff9640 1 -- 172.21.15.92:0/4213994898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945400f450 con 0x7f946c1052d0 2024-09-21T02:03:46.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.962+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c1a8f50 con 0x7f946c1052d0 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.965+0000 7f9468ff9640 1 -- 172.21.15.92:0/4213994898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f9454004ec0 con 0x7f946c1052d0 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 Processor -- start 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 -- start start 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a33c13cc10 con 0x55a33bdf60a0 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a33c12c040 con 0x55a33bdfe280 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9478105740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a33c12fe50 con 0x55a33be1ef60 2024-09-21T02:03:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f946b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f946b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34794/0 (socket says 172.21.15.92:34794) 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f946b7fe640 1 -- 172.21.15.92:0/3633928444 learned_addr learned my addr 172.21.15.92:0/3633928444 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f9468ff9640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f94580b65a0 0x7f94580b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f946affd640 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.966+0000 7f946bfff640 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946bfff640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f94580b65a0 0x7f94580b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946b7fe640 1 -- 172.21.15.92:0/3633928444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 msgr2=0x55a33c1c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946b7fe640 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946b7fe640 1 -- 172.21.15.92:0/3633928444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 msgr2=0x55a33c1c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f9468ff9640 1 -- 172.21.15.92:0/4213994898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f94540e71d0 con 0x7f946c1052d0 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946b7fe640 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946b7fe640 1 -- 172.21.15.92:0/3633928444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33be8a540 con 0x55a33bdf60a0 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946affd640 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946b7fe640 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1c5440 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f945c00bc70 tx=0x7f945c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e01b9450e4f859e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f946bfff640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f94580b65a0 0x7f94580b8ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f946402f500 tx=0x7f94640095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f9461ffb640 1 -- 172.21.15.92:0/3633928444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c01c480 con 0x55a33bdf60a0 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f9461ffb640 1 -- 172.21.15.92:0/3633928444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945c0118f0 con 0x55a33bdf60a0 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.967+0000 7f9461ffb640 1 -- 172.21.15.92:0/3633928444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c01a880 con 0x55a33bdf60a0 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 -- 172.21.15.92:0/3633928444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 msgr2=0x55a33c1c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1c5440 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f945c00bc70 tx=0x7f945c0107a0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 -- 172.21.15.92:0/3633928444 shutdown_connections 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1c5440 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 --2- 172.21.15.92:0/3633928444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 -- 172.21.15.92:0/3633928444 >> 172.21.15.92:0/3633928444 conn(0x55a33be13350 msgr2=0x55a33c1c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:46.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 -- 172.21.15.92:0/3633928444 shutdown_connections 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.968+0000 7f9478105740 1 -- 172.21.15.92:0/3633928444 wait complete. 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 -- start start 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 Processor -- start 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33bdf60a0 0x55a33c1d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33be1ef60 0x55a33c1dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a33bf8d5f0 con 0x55a33bdfe280 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a33c0c7010 con 0x55a33be1ef60 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1d36e0 con 0x55a33bdf60a0 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f946affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f9478105740 10 client.0 _ll_register_callbacks cb 0x55a33c126340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f946affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34804/0 (socket says 172.21.15.92:34804) 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.969+0000 7f946affd640 1 -- 172.21.15.92:0/3163412011 learned_addr learned my addr 172.21.15.92:0/3163412011 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946bfff640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33be1ef60 0x55a33c1dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946b7fe640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33bdf60a0 0x55a33c1d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946affd640 1 -- 172.21.15.92:0/3163412011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33bdf60a0 msgr2=0x55a33c1d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946affd640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33bdf60a0 0x55a33c1d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946affd640 1 -- 172.21.15.92:0/3163412011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33be1ef60 msgr2=0x55a33c1dca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946affd640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33be1ef60 0x55a33c1dca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946affd640 1 -- 172.21.15.92:0/3163412011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33c0f4a00 con 0x55a33bdfe280 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946bfff640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33be1ef60 0x55a33c1dca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f946affd640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1d0db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f94540e4b90 tx=0x7f94540ac810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9852941ec9ad15c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f944affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f944affd640 1 -- 172.21.15.92:0/3163412011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94540e6070 con 0x55a33bdfe280 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f944affd640 1 -- 172.21.15.92:0/3163412011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f94540abeb0 con 0x55a33bdfe280 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945c0118f0 con 0x55a33bdfe280 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f944affd640 1 -- 172.21.15.92:0/3163412011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f945401d070 con 0x55a33bdfe280 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f9478105740 10 client.15159 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a33c1df6b0 con 0x55a33bdfe280 2024-09-21T02:03:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.970+0000 7f944affd640 1 -- 172.21.15.92:0/3163412011 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94540b6e50 con 0x55a33bdfe280 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f944affd640 1 -- 172.21.15.92:0/3163412011 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f94540ac0b0 con 0x55a33bdfe280 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f9478105740 10 client.15159 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f9478105740 10 client.15159 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f9478105740 20 client.15159 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f9478105740 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bddd8e0 0x55a33c1e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f9478105740 4 client.15159 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a33c13eb60 con 0x55a33bddd8e0 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f946b7fe640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bddd8e0 0x55a33c1e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f944affd640 10 client.15159 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:03:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.971+0000 7f946b7fe640 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bddd8e0 0x55a33c1e30a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2116c7f938eade37 server_cookie=db7a0be565615275 in_seq=0 out_seq=0 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f944affd640 1 -- 172.21.15.92:0/3163412011 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a33c13eb60 con 0x55a33bddd8e0 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f944affd640 10 client.15159 handle_command_reply: tid=0 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 client.15159 shutdown 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 msgr2=0x55a33c1d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1d0db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f94540e4b90 tx=0x7f94540ac810 comp rx=0 tx=0).stop 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 shutdown_connections 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bddd8e0 0x55a33c1e30a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33be1ef60 0x55a33c1dca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1d0db0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f944affd640 0 client.15159 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 --2- 172.21.15.92:0/3163412011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33bdf60a0 0x55a33c1d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.972+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 >> 172.21.15.92:0/3163412011 conn(0x55a33be13350 msgr2=0x55a33c1c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.973+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 shutdown_connections 2024-09-21T02:03:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.973+0000 7f9478105740 1 -- 172.21.15.92:0/3163412011 wait complete. 2024-09-21T02:03:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.973+0000 7f9478105740 20 client.15159 trim_cache size 0 max 16384 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.990+0000 7f9478105740 1 Processor -- start 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 -- start start 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdf60a0 0x55a33c1dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1e13e0 con 0x55a33bdfe280 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1e1ab0 con 0x55a33bdf60a0 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f9478105740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1e2180 con 0x55a33be1ef60 2024-09-21T02:03:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34820/0 (socket says 172.21.15.92:34820) 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 -- 172.21.15.92:0/1589200458 learned_addr learned my addr 172.21.15.92:0/1589200458 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946b7fe640 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdf60a0 0x55a33c1dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 -- 172.21.15.92:0/1589200458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 msgr2=0x55a33c1cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946bfff640 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 -- 172.21.15.92:0/1589200458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdf60a0 msgr2=0x55a33c1dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdf60a0 0x55a33c1dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 -- 172.21.15.92:0/1589200458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33c1c71f0 con 0x55a33bdfe280 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946b7fe640 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdf60a0 0x55a33c1dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f946affd640 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1dcee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f94540a97b0 tx=0x7f94540b6cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a68bbe19a6c069b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.991+0000 7f94617fa640 1 -- 172.21.15.92:0/1589200458 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94540d4020 con 0x55a33bdfe280 2024-09-21T02:03:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f94617fa640 1 -- 172.21.15.92:0/1589200458 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94540b6e50 con 0x55a33bdfe280 2024-09-21T02:03:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f94617fa640 1 -- 172.21.15.92:0/1589200458 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94540e6070 con 0x55a33bdfe280 2024-09-21T02:03:46.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 -- 172.21.15.92:0/1589200458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 msgr2=0x55a33c1dcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1dcee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f94540a97b0 tx=0x7f94540b6cd0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 -- 172.21.15.92:0/1589200458 shutdown_connections 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdfe280 0x55a33c1dcee0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 --2- 172.21.15.92:0/1589200458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdf60a0 0x55a33c1dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 -- 172.21.15.92:0/1589200458 >> 172.21.15.92:0/1589200458 conn(0x55a33be13350 msgr2=0x55a33c1e0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 -- 172.21.15.92:0/1589200458 shutdown_connections 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.992+0000 7f9478105740 1 -- 172.21.15.92:0/1589200458 wait complete. 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 -- start start 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 Processor -- start 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:46.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1e2f20 con 0x55a33bdf60a0 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1e2850 con 0x55a33bdfe280 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.993+0000 7f9478105740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a33c1e35f0 con 0x55a33be1ef60 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f9478105740 10 client.0 _ll_register_callbacks cb 0x7f9454007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1ebb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34828/0 (socket says 172.21.15.92:34828) 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 -- 172.21.15.92:0/2838023682 learned_addr learned my addr 172.21.15.92:0/2838023682 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946affd640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 -- 172.21.15.92:0/2838023682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 msgr2=0x55a33c1e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946bfff640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 -- 172.21.15.92:0/2838023682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 msgr2=0x55a33c1e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 -- 172.21.15.92:0/2838023682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33c1e5c60 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946bfff640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946affd640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f946b7fe640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1ebb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b620 tx=0x7f945c005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f31e0abea644d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f944affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f944affd640 1 -- 172.21.15.92:0/2838023682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00be60 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f944affd640 1 -- 172.21.15.92:0/2838023682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f945c011650 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.994+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94540b6e50 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f944affd640 1 -- 172.21.15.92:0/2838023682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f945c011aa0 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 10 client.15165 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a33c24e570 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f944affd640 1 -- 172.21.15.92:0/2838023682 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94540b6e50 con 0x55a33bdf60a0 2024-09-21T02:03:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f944affd640 1 -- 172.21.15.92:0/2838023682 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f945c004780 con 0x55a33bdf60a0 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 10 client.15165 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 10 client.15165 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 20 client.15165 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bdb2c90 0x55a33c251d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 4 client.15165 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "status"}) -- 0x55a33c13a610 con 0x55a33bdb2c90 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f946affd640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bdb2c90 0x55a33c251d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.995+0000 7f944affd640 10 client.15165 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f946affd640 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bdb2c90 0x55a33c251d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=48ddf0f7a2779f62 server_cookie=f94f4442a0dbd63c in_seq=0 out_seq=0 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f944affd640 1 -- 172.21.15.92:0/2838023682 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a33c13a610 con 0x55a33bdb2c90 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f944affd640 10 client.15165 handle_command_reply: tid=0 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 client.15165 shutdown 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 msgr2=0x55a33c1ebb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1ebb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b620 tx=0x7f945c005fc0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 shutdown_connections 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a33bdb2c90 0x55a33c251d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a33be1ef60 0x55a33c1e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33bdfe280 0x55a33c1e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 --2- 172.21.15.92:0/2838023682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a33bdf60a0 0x55a33c1ebb00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 >> 172.21.15.92:0/2838023682 conn(0x55a33be13350 msgr2=0x55a33c1cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f944affd640 0 client.15165 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 shutdown_connections 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 1 -- 172.21.15.92:0/2838023682 wait complete. 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.996+0000 7f9478105740 20 client.15165 trim_cache size 0 max 16384 2024-09-21T02:03:47.003 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 14799, 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 65.695829639999999, 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:03:47.004 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:03:47.005 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 88.237526643999999 2024-09-21T02:03:47.005 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:03:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f94580b65a0 msgr2=0x7f94580b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:47.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f94580b65a0 0x7f94580b8ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f946402f500 tx=0x7f94640095c0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 msgr2=0x7f946c102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 0x7f946c102cf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f945400c970 tx=0x7f945400ce30 comp rx=0 tx=0).stop 2024-09-21T02:03:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 shutdown_connections 2024-09-21T02:03:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f94580b65a0 0x7f94580b8ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c10de50 0x7f946c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f946c1052d0 0x7f946c102cf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 --2- 172.21.15.92:0/4213994898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f946c1048a0 0x7f946c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:47.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 >> 172.21.15.92:0/4213994898 conn(0x7f946c079730 msgr2=0x7f946c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:47.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 shutdown_connections 2024-09-21T02:03:47.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:46.997+0000 7f9471b91640 1 -- 172.21.15.92:0/4213994898 wait complete. 2024-09-21T02:03:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:46 smithi136 ceph-mon[34754]: pgmap v324: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:46 smithi099 ceph-mon[34404]: pgmap v324: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:47.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T02:03:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 -- 172.21.15.92:0/1004663720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 msgr2=0x7f5e8410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/1004663720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8410f540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009ca0 tx=0x7f5e7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 -- 172.21.15.92:0/1004663720 shutdown_connections 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/1004663720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8410f540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/1004663720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841083c0 0x7f5e841087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/1004663720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e84102a80 0x7f5e84102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 -- 172.21.15.92:0/1004663720 >> 172.21.15.92:0/1004663720 conn(0x7f5e84079740 msgr2=0x7f5e84079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 -- 172.21.15.92:0/1004663720 shutdown_connections 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.388+0000 7f5e8aea3640 1 -- 172.21.15.92:0/1004663720 wait complete. 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.389+0000 7f5e8aea3640 1 Processor -- start 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.389+0000 7f5e8aea3640 1 -- start start 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e84102a80 0x7f5e841993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e841083c0 0x7f5e84199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84112980 con 0x7f5e84108d10 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84112800 con 0x7f5e84102a80 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84112b00 con 0x7f5e841083c0 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34846/0 (socket says 172.21.15.92:34846) 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 -- 172.21.15.92:0/723809384 learned_addr learned my addr 172.21.15.92:0/723809384 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e84102a80 0x7f5e841993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e83fff640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e841083c0 0x7f5e84199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 -- 172.21.15.92:0/723809384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e841083c0 msgr2=0x7f5e84199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e841083c0 0x7f5e84199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 -- 172.21.15.92:0/723809384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e84102a80 msgr2=0x7f5e841993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e84102a80 0x7f5e841993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 -- 172.21.15.92:0/723809384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8419f2a0 con 0x7f5e84108d10 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e83fff640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e841083c0 0x7f5e84199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e89440640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8419ec30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c005cc0 tx=0x7f5e7c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e81ffb640 1 -- 172.21.15.92:0/723809384 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c003bf0 con 0x7f5e84108d10 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e81ffb640 1 -- 172.21.15.92:0/723809384 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7c003d90 con 0x7f5e84108d10 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e81ffb640 1 -- 172.21.15.92:0/723809384 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c02fbc0 con 0x7f5e84108d10 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.390+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8419f530 con 0x7f5e84108d10 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.391+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8419fa20 con 0x7f5e84108d10 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.393+0000 7f5e91417740 1 Processor -- start 2024-09-21T02:03:48.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.393+0000 7f5e81ffb640 1 -- 172.21.15.92:0/723809384 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f5e7c02fd60 con 0x7f5e84108d10 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.393+0000 7f5e91417740 1 -- start start 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.393+0000 7f5e91417740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x55692545bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.393+0000 7f5e91417740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 0x55692545c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e91417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x55692545c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e91417740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569253d3c10 con 0x55692508d0a0 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e91417740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5569253c3040 con 0x5569250b5f60 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e91417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5569253c6e50 con 0x556925095280 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 0x55692545c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 0x55692545c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34862/0 (socket says 172.21.15.92:34862) 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 -- 172.21.15.92:0/1186850792 learned_addr learned my addr 172.21.15.92:0/1186850792 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e83fff640 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x55692545c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e83fff640 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x55692545bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 -- 172.21.15.92:0/1186850792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 msgr2=0x55692545c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x55692545c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 -- 172.21.15.92:0/1186850792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 msgr2=0x55692545bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x55692545bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 -- 172.21.15.92:0/1186850792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556925121540 con 0x55692508d0a0 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e83fff640 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x55692545bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e81ffb640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5e740b65d0 0x7f5e740b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 0x55692545c440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800b6e0 tx=0x7f5e7800bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ee76c99bb8c427 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e83fff640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5e740b65d0 0x7f5e740b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e6effd640 1 -- 172.21.15.92:0/1186850792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800bdf0 con 0x55692508d0a0 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e81ffb640 1 -- 172.21.15.92:0/723809384 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5e7c108840 con 0x7f5e84108d10 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e6effd640 1 -- 172.21.15.92:0/1186850792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e780054d0 con 0x55692508d0a0 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.394+0000 7f5e6effd640 1 -- 172.21.15.92:0/1186850792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78004c40 con 0x55692508d0a0 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 -- 172.21.15.92:0/1186850792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 msgr2=0x55692545c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e83fff640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5e740b65d0 0x7f5e740b8ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005940 tx=0x7f5e70005890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 0x55692545c440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800b6e0 tx=0x7f5e7800bba0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 -- 172.21.15.92:0/1186850792 shutdown_connections 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x55692545c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55692508d0a0 0x55692545c440 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 --2- 172.21.15.92:0/1186850792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x55692545bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 -- 172.21.15.92:0/1186850792 >> 172.21.15.92:0/1186850792 conn(0x5569250aa350 msgr2=0x556925459df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 -- 172.21.15.92:0/1186850792 shutdown_connections 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.395+0000 7f5e91417740 1 -- 172.21.15.92:0/1186850792 wait complete. 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 -- start start 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 Processor -- start 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925467a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 0x556925467f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x5569254739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569252245f0 con 0x556925095280 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55692535e010 con 0x5569250b5f60 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55692546a6e0 con 0x55692508d0a0 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e91417740 10 client.0 _ll_register_callbacks cb 0x5569253bd340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.396+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 0x556925467f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 0x556925467f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34864/0 (socket says 172.21.15.92:34864) 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 -- 172.21.15.92:0/3286803495 learned_addr learned my addr 172.21.15.92:0/3286803495 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925467a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e89440640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x5569254739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 -- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 msgr2=0x556925467a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925467a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 -- 172.21.15.92:0/3286803495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 msgr2=0x5569254739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x5569254739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 -- 172.21.15.92:0/3286803495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692538ba00 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e83fff640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 0x556925467f60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011010 tx=0x7f5e70005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cb104982099de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e63fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e63fff640 1 -- 172.21.15.92:0/3286803495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7001e620 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e63fff640 1 -- 172.21.15.92:0/3286803495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5e7001e7c0 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e780054d0 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e63fff640 1 -- 172.21.15.92:0/3286803495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e70011e10 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e91417740 10 client.15177 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569254765d0 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.397+0000 7f5e63fff640 1 -- 172.21.15.92:0/3286803495 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e780054d0 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e63fff640 1 -- 172.21.15.92:0/3286803495 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5e70031910 con 0x556925095280 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e91417740 10 client.15177 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e91417740 10 client.15177 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e91417740 20 client.15177 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e91417740 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5569250748e0 0x55692547a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e91417740 4 client.15177 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569253d5b60 con 0x5569250748e0 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.398+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5569250748e0 0x55692547a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.399+0000 7f5e63fff640 10 client.15177 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.399+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5569250748e0 0x55692547a0b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1d24ec6851d1c78 server_cookie=577f752b83ce816c in_seq=0 out_seq=0 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.399+0000 7f5e63fff640 1 -- 172.21.15.92:0/3286803495 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5569253d5b60 con 0x5569250748e0 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.399+0000 7f5e63fff640 10 client.15177 handle_command_reply: tid=0 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.399+0000 7f5e91417740 1 client.15177 shutdown 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 msgr2=0x556925467f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 0x556925467f60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011010 tx=0x7f5e70005c70 comp rx=0 tx=0).stop 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 shutdown_connections 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5569250748e0 0x55692547a0b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5569250b5f60 0x5569254739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556925095280 0x556925467f60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 --2- 172.21.15.92:0/3286803495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925467a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 >> 172.21.15.92:0/3286803495 conn(0x5569250aa350 msgr2=0x55692545f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:48.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e63fff640 0 client.15177 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:03:48.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 shutdown_connections 2024-09-21T02:03:48.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 1 -- 172.21.15.92:0/3286803495 wait complete. 2024-09-21T02:03:48.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.400+0000 7f5e91417740 20 client.15177 trim_cache size 0 max 16384 2024-09-21T02:03:48.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 Processor -- start 2024-09-21T02:03:48.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 -- start start 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925462020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556925095280 0x556925473940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x556925473e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5569254783f0 con 0x5569250b5f60 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556925478ac0 con 0x556925095280 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e91417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556925479190 con 0x55692508d0a0 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e89440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x556925473e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e89440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x556925473e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34878/0 (socket says 172.21.15.92:34878) 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e89440640 1 -- 172.21.15.92:0/3263268355 learned_addr learned my addr 172.21.15.92:0/3263268355 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925462020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.418+0000 7f5e83fff640 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556925095280 0x556925473940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e89440640 1 -- 172.21.15.92:0/3263268355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 msgr2=0x556925462020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e89440640 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925462020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e89440640 1 -- 172.21.15.92:0/3263268355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556925095280 msgr2=0x556925473940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e89440640 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556925095280 0x556925473940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e89440640 1 -- 172.21.15.92:0/3263268355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556925454080 con 0x5569250b5f60 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e83fff640 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556925095280 0x556925473940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e89440640 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x556925473e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0ce7e0 tx=0x7f5e7c0cdfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f6cd855c85cee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e6e7fc640 1 -- 172.21.15.92:0/3263268355 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0cef50 con 0x5569250b5f60 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e6e7fc640 1 -- 172.21.15.92:0/3263268355 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7c036040 con 0x5569250b5f60 2024-09-21T02:03:48.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.419+0000 7f5e6e7fc640 1 -- 172.21.15.92:0/3263268355 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0d7de0 con 0x5569250b5f60 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 -- 172.21.15.92:0/3263268355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 msgr2=0x556925473e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x556925473e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0ce7e0 tx=0x7f5e7c0cdfc0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 -- 172.21.15.92:0/3263268355 shutdown_connections 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x556925473e80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556925095280 0x556925473940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 --2- 172.21.15.92:0/3263268355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55692508d0a0 0x556925462020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 -- 172.21.15.92:0/3263268355 >> 172.21.15.92:0/3263268355 conn(0x5569250aa350 msgr2=0x556925467a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 -- 172.21.15.92:0/3263268355 shutdown_connections 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 -- 172.21.15.92:0/3263268355 wait complete. 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 -- start start 2024-09-21T02:03:48.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.420+0000 7f5e91417740 1 Processor -- start 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55692508d0a0 0x556925482750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x556925482d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x55692547bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556925479f30 con 0x5569250b5f60 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556925479860 con 0x55692508d0a0 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55692547a600 con 0x556925095280 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e91417740 10 client.0 _ll_register_callbacks cb 0x7f5e7000ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x55692547bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x55692547bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34890/0 (socket says 172.21.15.92:34890) 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 -- 172.21.15.92:0/3171774351 learned_addr learned my addr 172.21.15.92:0/3171774351 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55692508d0a0 0x556925482750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e83fff640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x556925482d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 -- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 msgr2=0x556925482d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x556925482d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 -- 172.21.15.92:0/3171774351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55692508d0a0 msgr2=0x556925482750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55692508d0a0 0x556925482750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 -- 172.21.15.92:0/3171774351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692547c540 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e89440640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x55692547bdb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0ce550 tx=0x7f5e7c0ca820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478c77dd0ff20cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.421+0000 7f5e63fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e63fff640 1 -- 172.21.15.92:0/3171774351 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0cef50 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e63fff640 1 -- 172.21.15.92:0/3171774351 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5e7c0d7de0 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7c036040 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e63fff640 1 -- 172.21.15.92:0/3171774351 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e7c0d3330 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 10 client.15183 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556925481ef0 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e63fff640 1 -- 172.21.15.92:0/3171774351 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7c036040 con 0x5569250b5f60 2024-09-21T02:03:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e63fff640 1 -- 172.21.15.92:0/3171774351 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5e7c0ca1a0 con 0x5569250b5f60 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 10 client.15183 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 10 client.15183 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 20 client.15183 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x556925049c90 0x5569254e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 4 client.15183 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.422+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "status"}) -- 0x5569253d1610 con 0x556925049c90 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.423+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x556925049c90 0x5569254e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.423+0000 7f5e63fff640 10 client.15183 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.423+0000 7f5e88c3f640 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x556925049c90 0x5569254e86e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70847056cbc6b9cf server_cookie=63688bd23bc68eea in_seq=0 out_seq=0 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.423+0000 7f5e63fff640 1 -- 172.21.15.92:0/3171774351 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5569253d1610 con 0x556925049c90 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.423+0000 7f5e63fff640 10 client.15183 handle_command_reply: tid=0 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.423+0000 7f5e91417740 1 client.15183 shutdown 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 msgr2=0x55692547bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x55692547bdb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0ce550 tx=0x7f5e7c0ca820 comp rx=0 tx=0).stop 2024-09-21T02:03:48.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 shutdown_connections 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5569250b5f60 0x55692547bdb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556925095280 0x556925482d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x556925049c90 0x5569254e86e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 --2- 172.21.15.92:0/3171774351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55692508d0a0 0x556925482750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e63fff640 0 client.15183 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 >> 172.21.15.92:0/3171774351 conn(0x5569250aa350 msgr2=0x556925459600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 shutdown_connections 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 1 -- 172.21.15.92:0/3171774351 wait complete. 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.424+0000 7f5e91417740 20 client.15183 trim_cache size 0 max 16384 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:03:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24610, 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 67.388713949999996, 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 102.756462025 2024-09-21T02:03:48.431 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5e740b65d0 msgr2=0x7f5e740b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5e740b65d0 0x7f5e740b8ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005940 tx=0x7f5e70005890 comp rx=0 tx=0).stop 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 msgr2=0x7f5e8419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8419ec30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c005cc0 tx=0x7f5e7c02f380 comp rx=0 tx=0).stop 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 shutdown_connections 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5e740b65d0 0x7f5e740b8ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e84108d10 0x7f5e8419ec30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e841083c0 0x7f5e84199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 --2- 172.21.15.92:0/723809384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e84102a80 0x7f5e841993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:48.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 >> 172.21.15.92:0/723809384 conn(0x7f5e84079740 msgr2=0x7f5e84104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:48.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 shutdown_connections 2024-09-21T02:03:48.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:48.425+0000 7f5e8aea3640 1 -- 172.21.15.92:0/723809384 wait complete. 2024-09-21T02:03:49.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:48 smithi136 ceph-mon[34754]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:49.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:48 smithi092 ceph-mon[31190]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:49.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T02:03:49.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:48 smithi099 ceph-mon[34404]: pgmap v325: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:49.324 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.812+0000 7f6ec7acf640 1 -- 172.21.15.92:0/922793746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 msgr2=0x7f6ec01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.812+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/922793746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec01087e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009900 tx=0x7f6eb002a410 comp rx=0 tx=0).stop 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 -- 172.21.15.92:0/922793746 shutdown_connections 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/922793746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ec0108d20 0x7f6ec010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/922793746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec01087e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/922793746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0102a90 0x7f6ec0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 -- 172.21.15.92:0/922793746 >> 172.21.15.92:0/922793746 conn(0x7f6ec0079750 msgr2=0x7f6ec0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 -- 172.21.15.92:0/922793746 shutdown_connections 2024-09-21T02:03:49.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 -- 172.21.15.92:0/922793746 wait complete. 2024-09-21T02:03:49.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 Processor -- start 2024-09-21T02:03:49.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.813+0000 7f6ec7acf640 1 -- start start 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec7acf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ec0102a90 0x7f6ec00ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec7acf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec00ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec7acf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0108d20 0x7f6ec0100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec7acf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0112990 con 0x7f6ec01083d0 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec7acf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0112810 con 0x7f6ec0102a90 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec7acf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0112b10 con 0x7f6ec0108d20 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec00ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec00ffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34908/0 (socket says 172.21.15.92:34908) 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 -- 172.21.15.92:0/22089731 learned_addr learned my addr 172.21.15.92:0/22089731 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:49.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec586b640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ec0102a90 0x7f6ec00ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec606c640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0108d20 0x7f6ec0100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 -- 172.21.15.92:0/22089731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0108d20 msgr2=0x7f6ec0100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0108d20 0x7f6ec0100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 -- 172.21.15.92:0/22089731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ec0102a90 msgr2=0x7f6ec00ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ec0102a90 0x7f6ec00ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 -- 172.21.15.92:0/22089731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec01a8700 con 0x7f6ec01083d0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec606c640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0108d20 0x7f6ec0100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6ec506a640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec00ffb30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0032fd0 tx=0x7f6eb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.814+0000 7f6eb6ffd640 1 -- 172.21.15.92:0/22089731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb003f030 con 0x7f6ec01083d0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.815+0000 7f6eb6ffd640 1 -- 172.21.15.92:0/22089731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb0004490 con 0x7f6ec01083d0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.815+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec01a8990 con 0x7f6ec01083d0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.815+0000 7f6eb6ffd640 1 -- 172.21.15.92:0/22089731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0035070 con 0x7f6ec01083d0 2024-09-21T02:03:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.815+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec01a8ec0 con 0x7f6ec01083d0 2024-09-21T02:03:49.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.817+0000 7f6ece043740 1 Processor -- start 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.817+0000 7f6ece043740 1 -- start start 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.817+0000 7f6eb6ffd640 1 -- 172.21.15.92:0/22089731 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f6eb0042030 con 0x7f6ec01083d0 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ece043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1aaf60 0x5562eb550f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ece043740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb551440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ece043740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb18a280 0x5562eb551980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ece043740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5562eb4c8c10 con 0x5562eb1820a0 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ece043740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5562eb4b8040 con 0x5562eb18a280 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ece043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562eb4bbe50 con 0x5562eb1aaf60 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb551440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb551440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34916/0 (socket says 172.21.15.92:34916) 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 -- 172.21.15.92:0/228804723 learned_addr learned my addr 172.21.15.92:0/228804723 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec606c640 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1aaf60 0x5562eb550f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec506a640 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb18a280 0x5562eb551980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 -- 172.21.15.92:0/228804723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1aaf60 msgr2=0x5562eb550f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1aaf60 0x5562eb550f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 -- 172.21.15.92:0/228804723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb18a280 msgr2=0x5562eb551980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb18a280 0x5562eb551980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 -- 172.21.15.92:0/228804723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562eb216540 con 0x5562eb1820a0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec606c640 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1aaf60 0x5562eb550f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6eb6ffd640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6eac0b6620 0x7f6eac0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec586b640 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb551440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc004650 tx=0x7f6ebc00bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db07202af588be96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.818+0000 7f6ec606c640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6eac0b6620 0x7f6eac0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.819+0000 7f6eb6ffd640 1 -- 172.21.15.92:0/22089731 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6eb0107940 con 0x7f6ec01083d0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.819+0000 7f6eabfff640 1 -- 172.21.15.92:0/228804723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc003bd0 con 0x5562eb1820a0 2024-09-21T02:03:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.819+0000 7f6eabfff640 1 -- 172.21.15.92:0/228804723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc015070 con 0x5562eb1820a0 2024-09-21T02:03:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.819+0000 7f6eabfff640 1 -- 172.21.15.92:0/228804723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc004d80 con 0x5562eb1820a0 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 -- 172.21.15.92:0/228804723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 msgr2=0x5562eb551440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ec606c640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6eac0b6620 0x7f6eac0b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8007f30 tx=0x7f6eb8007910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb551440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc004650 tx=0x7f6ebc00bba0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 -- 172.21.15.92:0/228804723 shutdown_connections 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb18a280 0x5562eb551980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb551440 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 --2- 172.21.15.92:0/228804723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1aaf60 0x5562eb550f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 -- 172.21.15.92:0/228804723 >> 172.21.15.92:0/228804723 conn(0x5562eb19f350 msgr2=0x5562eb54edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 -- 172.21.15.92:0/228804723 shutdown_connections 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 -- 172.21.15.92:0/228804723 wait complete. 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 -- start start 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.820+0000 7f6ece043740 1 Processor -- start 2024-09-21T02:03:49.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb55ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb18a280 0x5562eb55cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb568a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5562eb3195f0 con 0x5562eb1820a0 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5562eb453010 con 0x5562eb1aaf60 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562eb55f6e0 con 0x5562eb18a280 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb55ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb55ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34932/0 (socket says 172.21.15.92:34932) 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 -- 172.21.15.92:0/3695444181 learned_addr learned my addr 172.21.15.92:0/3695444181 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ece043740 10 client.0 _ll_register_callbacks cb 0x5562eb4b2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec506a640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb18a280 0x5562eb55cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec606c640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb568a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 -- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb18a280 msgr2=0x5562eb55cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb18a280 0x5562eb55cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 -- 172.21.15.92:0/3695444181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 msgr2=0x5562eb568a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb568a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 -- 172.21.15.92:0/3695444181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562eb480a00 con 0x5562eb1820a0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec606c640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb568a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ec586b640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb55ca90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005880 tx=0x7f6ebc0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00f4f3799fa0482 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.821+0000 7f6ea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ea97fa640 1 -- 172.21.15.92:0/3695444181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc028060 con 0x5562eb1820a0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ea97fa640 1 -- 172.21.15.92:0/3695444181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6ebc011640 con 0x5562eb1820a0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc015070 con 0x5562eb1820a0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ea97fa640 1 -- 172.21.15.92:0/3695444181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ebc011840 con 0x5562eb1820a0 2024-09-21T02:03:49.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 10 client.15195 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562eb56b680 con 0x5562eb1820a0 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ea97fa640 1 -- 172.21.15.92:0/3695444181 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc015070 con 0x5562eb1820a0 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ea97fa640 1 -- 172.21.15.92:0/3695444181 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ebc022070 con 0x5562eb1820a0 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 10 client.15195 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 10 client.15195 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 20 client.15195 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.822+0000 7f6ece043740 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb1698e0 0x5562eb56f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.823+0000 7f6ece043740 4 client.15195 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.823+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562eb4cab60 con 0x5562eb1698e0 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.823+0000 7f6ec506a640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb1698e0 0x5562eb56f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.823+0000 7f6ec506a640 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb1698e0 0x5562eb56f490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be6b6a3e0277826 server_cookie=d0d74b4009c0e735 in_seq=0 out_seq=0 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.823+0000 7f6ea97fa640 10 client.15195 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.824+0000 7f6ea97fa640 1 -- 172.21.15.92:0/3695444181 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562eb4cab60 con 0x5562eb1698e0 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.824+0000 7f6ea97fa640 10 client.15195 handle_command_reply: tid=0 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.824+0000 7f6ece043740 1 client.15195 shutdown 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 msgr2=0x5562eb55ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb55ca90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005880 tx=0x7f6ebc0043a0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 shutdown_connections 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb1698e0 0x5562eb56f490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb568a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb18a280 0x5562eb55cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 --2- 172.21.15.92:0/3695444181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb1820a0 0x5562eb55ca90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 >> 172.21.15.92:0/3695444181 conn(0x5562eb19f350 msgr2=0x5562eb554090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ea97fa640 0 client.15195 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:03:49.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 shutdown_connections 2024-09-21T02:03:49.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 1 -- 172.21.15.92:0/3695444181 wait complete. 2024-09-21T02:03:49.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.825+0000 7f6ece043740 20 client.15195 trim_cache size 0 max 16384 2024-09-21T02:03:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 Processor -- start 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 -- start start 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb55bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb55ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb55cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5562eb56d7d0 con 0x5562eb18a280 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5562eb56dea0 con 0x5562eb1aaf60 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ece043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562eb56e570 con 0x5562eb1820a0 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb55ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb55ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34944/0 (socket says 172.21.15.92:34944) 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 -- 172.21.15.92:0/2133359884 learned_addr learned my addr 172.21.15.92:0/2133359884 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:49.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec586b640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb55bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec606c640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb55cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 -- 172.21.15.92:0/2133359884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 msgr2=0x5562eb55bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb55bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 -- 172.21.15.92:0/2133359884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 msgr2=0x5562eb55cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb55cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec506a640 1 -- 172.21.15.92:0/2133359884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562eb55d610 con 0x5562eb18a280 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec586b640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb55bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.843+0000 7f6ec606c640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb55cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ec506a640 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb55ca70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00cd650 tx=0x7f6eb0109e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c2e00e58fc970d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6eab7fe640 1 -- 172.21.15.92:0/2133359884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0049040 con 0x5562eb18a280 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6eab7fe640 1 -- 172.21.15.92:0/2133359884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00cd460 con 0x5562eb18a280 2024-09-21T02:03:49.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6eab7fe640 1 -- 172.21.15.92:0/2133359884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00c9bc0 con 0x5562eb18a280 2024-09-21T02:03:49.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 -- 172.21.15.92:0/2133359884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 msgr2=0x5562eb55ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb55ca70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00cd650 tx=0x7f6eb0109e30 comp rx=0 tx=0).stop 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 -- 172.21.15.92:0/2133359884 shutdown_connections 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb55cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb55ca70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 --2- 172.21.15.92:0/2133359884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb55bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.844+0000 7f6ece043740 1 -- 172.21.15.92:0/2133359884 >> 172.21.15.92:0/2133359884 conn(0x5562eb19f350 msgr2=0x5562eb54e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 -- 172.21.15.92:0/2133359884 shutdown_connections 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 -- 172.21.15.92:0/2133359884 wait complete. 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 -- start start 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 Processor -- start 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb5707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb570d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb575bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5562eb56f310 con 0x5562eb18a280 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5562eb56ec40 con 0x5562eb1aaf60 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.845+0000 7f6ece043740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562eb56f9e0 con 0x5562eb1820a0 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ece043740 10 client.0 _ll_register_callbacks cb 0x7f6ebc004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb570d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb570d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34952/0 (socket says 172.21.15.92:34952) 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 -- 172.21.15.92:0/517463564 learned_addr learned my addr 172.21.15.92:0/517463564 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec586b640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb5707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec606c640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb575bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 -- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 msgr2=0x5562eb5707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb5707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 -- 172.21.15.92:0/517463564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 msgr2=0x5562eb575bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb575bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 -- 172.21.15.92:0/517463564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562eb576230 con 0x5562eb18a280 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ec506a640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb570d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00cabc0 tx=0x7f6eb0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7726aac53ac88d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ea97fa640 1 -- 172.21.15.92:0/517463564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0049070 con 0x5562eb18a280 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ea97fa640 1 -- 172.21.15.92:0/517463564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6eb00d2090 con 0x5562eb18a280 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb00cd460 con 0x5562eb18a280 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ea97fa640 1 -- 172.21.15.92:0/517463564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6eb00d2430 con 0x5562eb18a280 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ece043740 10 client.15201 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.846+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562eb5775f0 con 0x5562eb18a280 2024-09-21T02:03:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ea97fa640 1 -- 172.21.15.92:0/517463564 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00cd460 con 0x5562eb18a280 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ea97fa640 1 -- 172.21.15.92:0/517463564 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb00c92a0 con 0x5562eb18a280 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ece043740 10 client.15201 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ece043740 10 client.15201 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ece043740 20 client.15201 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ece043740 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb13ec90 0x5562eb5dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ece043740 4 client.15201 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "status"}) -- 0x5562eb4c6610 con 0x5562eb13ec90 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.847+0000 7f6ec586b640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb13ec90 0x5562eb5dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ea97fa640 10 client.15201 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ec586b640 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb13ec90 0x5562eb5dcf90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ded32aae77f7cf6 server_cookie=5f9b09d778db8775 in_seq=0 out_seq=0 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ea97fa640 1 -- 172.21.15.92:0/517463564 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5562eb4c6610 con 0x5562eb13ec90 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ea97fa640 10 client.15201 handle_command_reply: tid=0 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 client.15201 shutdown 2024-09-21T02:03:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 msgr2=0x5562eb570d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb570d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00cabc0 tx=0x7f6eb0009680 comp rx=0 tx=0).stop 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 shutdown_connections 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5562eb13ec90 0x5562eb5dcf90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562eb1aaf60 0x5562eb575bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562eb18a280 0x5562eb570d60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 --2- 172.21.15.92:0/517463564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562eb1820a0 0x5562eb5707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ea97fa640 0 client.15201 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 >> 172.21.15.92:0/517463564 conn(0x5562eb19f350 msgr2=0x5562eb557a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 shutdown_connections 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 1 -- 172.21.15.92:0/517463564 wait complete. 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.848+0000 7f6ece043740 20 client.15201 trim_cache size 0 max 16384 2024-09-21T02:03:49.856 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24619, 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 69.822880237000007, 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 101.79252973 2024-09-21T02:03:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6eac0b6620 msgr2=0x7f6eac0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6eac0b6620 0x7f6eac0b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8007f30 tx=0x7f6eb8007910 comp rx=0 tx=0).stop 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 msgr2=0x7f6ec00ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec00ffb30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0032fd0 tx=0x7f6eb0004270 comp rx=0 tx=0).stop 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 shutdown_connections 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6eac0b6620 0x7f6eac0b8b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec0108d20 0x7f6ec0100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ec01083d0 0x7f6ec00ffb30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 --2- 172.21.15.92:0/22089731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ec0102a90 0x7f6ec00ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 >> 172.21.15.92:0/22089731 conn(0x7f6ec0079750 msgr2=0x7f6ec0101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 shutdown_connections 2024-09-21T02:03:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:49.849+0000 7f6ec7acf640 1 -- 172.21.15.92:0/22089731 wait complete. 2024-09-21T02:03:50.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T02:03:50.791 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:51.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:50 smithi136 ceph-mon[34754]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:03:51.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:50 smithi092 ceph-mon[31190]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:03:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:50 smithi099 ceph-mon[34404]: pgmap v326: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:03:51.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 -- 172.21.15.92:0/1397017501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 msgr2=0x7fa314102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 --2- 172.21.15.92:0/1397017501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314102e90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009ca0 tx=0x7fa30c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 -- 172.21.15.92:0/1397017501 shutdown_connections 2024-09-21T02:03:51.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 --2- 172.21.15.92:0/1397017501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa314108d10 0x7fa31410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 --2- 172.21.15.92:0/1397017501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3141083c0 0x7fa3141087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 --2- 172.21.15.92:0/1397017501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314102e90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 -- 172.21.15.92:0/1397017501 >> 172.21.15.92:0/1397017501 conn(0x7fa314079740 msgr2=0x7fa314079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.311+0000 7fa31c292640 1 -- 172.21.15.92:0/1397017501 shutdown_connections 2024-09-21T02:03:51.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.312+0000 7fa31c292640 1 -- 172.21.15.92:0/1397017501 wait complete. 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.312+0000 7fa31c292640 1 Processor -- start 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.312+0000 7fa31c292640 1 -- start start 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31c292640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31c292640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141083c0 0x7fa314199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31c292640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa314108d10 0x7fa31419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31c292640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa314112980 con 0x7fa314102a80 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31c292640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa314112800 con 0x7fa314108d10 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31c292640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa314112b00 con 0x7fa3141083c0 2024-09-21T02:03:51.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34958/0 (socket says 172.21.15.92:34958) 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 -- 172.21.15.92:0/3577638825 learned_addr learned my addr 172.21.15.92:0/3577638825 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31982d640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141083c0 0x7fa314199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa314108d10 0x7fa31419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 -- 172.21.15.92:0/3577638825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141083c0 msgr2=0x7fa314199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141083c0 0x7fa314199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 -- 172.21.15.92:0/3577638825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa314108d10 msgr2=0x7fa31419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa314108d10 0x7fa31419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 -- 172.21.15.92:0/3577638825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31419f240 con 0x7fa314102a80 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa314108d10 0x7fa31419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.313+0000 7fa31a02e640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314199390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa30c006010 tx=0x7fa30c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.314+0000 7fa30b7fe640 1 -- 172.21.15.92:0/3577638825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c003bf0 con 0x7fa314102a80 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.314+0000 7fa30b7fe640 1 -- 172.21.15.92:0/3577638825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30c003d90 con 0x7fa314102a80 2024-09-21T02:03:51.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.314+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31419f4d0 con 0x7fa314102a80 2024-09-21T02:03:51.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.314+0000 7fa30b7fe640 1 -- 172.21.15.92:0/3577638825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c02fc20 con 0x7fa314102a80 2024-09-21T02:03:51.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.314+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31419fa10 con 0x7fa314102a80 2024-09-21T02:03:51.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.316+0000 7fa322806740 1 Processor -- start 2024-09-21T02:03:51.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.316+0000 7fa30b7fe640 1 -- 172.21.15.92:0/3577638825 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa30c0305c0 con 0x7fa314102a80 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.316+0000 7fa322806740 1 -- start start 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa322806740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24cef60 0x559be2874f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa322806740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 0x559be2875440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa322806740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2875980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa322806740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559be27ecc10 con 0x559be24a60a0 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa322806740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559be27dc040 con 0x559be24cef60 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa322806740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559be27dfe50 con 0x559be24ae280 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 0x559be2875440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 0x559be2875440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34962/0 (socket says 172.21.15.92:34962) 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 -- 172.21.15.92:0/95143401 learned_addr learned my addr 172.21.15.92:0/95143401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a82f640 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24cef60 0x559be2874f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31982d640 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2875980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 -- 172.21.15.92:0/95143401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 msgr2=0x559be2875980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2875980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 -- 172.21.15.92:0/95143401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24cef60 msgr2=0x559be2874f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa30b7fe640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3000b65d0 0x7fa3000b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24cef60 0x559be2874f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 -- 172.21.15.92:0/95143401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be253a540 con 0x559be24a60a0 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a82f640 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24cef60 0x559be2874f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3000b65d0 0x7fa3000b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.317+0000 7fa31a02e640 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 0x559be2875440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa30c1062c0 tx=0x7fa30c0cebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6dfe641efcceb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.318+0000 7fa30b7fe640 1 -- 172.21.15.92:0/3577638825 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa30c1089f0 con 0x7fa314102a80 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.318+0000 7fa308d9a640 1 -- 172.21.15.92:0/95143401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c108e60 con 0x559be24a60a0 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.318+0000 7fa308d9a640 1 -- 172.21.15.92:0/95143401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30c112040 con 0x559be24a60a0 2024-09-21T02:03:51.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.318+0000 7fa308d9a640 1 -- 172.21.15.92:0/95143401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c111070 con 0x559be24a60a0 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 -- 172.21.15.92:0/95143401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 msgr2=0x559be2875440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 0x559be2875440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa30c1062c0 tx=0x7fa30c0cebf0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3000b65d0 0x7fa3000b8ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa31000cf50 tx=0x7fa3100046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 -- 172.21.15.92:0/95143401 shutdown_connections 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2875980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24a60a0 0x559be2875440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 --2- 172.21.15.92:0/95143401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24cef60 0x559be2874f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 -- 172.21.15.92:0/95143401 >> 172.21.15.92:0/95143401 conn(0x559be24c3350 msgr2=0x559be2872df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 -- 172.21.15.92:0/95143401 shutdown_connections 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 -- 172.21.15.92:0/95143401 wait complete. 2024-09-21T02:03:51.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 -- start start 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.319+0000 7fa322806740 1 Processor -- start 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2880760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 0x559be2880d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24cef60 0x559be288c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559be263d5f0 con 0x559be24ae280 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559be2777010 con 0x559be24a60a0 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559be27ecc10 con 0x559be24cef60 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 10 client.0 _ll_register_callbacks cb 0x559be27d6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 0x559be2880d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 0x559be2880d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34978/0 (socket says 172.21.15.92:34978) 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 -- 172.21.15.92:0/2887221028 learned_addr learned my addr 172.21.15.92:0/2887221028 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31a02e640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2880760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31a82f640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24cef60 0x559be288c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 -- 172.21.15.92:0/2887221028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24cef60 msgr2=0x559be288c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24cef60 0x559be288c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 -- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 msgr2=0x559be2880760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2880760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 -- 172.21.15.92:0/2887221028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be27a4a00 con 0x559be24ae280 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31a82f640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24cef60 0x559be288c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31a02e640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2880760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa31982d640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 0x559be2880d00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa30400bad0 tx=0x7fa30400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762c0a963227d477 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa2fdffb640 1 -- 172.21.15.92:0/2887221028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304011440 con 0x559be24ae280 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa2fdffb640 1 -- 172.21.15.92:0/2887221028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa3040115e0 con 0x559be24ae280 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.320+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c112040 con 0x559be24ae280 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa2fdffb640 1 -- 172.21.15.92:0/2887221028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa304005020 con 0x559be24ae280 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 10 client.15213 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559be288f180 con 0x559be24ae280 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa2fdffb640 1 -- 172.21.15.92:0/2887221028 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa30c112040 con 0x559be24ae280 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa2fdffb640 1 -- 172.21.15.92:0/2887221028 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa3040047a0 con 0x559be24ae280 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 10 client.15213 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 20 client.15213 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be248d8e0 0x559be2892ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 4 client.15213 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be27eeb60 con 0x559be248d8e0 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.321+0000 7fa31a02e640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be248d8e0 0x559be2892ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.322+0000 7fa2fdffb640 10 client.15213 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.322+0000 7fa31a02e640 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be248d8e0 0x559be2892ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49f2e6d72c581199 server_cookie=a184cce9a5416add in_seq=0 out_seq=0 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa2fdffb640 1 -- 172.21.15.92:0/2887221028 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559be27eeb60 con 0x559be248d8e0 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa2fdffb640 10 client.15213 handle_command_reply: tid=0 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 client.15213 shutdown 2024-09-21T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 msgr2=0x559be2880d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 0x559be2880d00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa30400bad0 tx=0x7fa30400bf90 comp rx=0 tx=0).stop 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 shutdown_connections 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24cef60 0x559be288c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be248d8e0 0x559be2892ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24ae280 0x559be2880d00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 --2- 172.21.15.92:0/2887221028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2880760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa2fdffb640 0 client.15213 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 >> 172.21.15.92:0/2887221028 conn(0x559be24c3350 msgr2=0x559be28780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 shutdown_connections 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 1 -- 172.21.15.92:0/2887221028 wait complete. 2024-09-21T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.323+0000 7fa322806740 20 client.15213 trim_cache size 0 max 16384 2024-09-21T02:03:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.341+0000 7fa322806740 1 Processor -- start 2024-09-21T02:03:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 -- start start 2024-09-21T02:03:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be287b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be287be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be287efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559be2890fe0 con 0x559be24cef60 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559be28916b0 con 0x559be24a60a0 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa322806740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559be2891d80 con 0x559be24ae280 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be287efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be287efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34990/0 (socket says 172.21.15.92:34990) 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 -- 172.21.15.92:0/3910520052 learned_addr learned my addr 172.21.15.92:0/3910520052 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31982d640 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be287be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a02e640 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be287b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 -- 172.21.15.92:0/3910520052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 msgr2=0x559be287be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be287be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 -- 172.21.15.92:0/3910520052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 msgr2=0x559be287b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be287b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 -- 172.21.15.92:0/3910520052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be2881a20 con 0x559be24cef60 2024-09-21T02:03:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.342+0000 7fa31a82f640 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be287efe0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa31001e7d0 tx=0x7fa310014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feffb2ef0e7e1f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa2fffff640 1 -- 172.21.15.92:0/3910520052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310027430 con 0x559be24cef60 2024-09-21T02:03:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa2fffff640 1 -- 172.21.15.92:0/3910520052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa310016070 con 0x559be24cef60 2024-09-21T02:03:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa2fffff640 1 -- 172.21.15.92:0/3910520052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310026640 con 0x559be24cef60 2024-09-21T02:03:51.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 -- 172.21.15.92:0/3910520052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 msgr2=0x559be287efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be287efe0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa31001e7d0 tx=0x7fa310014000 comp rx=0 tx=0).stop 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 -- 172.21.15.92:0/3910520052 shutdown_connections 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be287efe0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be287be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 --2- 172.21.15.92:0/3910520052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be287b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.343+0000 7fa322806740 1 -- 172.21.15.92:0/3910520052 >> 172.21.15.92:0/3910520052 conn(0x559be24c3350 msgr2=0x559be2890640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 -- 172.21.15.92:0/3910520052 shutdown_connections 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 -- 172.21.15.92:0/3910520052 wait complete. 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 -- start start 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 Processor -- start 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2897a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2897fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be2878000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559be2892b20 con 0x559be24cef60 2024-09-21T02:03:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559be2892450 con 0x559be24a60a0 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559be28931f0 con 0x559be24ae280 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa322806740 10 client.0 _ll_register_callbacks cb 0x7fa304004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.344+0000 7fa31a82f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be2878000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be2878000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34992/0 (socket says 172.21.15.92:34992) 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 -- 172.21.15.92:0/1089887164 learned_addr learned my addr 172.21.15.92:0/1089887164 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31982d640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2897fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a02e640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2897a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 -- 172.21.15.92:0/1089887164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 msgr2=0x559be2897fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2897fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 -- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 msgr2=0x559be2897a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2897a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 -- 172.21.15.92:0/1089887164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be2878540 con 0x559be24cef60 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa31a82f640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be2878000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa310011c40 tx=0x7fa31000c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbc085edca5c5cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa2fdffb640 1 -- 172.21.15.92:0/1089887164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310037060 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa2fdffb640 1 -- 172.21.15.92:0/1089887164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa31001e200 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.345+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa310016070 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa2fdffb640 1 -- 172.21.15.92:0/1089887164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa31001e400 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 10 client.15219 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559be2896f50 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa2fdffb640 1 -- 172.21.15.92:0/1089887164 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa310016070 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa2fdffb640 1 -- 172.21.15.92:0/1089887164 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa31001d870 con 0x559be24cef60 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 10 client.15219 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 20 client.15219 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be2462c90 0x559be2900f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 4 client.15219 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:03:51.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "status"}) -- 0x559be27ea610 con 0x559be2462c90 2024-09-21T02:03:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.346+0000 7fa31a02e640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be2462c90 0x559be2900f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa2fdffb640 10 client.15219 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:03:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa31a02e640 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be2462c90 0x559be2900f40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b4d1cd4fa0c0449 server_cookie=ab1326f8c07d2a00 in_seq=0 out_seq=0 2024-09-21T02:03:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa2fdffb640 1 -- 172.21.15.92:0/1089887164 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559be27ea610 con 0x559be2462c90 2024-09-21T02:03:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa2fdffb640 10 client.15219 handle_command_reply: tid=0 2024-09-21T02:03:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 client.15219 shutdown 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 msgr2=0x559be2878000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be2878000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa310011c40 tx=0x7fa31000c560 comp rx=0 tx=0).stop 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 shutdown_connections 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x559be2462c90 0x559be2900f40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559be24cef60 0x559be2878000 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559be24ae280 0x559be2897fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 --2- 172.21.15.92:0/1089887164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be24a60a0 0x559be2897a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 >> 172.21.15.92:0/1089887164 conn(0x559be24c3350 msgr2=0x559be287b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.347+0000 7fa2fdffb640 0 client.15219 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:03:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 shutdown_connections 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa322806740 1 -- 172.21.15.92:0/1089887164 wait complete. 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa322806740 20 client.15219 trim_cache size 0 max 16384 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24623, 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:03:51.354 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 72.330079767000001, 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 90.033905876000006 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:03:51.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3000b65d0 msgr2=0x7fa3000b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3000b65d0 0x7fa3000b8ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa31000cf50 tx=0x7fa3100046c0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 msgr2=0x7fa314199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314199390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa30c006010 tx=0x7fa30c0319f0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 shutdown_connections 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3000b65d0 0x7fa3000b8ae0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa314108d10 0x7fa31419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141083c0 0x7fa314199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 --2- 172.21.15.92:0/3577638825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa314102a80 0x7fa314199390 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.348+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 >> 172.21.15.92:0/3577638825 conn(0x7fa314079740 msgr2=0x7fa314101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.349+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 shutdown_connections 2024-09-21T02:03:51.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:51.349+0000 7fa31c292640 1 -- 172.21.15.92:0/3577638825 wait complete. 2024-09-21T02:03:52.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T02:03:52.269 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:52.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 -- 172.21.15.92:0/3102812867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 msgr2=0x7f8524108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 --2- 172.21.15.92:0/3102812867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f8524108830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8518009900 tx=0x7f851802a410 comp rx=0 tx=0).stop 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 -- 172.21.15.92:0/3102812867 shutdown_connections 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 --2- 172.21.15.92:0/3102812867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8524108d70 0x7f852410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 --2- 172.21.15.92:0/3102812867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f8524108830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 --2- 172.21.15.92:0/3102812867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524102ae0 0x7f8524102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 -- 172.21.15.92:0/3102812867 >> 172.21.15.92:0/3102812867 conn(0x7f8524079770 msgr2=0x7f8524079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 -- 172.21.15.92:0/3102812867 shutdown_connections 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.768+0000 7f8528bdc640 1 -- 172.21.15.92:0/3102812867 wait complete. 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.769+0000 7f8528bdc640 1 Processor -- start 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.769+0000 7f8528bdc640 1 -- start start 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8528bdc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8524102ae0 0x7f852419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8528bdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f852419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8528bdc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524108d70 0x7f852419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8528bdc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112a50 con 0x7f8524108420 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8528bdc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85241128d0 con 0x7f8524102ae0 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8528bdc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112bd0 con 0x7f8524108d70 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f852419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f852419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35018/0 (socket says 172.21.15.92:35018) 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 -- 172.21.15.92:0/1699073025 learned_addr learned my addr 172.21.15.92:0/1699073025 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8522d9d640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524108d70 0x7f852419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f852259c640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8524102ae0 0x7f852419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 -- 172.21.15.92:0/1699073025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524108d70 msgr2=0x7f852419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524108d70 0x7f852419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 -- 172.21.15.92:0/1699073025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8524102ae0 msgr2=0x7f852419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8524102ae0 0x7f852419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 -- 172.21.15.92:0/1699073025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85241a05c0 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8522d9d640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524108d70 0x7f852419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.770+0000 7f8521d9b640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f852419abf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f851802a930 tx=0x7f8518004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.771+0000 7f85137fe640 1 -- 172.21.15.92:0/1699073025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851803f030 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.771+0000 7f85137fe640 1 -- 172.21.15.92:0/1699073025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8518003cb0 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.771+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241a0850 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.771+0000 7f85137fe640 1 -- 172.21.15.92:0/1699073025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518035070 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.771+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a7010 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.773+0000 7f852f150740 1 Processor -- start 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 -- start start 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f85137fe640 1 -- 172.21.15.92:0/1699073025 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f8518042030 con 0x7f8524108420 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 0x55dbb6a9df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66cf0a0 0x55dbb6a9e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6a9e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6a15c10 con 0x55dbb66f7f60 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6a05040 con 0x55dbb66cf0a0 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852f150740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6a08e50 con 0x55dbb66d7280 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 0x55dbb6a9df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 0x55dbb6a9df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35030/0 (socket says 172.21.15.92:35030) 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 -- 172.21.15.92:0/2378792497 learned_addr learned my addr 172.21.15.92:0/2378792497 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f852259c640 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66cf0a0 0x55dbb6a9e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8521d9b640 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6a9e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 -- 172.21.15.92:0/2378792497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 msgr2=0x55dbb6a9e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6a9e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 -- 172.21.15.92:0/2378792497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66cf0a0 msgr2=0x55dbb6a9e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66cf0a0 0x55dbb6a9e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.774+0000 7f8522d9d640 1 -- 172.21.15.92:0/2378792497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb6763540 con 0x55dbb66f7f60 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f85137fe640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f850c0b6620 0x7f850c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f8522d9d640 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 0x55dbb6a9df00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c450 tx=0x7f851c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4a5d7ac557c5e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f852259c640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f850c0b6620 0x7f850c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f85137fe640 1 -- 172.21.15.92:0/1699073025 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8518107b40 con 0x7f8524108420 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f8510ff9640 1 -- 172.21.15.92:0/2378792497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c007430 con 0x55dbb66f7f60 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f8510ff9640 1 -- 172.21.15.92:0/2378792497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851c007a80 con 0x55dbb66f7f60 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.775+0000 7f8510ff9640 1 -- 172.21.15.92:0/2378792497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c00aa60 con 0x55dbb66f7f60 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 -- 172.21.15.92:0/2378792497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 msgr2=0x55dbb6a9df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852259c640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f850c0b6620 0x7f850c0b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55dbb6a9cb80 tx=0x7f851400c430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 0x55dbb6a9df00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c450 tx=0x7f851c00c910 comp rx=0 tx=0).stop 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 -- 172.21.15.92:0/2378792497 shutdown_connections 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6a9e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66cf0a0 0x55dbb6a9e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 --2- 172.21.15.92:0/2378792497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66f7f60 0x55dbb6a9df00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 -- 172.21.15.92:0/2378792497 >> 172.21.15.92:0/2378792497 conn(0x55dbb66ec350 msgr2=0x55dbb6a9bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 -- 172.21.15.92:0/2378792497 shutdown_connections 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 -- 172.21.15.92:0/2378792497 wait complete. 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.776+0000 7f852f150740 1 -- start start 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 Processor -- start 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6aa9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66f7f60 0x55dbb6ab5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dbb68665f0 con 0x55dbb66cf0a0 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbb69a0010 con 0x55dbb66f7f60 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6aac6e0 con 0x55dbb66d7280 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852f150740 10 client.0 _ll_register_callbacks cb 0x55dbb69ff340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35040/0 (socket says 172.21.15.92:35040) 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 -- 172.21.15.92:0/3880250448 learned_addr learned my addr 172.21.15.92:0/3880250448 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 -- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 msgr2=0x55dbb6aa9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6aa9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 -- 172.21.15.92:0/3880250448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66f7f60 msgr2=0x55dbb6ab5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66f7f60 0x55dbb6ab5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 -- 172.21.15.92:0/3880250448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb69cda00 con 0x55dbb66cf0a0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f852259c640 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa9a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8514005c50 tx=0x7f85140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf19218319a36ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.777+0000 7f8509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f8509ffb640 1 -- 172.21.15.92:0/3880250448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851400cd40 con 0x55dbb66cf0a0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f8509ffb640 1 -- 172.21.15.92:0/3880250448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8514004020 con 0x55dbb66cf0a0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851c007a80 con 0x55dbb66cf0a0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f8509ffb640 1 -- 172.21.15.92:0/3880250448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8514004260 con 0x55dbb66cf0a0 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 10 client.15231 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:52.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbb6ab8690 con 0x55dbb66cf0a0 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f8509ffb640 1 -- 172.21.15.92:0/3880250448 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851c007a80 con 0x55dbb66cf0a0 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f8509ffb640 1 -- 172.21.15.92:0/3880250448 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8514018040 con 0x55dbb66cf0a0 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 10 client.15231 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 10 client.15231 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 20 client.15231 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66b68e0 0x55dbb6abc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 4 client.15231 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.778+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbb6a17b60 con 0x55dbb66b68e0 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.779+0000 7f8521d9b640 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66b68e0 0x55dbb6abc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.779+0000 7f8509ffb640 10 client.15231 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.779+0000 7f8521d9b640 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66b68e0 0x55dbb6abc2a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f1d6ff3aef46a1 server_cookie=a18a78628da3e6dd in_seq=0 out_seq=0 2024-09-21T02:03:52.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f8509ffb640 1 -- 172.21.15.92:0/3880250448 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dbb6a17b60 con 0x55dbb66b68e0 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f8509ffb640 10 client.15231 handle_command_reply: tid=0 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 client.15231 shutdown 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 msgr2=0x55dbb6aa9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa9a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8514005c50 tx=0x7f85140046c0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 shutdown_connections 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66b68e0 0x55dbb6abc2a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66f7f60 0x55dbb6ab5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66d7280 0x55dbb6aa9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f8509ffb640 0 client.15231 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 --2- 172.21.15.92:0/3880250448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa9a20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 >> 172.21.15.92:0/3880250448 conn(0x55dbb66ec350 msgr2=0x55dbb6aa10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 shutdown_connections 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.780+0000 7f852f150740 1 -- 172.21.15.92:0/3880250448 wait complete. 2024-09-21T02:03:52.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.781+0000 7f852f150740 20 client.15231 trim_cache size 0 max 16384 2024-09-21T02:03:52.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.801+0000 7f852f150740 1 Processor -- start 2024-09-21T02:03:52.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.801+0000 7f852f150740 1 -- start start 2024-09-21T02:03:52.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852f150740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6aa49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852f150740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852f150740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6ab5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852f150740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6aba5e0 con 0x55dbb66d7280 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852f150740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6abacb0 con 0x55dbb66b68e0 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852f150740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6abb380 con 0x55dbb66cf0a0 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6ab5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6ab5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35046/0 (socket says 172.21.15.92:35046) 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 -- 172.21.15.92:0/4061317130 learned_addr learned my addr 172.21.15.92:0/4061317130 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852259c640 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6aa49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 -- 172.21.15.92:0/4061317130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 msgr2=0x55dbb6aa4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8521d9b640 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 -- 172.21.15.92:0/4061317130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 msgr2=0x55dbb6aa49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6aa49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 -- 172.21.15.92:0/4061317130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb6aa0790 con 0x55dbb66d7280 2024-09-21T02:03:52.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f852259c640 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6aa49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:52.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f8522d9d640 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6ab5970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f851c007080 tx=0x7f851c0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2de7a2cecf23b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f850bfff640 1 -- 172.21.15.92:0/4061317130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c009b20 con 0x55dbb66d7280 2024-09-21T02:03:52.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f850bfff640 1 -- 172.21.15.92:0/4061317130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851c023070 con 0x55dbb66d7280 2024-09-21T02:03:52.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.802+0000 7f850bfff640 1 -- 172.21.15.92:0/4061317130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c016420 con 0x55dbb66d7280 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 -- 172.21.15.92:0/4061317130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 msgr2=0x55dbb6ab5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6ab5970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f851c007080 tx=0x7f851c0049c0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 -- 172.21.15.92:0/4061317130 shutdown_connections 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6ab5970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6aa4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 --2- 172.21.15.92:0/4061317130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6aa49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 -- 172.21.15.92:0/4061317130 >> 172.21.15.92:0/4061317130 conn(0x55dbb66ec350 msgr2=0x55dbb6ab9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 -- 172.21.15.92:0/4061317130 shutdown_connections 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.803+0000 7f852f150740 1 -- 172.21.15.92:0/4061317130 wait complete. 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 -- start start 2024-09-21T02:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 Processor -- start 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6ac4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6abe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6abe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6abc120 con 0x55dbb66d7280 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6abba50 con 0x55dbb66b68e0 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dbb6abc7f0 con 0x55dbb66cf0a0 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852f150740 10 client.0 _ll_register_callbacks cb 0x7f851400c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6abe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6abe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35050/0 (socket says 172.21.15.92:35050) 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f8522d9d640 1 -- 172.21.15.92:0/2610862980 learned_addr learned my addr 172.21.15.92:0/2610862980 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f8521d9b640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6abe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.804+0000 7f852259c640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6ac4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8522d9d640 1 -- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 msgr2=0x55dbb6abe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8522d9d640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6abe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8522d9d640 1 -- 172.21.15.92:0/2610862980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 msgr2=0x55dbb6ac4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8522d9d640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6ac4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8522d9d640 1 -- 172.21.15.92:0/2610862980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb6abed50 con 0x55dbb66d7280 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8521d9b640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6abe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8522d9d640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6abe660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c950 tx=0x7f851c01e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7228f570f5718302 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8509ffb640 1 -- 172.21.15.92:0/2610862980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c02b020 con 0x55dbb66d7280 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8509ffb640 1 -- 172.21.15.92:0/2610862980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f851c01e8e0 con 0x55dbb66d7280 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851c023070 con 0x55dbb66d7280 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8509ffb640 1 -- 172.21.15.92:0/2610862980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f851c016420 con 0x55dbb66d7280 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f852f150740 10 client.15237 fetch_fsmap learned FSMap version 28 2024-09-21T02:03:52.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbb6b27770 con 0x55dbb66d7280 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8509ffb640 1 -- 172.21.15.92:0/2610862980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851c023070 con 0x55dbb66d7280 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.805+0000 7f8509ffb640 1 -- 172.21.15.92:0/2610862980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f851c0276b0 con 0x55dbb66d7280 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 10 client.15237 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 20 client.15237 populate_metadata read hostname 'smithi092' 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66f7f60 0x55dbb6b2a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 4 client.15237 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "status"}) -- 0x55dbb6a13610 con 0x55dbb66f7f60 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852259c640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66f7f60 0x55dbb6b2a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f8509ffb640 10 client.15237 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852259c640 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66f7f60 0x55dbb6b2a790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a737f6f3f98ef73f server_cookie=729a1548f9703fe6 in_seq=0 out_seq=0 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f8509ffb640 1 -- 172.21.15.92:0/2610862980 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dbb6a13610 con 0x55dbb66f7f60 2024-09-21T02:03:52.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f8509ffb640 10 client.15237 handle_command_reply: tid=0 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.806+0000 7f852f150740 1 client.15237 shutdown 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 msgr2=0x55dbb6abe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6abe660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c950 tx=0x7f851c01e420 comp rx=0 tx=0).stop 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 shutdown_connections 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dbb66f7f60 0x55dbb6b2a790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dbb66d7280 0x55dbb6abe660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbb66cf0a0 0x55dbb6abe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 --2- 172.21.15.92:0/2610862980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbb66b68e0 0x55dbb6ac4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8509ffb640 0 client.15237 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 >> 172.21.15.92:0/2610862980 conn(0x55dbb66ec350 msgr2=0x55dbb6ab9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 shutdown_connections 2024-09-21T02:03:52.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 1 -- 172.21.15.92:0/2610862980 wait complete. 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f852f150740 20 client.15237 trim_cache size 0 max 16384 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24637, 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 80.837276568999997, 2024-09-21T02:03:52.814 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 99.663592808000004 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f850c0b6620 msgr2=0x7f850c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f850c0b6620 0x7f850c0b8b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55dbb6a9cb80 tx=0x7f851400c430 comp rx=0 tx=0).stop 2024-09-21T02:03:52.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 msgr2=0x7f852419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f852419abf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f851802a930 tx=0x7f8518004270 comp rx=0 tx=0).stop 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 shutdown_connections 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f850c0b6620 0x7f850c0b8b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524108d70 0x7f852419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8524108420 0x7f852419abf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 --2- 172.21.15.92:0/1699073025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8524102ae0 0x7f852419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.807+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 >> 172.21.15.92:0/1699073025 conn(0x7f8524079770 msgr2=0x7f8524101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.808+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 shutdown_connections 2024-09-21T02:03:52.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:52.808+0000 7f8528bdc640 1 -- 172.21.15.92:0/1699073025 wait complete. 2024-09-21T02:03:52.967 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:52 smithi092 ceph-mon[31190]: pgmap v327: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:53.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:52 smithi099 ceph-mon[34404]: pgmap v327: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:53.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:52 smithi136 ceph-mon[34754]: pgmap v327: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:53.455 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T02:03:53.455 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T02:03:53.464 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:03:53.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T02:03:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 -- 172.21.15.92:0/814034927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 msgr2=0x7fb56010f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 --2- 172.21.15.92:0/814034927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb56010f610 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb550009900 tx=0x7fb55002a480 comp rx=0 tx=0).stop 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 -- 172.21.15.92:0/814034927 shutdown_connections 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 --2- 172.21.15.92:0/814034927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 0x7fb560116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 --2- 172.21.15.92:0/814034927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb56010f610 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 --2- 172.21.15.92:0/814034927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56010e170 0x7fb56010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 -- 172.21.15.92:0/814034927 >> 172.21.15.92:0/814034927 conn(0x7fb56006e6a0 msgr2=0x7fb56010c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 -- 172.21.15.92:0/814034927 shutdown_connections 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.171+0000 7fb567e62640 1 -- 172.21.15.92:0/814034927 wait complete. 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 Processor -- start 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 -- start start 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56010e170 0x7fb5601a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb5601a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 0x7fb5601a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb560119e30 con 0x7fb56010f200 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb560119cb0 con 0x7fb56010fba0 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb567e62640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb560119fb0 con 0x7fb56010e170 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb5653fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb5601a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb5653fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb5601a5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41720/0 (socket says 172.21.15.92:41720) 2024-09-21T02:03:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.172+0000 7fb5653fd640 1 -- 172.21.15.92:0/1107712327 learned_addr learned my addr 172.21.15.92:0/1107712327 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb565bfe640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56010e170 0x7fb5601a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5663ff640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 0x7fb5601a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5653fd640 1 -- 172.21.15.92:0/1107712327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56010e170 msgr2=0x7fb5601a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5653fd640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56010e170 0x7fb5601a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5653fd640 1 -- 172.21.15.92:0/1107712327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 msgr2=0x7fb5601a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5653fd640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 0x7fb5601a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5653fd640 1 -- 172.21.15.92:0/1107712327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5601a64a0 con 0x7fb56010f200 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5663ff640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 0x7fb5601a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb5653fd640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb5601a5150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb550006010 tx=0x7fb550004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55003f030 con 0x7fb56010f200 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb550003cb0 con 0x7fb56010f200 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5601a6730 con 0x7fb56010f200 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550035070 con 0x7fb56010f200 2024-09-21T02:03:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.173+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5601acd40 con 0x7fb56010f200 2024-09-21T02:03:54.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.174+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb560115350 con 0x7fb56010f200 2024-09-21T02:03:54.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.176+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fb550042030 con 0x7fb56010f200 2024-09-21T02:03:54.177 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.177+0000 7fb556ffd640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb54c0b65a0 0x7fb54c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.177+0000 7fb565bfe640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb54c0b65a0 0x7fb54c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.177+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb550108260 con 0x7fb56010f200 2024-09-21T02:03:54.178 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.177+0000 7fb565bfe640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb54c0b65a0 0x7fb54c0b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005fd0 tx=0x7fb55c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.181+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5500ce970 con 0x7fb56010f200 2024-09-21T02:03:54.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.372+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb560063800 con 0x7fb54c0b65a0 2024-09-21T02:03:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.373+0000 7fb556ffd640 1 -- 172.21.15.92:0/1107712327 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb560063800 con 0x7fb54c0b65a0 2024-09-21T02:03:54.374 INFO:teuthology.orchestra.run.smithi092.stdout:Backend: cephadm 2024-09-21T02:03:54.374 INFO:teuthology.orchestra.run.smithi092.stdout:Available: Yes 2024-09-21T02:03:54.374 INFO:teuthology.orchestra.run.smithi092.stdout:Paused: No 2024-09-21T02:03:54.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb54c0b65a0 msgr2=0x7fb54c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb54c0b65a0 0x7fb54c0b8ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005fd0 tx=0x7fb55c00b390 comp rx=0 tx=0).stop 2024-09-21T02:03:54.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 msgr2=0x7fb5601a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb5601a5150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb550006010 tx=0x7fb550004340 comp rx=0 tx=0).stop 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 shutdown_connections 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb54c0b65a0 0x7fb54c0b8ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb56010fba0 0x7fb5601a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56010f200 0x7fb5601a5150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 --2- 172.21.15.92:0/1107712327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56010e170 0x7fb5601a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 >> 172.21.15.92:0/1107712327 conn(0x7fb56006e6a0 msgr2=0x7fb560112520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 shutdown_connections 2024-09-21T02:03:54.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:54.376+0000 7fb567e62640 1 -- 172.21.15.92:0/1107712327 wait complete. 2024-09-21T02:03:54.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T02:03:55.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:54 smithi092 ceph-mon[31190]: pgmap v328: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:55.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:54 smithi099 ceph-mon[34404]: pgmap v328: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:55.084 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:54 smithi136 ceph-mon[34754]: pgmap v328: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:03:55.213 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:55.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.693+0000 7fbd71e49640 1 -- 172.21.15.92:0/1306563472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c103750 msgr2=0x7fbd6c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.693+0000 7fbd71e49640 1 --2- 172.21.15.92:0/1306563472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c103750 0x7fbd6c103bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009900 tx=0x7fbd5c02a480 comp rx=0 tx=0).stop 2024-09-21T02:03:55.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 -- 172.21.15.92:0/1306563472 shutdown_connections 2024-09-21T02:03:55.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 --2- 172.21.15.92:0/1306563472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd6c105180 0x7fbd6c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 --2- 172.21.15.92:0/1306563472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 --2- 172.21.15.92:0/1306563472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c103750 0x7fbd6c103bc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 -- 172.21.15.92:0/1306563472 >> 172.21.15.92:0/1306563472 conn(0x7fbd6c0ff0c0 msgr2=0x7fbd6c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 -- 172.21.15.92:0/1306563472 shutdown_connections 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 -- 172.21.15.92:0/1306563472 wait complete. 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 Processor -- start 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.694+0000 7fbd71e49640 1 -- start start 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd6c103750 0x7fbd6c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd6c105180 0x7fbd6c19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c111670 con 0x7fbd6c1047e0 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1114f0 con 0x7fbd6c105180 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1117f0 con 0x7fbd6c103750 2024-09-21T02:03:55.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41730/0 (socket says 172.21.15.92:41730) 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 -- 172.21.15.92:0/968074178 learned_addr learned my addr 172.21.15.92:0/968074178 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6bfff640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd6c105180 0x7fbd6c19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6b7fe640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd6c103750 0x7fbd6c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 -- 172.21.15.92:0/968074178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd6c103750 msgr2=0x7fbd6c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd6c103750 0x7fbd6c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 -- 172.21.15.92:0/968074178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd6c105180 msgr2=0x7fbd6c19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd6c105180 0x7fbd6c19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 -- 172.21.15.92:0/968074178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c19dd30 con 0x7fbd6c1047e0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd6affd640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c19c970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b4a0 tx=0x7fbd6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60014070 con 0x7fbd6c1047e0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c19e020 con 0x7fbd6c1047e0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd60010ac0 con 0x7fbd6c1047e0 2024-09-21T02:03:55.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60017c30 con 0x7fbd6c1047e0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.695+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c1a4720 con 0x7fbd6c1047e0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.696+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c10a930 con 0x7fbd6c1047e0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.698+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fbd60017070 con 0x7fbd6c1047e0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.698+0000 7fbd68ff9640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbd540b65a0 0x7fbd540b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.698+0000 7fbd6b7fe640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbd540b65a0 0x7fbd540b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.699+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbd600e6b90 con 0x7fbd6c1047e0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.699+0000 7fbd6b7fe640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbd540b65a0 0x7fbd540b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c02a9a0 tx=0x7fbd5c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:55.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.702+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd600ad2a0 con 0x7fbd6c1047e0 2024-09-21T02:03:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.889+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbd6c063800 con 0x7fbd540b65a0 2024-09-21T02:03:55.890 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:55 smithi092 ceph-mon[31190]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.901+0000 7fbd68ff9640 1 -- 172.21.15.92:0/968074178 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fbd6c063800 con 0x7fbd540b65a0 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:mds.a smithi092 running (97s) 19s ago 97s 85.0M - 19.3.0-5120-g18432218 ddce307e1f32 acd43fff26f0 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:mds.b smithi099 running (94s) 27s ago 94s 85.5M - 19.3.0-5120-g18432218 ddce307e1f32 4186d3a6afba 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:mds.c smithi136 running (108s) 27s ago 107s 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 66597b4ff0b7 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:mds.d smithi092 running (116s) 19s ago 115s 82.0M - 19.3.0-5120-g18432218 ddce307e1f32 20a3f8a231ed 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:mds.e smithi099 running (113s) 27s ago 112s 85.5M - 19.3.0-5120-g18432218 ddce307e1f32 1a54c297d594 2024-09-21T02:03:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:mds.f smithi136 running (110s) 27s ago 110s 86.0M - 19.3.0-5120-g18432218 ddce307e1f32 b140a19dfe02 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mds.g smithi092 running (100s) 19s ago 99s 86.3M - 19.3.0-5120-g18432218 ddce307e1f32 982f38fa7da1 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mds.h smithi099 running (105s) 27s ago 105s 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 63964c222653 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mds.i smithi136 running (103s) 27s ago 102s 94.1M - 19.3.0-5120-g18432218 ddce307e1f32 8e7935b628a0 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mds.j smithi092 running (118s) 19s ago 118s 81.4M - 19.3.0-5120-g18432218 ddce307e1f32 c5d7e7469ad5 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mds.k smithi099 running (2m) 27s ago 2m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 424c9ed6011c 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mds.l smithi136 running (2m) 27s ago 2m 84.9M - 19.3.0-5120-g18432218 ddce307e1f32 cdcff13c35e5 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.x smithi092 *:9283,8765 running (9m) 19s ago 9m 590M - 19.3.0-5120-g18432218 ddce307e1f32 024a4f715ef9 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.y smithi099 *:8443,8765 running (7m) 27s ago 7m 527M - 19.3.0-5120-g18432218 ddce307e1f32 9c2cd6367ad0 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.z smithi136 *:8443,8765 running (7m) 27s ago 7m 524M - 19.3.0-5120-g18432218 ddce307e1f32 4917858bb7a6 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a smithi092 running (9m) 19s ago 9m 120M 2048M 19.3.0-5120-g18432218 ddce307e1f32 082f4472a4da 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b smithi099 running (8m) 27s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 8ed0cbe597e5 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c smithi136 running (8m) 27s ago 8m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 e245b3464d87 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (6m) 19s ago 6m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 e7f5cee346ee 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi099 running (6m) 27s ago 6m 126M 4272M 19.3.0-5120-g18432218 ddce307e1f32 31a60bbe08b1 2024-09-21T02:03:55.903 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi136 running (6m) 27s ago 6m 117M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1bcce4f45573 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi092 running (5m) 19s ago 5m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ac8a8b2b877b 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi099 running (5m) 27s ago 5m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 82c3ec017e4a 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi136 running (4m) 27s ago 4m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 11097b715617 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 smithi092 running (4m) 19s ago 4m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 6b240f89d58c 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 smithi099 running (4m) 27s ago 4m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 cad197159864 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.8 smithi136 running (3m) 27s ago 3m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 b48df93016dc 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.9 smithi092 running (3m) 19s ago 3m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c72060b71898 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.10 smithi099 running (2m) 27s ago 2m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 2df28c490ca6 2024-09-21T02:03:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.11 smithi136 running (2m) 27s ago 2m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 2015d2b33f89 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbd540b65a0 msgr2=0x7fbd540b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbd540b65a0 0x7fbd540b8ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c02a9a0 tx=0x7fbd5c005fd0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 msgr2=0x7fbd6c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c19c970 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b4a0 tx=0x7fbd6000b960 comp rx=0 tx=0).stop 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 shutdown_connections 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbd540b65a0 0x7fbd540b8ab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd6c105180 0x7fbd6c19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6c1047e0 0x7fbd6c19c970 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 --2- 172.21.15.92:0/968074178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd6c103750 0x7fbd6c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 >> 172.21.15.92:0/968074178 conn(0x7fbd6c0ff0c0 msgr2=0x7fbd6c107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:55.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 shutdown_connections 2024-09-21T02:03:55.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:55.904+0000 7fbd71e49640 1 -- 172.21.15.92:0/968074178 wait complete. 2024-09-21T02:03:56.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:55 smithi099 ceph-mon[34404]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:56.105 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:55 smithi136 ceph-mon[34754]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:56.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T02:03:56.800 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:57.051 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:56 smithi092 ceph-mon[31190]: pgmap v329: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:57.051 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:56 smithi092 ceph-mon[31190]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:57.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:56 smithi099 ceph-mon[34404]: pgmap v329: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:57.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:56 smithi099 ceph-mon[34404]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:57.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 -- 172.21.15.92:0/381448850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494104810 msgr2=0x7f2494104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 --2- 172.21.15.92:0/381448850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494104810 0x7f2494104c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2488009900 tx=0x7f248802a480 comp rx=0 tx=0).stop 2024-09-21T02:03:57.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 -- 172.21.15.92:0/381448850 shutdown_connections 2024-09-21T02:03:57.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 --2- 172.21.15.92:0/381448850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24941051b0 0x7f249410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 --2- 172.21.15.92:0/381448850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494104810 0x7f2494104c20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 --2- 172.21.15.92:0/381448850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494103780 0x7f2494103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 -- 172.21.15.92:0/381448850 >> 172.21.15.92:0/381448850 conn(0x7f24940ff0f0 msgr2=0x7f24941015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 -- 172.21.15.92:0/381448850 shutdown_connections 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.250+0000 7f249bbd8640 1 -- 172.21.15.92:0/381448850 wait complete. 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 Processor -- start 2024-09-21T02:03:57.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 -- start start 2024-09-21T02:03:57.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 0x7f249419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104810 0x7f249419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24941051b0 0x7f249419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f249410f3b0 con 0x7f2494103780 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f249410f230 con 0x7f24941051b0 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249bbd8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f249410f530 con 0x7f2494104810 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 0x7f249419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 0x7f249419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41754/0 (socket says 172.21.15.92:41754) 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 -- 172.21.15.92:0/3589611194 learned_addr learned my addr 172.21.15.92:0/3589611194 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499173640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104810 0x7f249419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f249a175640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24941051b0 0x7f249419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 -- 172.21.15.92:0/3589611194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104810 msgr2=0x7f249419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104810 0x7f249419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 -- 172.21.15.92:0/3589611194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24941051b0 msgr2=0x7f249419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24941051b0 0x7f249419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.251+0000 7f2499974640 1 -- 172.21.15.92:0/3589611194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249419ba40 con 0x7f2494103780 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.252+0000 7f2499974640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 0x7f249419a010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f248400ebc0 tx=0x7f248400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.252+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484010630 con 0x7f2494103780 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.252+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2484010c80 con 0x7f2494103780 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.252+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249419bd30 con 0x7f2494103780 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.252+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484018b60 con 0x7f2494103780 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.252+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a2430 con 0x7f2494103780 2024-09-21T02:03:57.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.253+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249410a960 con 0x7f2494103780 2024-09-21T02:03:57.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.255+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f2484018d00 con 0x7f2494103780 2024-09-21T02:03:57.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.256+0000 7f2492ffd640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f247c0b65a0 0x7f247c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.256+0000 7f2499173640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f247c0b65a0 0x7f247c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.256+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f24840e7f60 con 0x7f2494103780 2024-09-21T02:03:57.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.257+0000 7f2499173640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f247c0b65a0 0x7f247c0b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2488000c00 tx=0x7f2488031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:57.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.259+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24840b2a50 con 0x7f2494103780 2024-09-21T02:03:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:56 smithi136 ceph-mon[34754]: pgmap v329: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:03:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:56 smithi136 ceph-mon[34754]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.448+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2494063800 con 0x7f247c0b65a0 2024-09-21T02:03:57.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.454+0000 7f2492ffd640 1 -- 172.21.15.92:0/3589611194 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2494063800 con 0x7f247c0b65a0 2024-09-21T02:03:57.455 INFO:teuthology.orchestra.run.smithi092.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T02:03:57.455 INFO:teuthology.orchestra.run.smithi092.stdout:mds.all 12/12 28s ago 2m smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-09-21T02:03:57.455 INFO:teuthology.orchestra.run.smithi092.stdout:mgr 3/3 28s ago 7m smithi092=x;smithi099=y;smithi136=z;count:3 2024-09-21T02:03:57.455 INFO:teuthology.orchestra.run.smithi092.stdout:mon 3/3 28s ago 8m smithi092:172.21.15.92=a;smithi099:172.21.15.99=b;smithi136:172.21.15.136=c;count:3 2024-09-21T02:03:57.455 INFO:teuthology.orchestra.run.smithi092.stdout:osd 12 28s ago - 2024-09-21T02:03:57.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f247c0b65a0 msgr2=0x7f247c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f247c0b65a0 0x7f247c0b8ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2488000c00 tx=0x7f2488031400 comp rx=0 tx=0).stop 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 msgr2=0x7f249419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 0x7f249419a010 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f248400ebc0 tx=0x7f248400c6a0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 shutdown_connections 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f247c0b65a0 0x7f247c0b8ab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24941051b0 0x7f249419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2494104810 0x7f249419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 --2- 172.21.15.92:0/3589611194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2494103780 0x7f249419a010 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 >> 172.21.15.92:0/3589611194 conn(0x7f24940ff0f0 msgr2=0x7f2494107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 shutdown_connections 2024-09-21T02:03:57.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:57.456+0000 7f249bbd8640 1 -- 172.21.15.92:0/3589611194 wait complete. 2024-09-21T02:03:58.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T02:03:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:03:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 -- 172.21.15.92:0/2671930446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994102570 msgr2=0x7fa994108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 --2- 172.21.15.92:0/2671930446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994102570 0x7fa994108d70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009900 tx=0x7fa97c02a410 comp rx=0 tx=0).stop 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 -- 172.21.15.92:0/2671930446 shutdown_connections 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 --2- 172.21.15.92:0/2671930446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9941094b0 0x7fa99410b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 --2- 172.21.15.92:0/2671930446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994102570 0x7fa994108d70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 --2- 172.21.15.92:0/2671930446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994101b40 0x7fa994101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 -- 172.21.15.92:0/2671930446 >> 172.21.15.92:0/2671930446 conn(0x7fa9940fd770 msgr2=0x7fa9940ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 -- 172.21.15.92:0/2671930446 shutdown_connections 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.789+0000 7fa999484640 1 -- 172.21.15.92:0/2671930446 wait complete. 2024-09-21T02:03:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 Processor -- start 2024-09-21T02:03:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 -- start start 2024-09-21T02:03:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 0x7fa99419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994102570 0x7fa99419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9941094b0 0x7fa99419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410f490 con 0x7fa994101b40 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410f310 con 0x7fa9941094b0 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa999484640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa99410f610 con 0x7fa994102570 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 0x7fa99419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.790+0000 7fa992ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 0x7fa99419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41770/0 (socket says 172.21.15.92:41770) 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 -- 172.21.15.92:0/2669991285 learned_addr learned my addr 172.21.15.92:0/2669991285 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa98a7fc640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994102570 0x7fa99419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa9937fe640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9941094b0 0x7fa99419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 -- 172.21.15.92:0/2669991285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994102570 msgr2=0x7fa99419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994102570 0x7fa99419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 -- 172.21.15.92:0/2669991285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9941094b0 msgr2=0x7fa99419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9941094b0 0x7fa99419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 -- 172.21.15.92:0/2669991285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99419bb20 con 0x7fa994101b40 2024-09-21T02:03:58.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa98a7fc640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994102570 0x7fa99419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa992ffd640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 0x7fa99419a0f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa98400b4a0 tx=0x7fa98400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984014070 con 0x7fa994101b40 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa984010ac0 con 0x7fa994101b40 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99419be10 con 0x7fa994101b40 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984017c30 con 0x7fa994101b40 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.791+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941a2510 con 0x7fa994101b40 2024-09-21T02:03:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.792+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9940ff290 con 0x7fa994101b40 2024-09-21T02:03:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.794+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7fa984017070 con 0x7fa994101b40 2024-09-21T02:03:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.794+0000 7fa990ff9640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9800b65a0 0x7fa9800b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.795+0000 7fa98a7fc640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9800b65a0 0x7fa9800b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.795+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa9840e6c10 con 0x7fa994101b40 2024-09-21T02:03:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.795+0000 7fa98a7fc640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9800b65a0 0x7fa9800b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005f70 tx=0x7fa97c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.798+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9840ad320 con 0x7fa994101b40 2024-09-21T02:03:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.983+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa994063800 con 0x7fa9800b65a0 2024-09-21T02:03:58.984 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:58 smithi092 ceph-mon[31190]: pgmap v330: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:58.984 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:58 smithi092 ceph-mon[31190]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.985+0000 7fa990ff9640 1 -- 172.21.15.92:0/2669991285 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa994063800 con 0x7fa9800b65a0 2024-09-21T02:03:58.985 INFO:teuthology.orchestra.run.smithi092.stdout:HOST ADDR LABELS STATUS 2024-09-21T02:03:58.985 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 172.21.15.92 2024-09-21T02:03:58.985 INFO:teuthology.orchestra.run.smithi092.stdout:smithi099 172.21.15.99 2024-09-21T02:03:58.985 INFO:teuthology.orchestra.run.smithi092.stdout:smithi136 172.21.15.136 2024-09-21T02:03:58.986 INFO:teuthology.orchestra.run.smithi092.stdout:3 hosts in cluster 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9800b65a0 msgr2=0x7fa9800b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9800b65a0 0x7fa9800b8ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005f70 tx=0x7fa97c000bc0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 msgr2=0x7fa99419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 0x7fa99419a0f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa98400b4a0 tx=0x7fa98400b960 comp rx=0 tx=0).stop 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 shutdown_connections 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9800b65a0 0x7fa9800b8ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9941094b0 0x7fa99419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994102570 0x7fa99419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 --2- 172.21.15.92:0/2669991285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994101b40 0x7fa99419a0f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 >> 172.21.15.92:0/2669991285 conn(0x7fa9940fd770 msgr2=0x7fa99410a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 shutdown_connections 2024-09-21T02:03:58.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:03:58.987+0000 7fa999484640 1 -- 172.21.15.92:0/2669991285 wait complete. 2024-09-21T02:03:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:58 smithi099 ceph-mon[34404]: pgmap v330: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:58 smithi099 ceph-mon[34404]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:58 smithi136 ceph-mon[34754]: pgmap v330: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:03:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:58 smithi136 ceph-mon[34754]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:03:59.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T02:03:59.870 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:00.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:03:59 smithi099 ceph-mon[34404]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:00.094 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:03:59 smithi092 ceph-mon[31190]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:00.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:03:59 smithi136 ceph-mon[34754]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 -- 172.21.15.92:0/3982381529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c102640 msgr2=0x7f473c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 --2- 172.21.15.92:0/3982381529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c102640 0x7f473c102ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f472c009900 tx=0x7f472c02a480 comp rx=0 tx=0).stop 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 -- 172.21.15.92:0/3982381529 shutdown_connections 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 --2- 172.21.15.92:0/3982381529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c104070 0x7f473c10aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 --2- 172.21.15.92:0/3982381529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 0x7f473c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 --2- 172.21.15.92:0/3982381529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c102640 0x7f473c102ab0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 -- 172.21.15.92:0/3982381529 >> 172.21.15.92:0/3982381529 conn(0x7f473c0fdfb0 msgr2=0x7f473c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 -- 172.21.15.92:0/3982381529 shutdown_connections 2024-09-21T02:04:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.332+0000 7f47408be640 1 -- 172.21.15.92:0/3982381529 wait complete. 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 Processor -- start 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 -- start start 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c102640 0x7f473c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 0x7f473c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 0x7f473c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10e280 con 0x7f473c104070 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10e100 con 0x7f473c102640 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f47408be640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f473c10e400 con 0x7f473c1036d0 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f473ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 0x7f473c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f473ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 0x7f473c19b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41778/0 (socket says 172.21.15.92:41778) 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.333+0000 7f473ad9d640 1 -- 172.21.15.92:0/2173114985 learned_addr learned my addr 172.21.15.92:0/2173114985 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:00.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473a59c640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c102640 0x7f473c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f4733fff640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 0x7f473c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473ad9d640 1 -- 172.21.15.92:0/2173114985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 msgr2=0x7f473c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473ad9d640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 0x7f473c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473ad9d640 1 -- 172.21.15.92:0/2173114985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c102640 msgr2=0x7f473c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473ad9d640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c102640 0x7f473c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473ad9d640 1 -- 172.21.15.92:0/2173114985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c19ba30 con 0x7f473c104070 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f4733fff640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 0x7f473c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473a59c640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c102640 0x7f473c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f473ad9d640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 0x7f473c19b210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f473400cad0 tx=0x7f473400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400ae20 con 0x7f473c104070 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4734005020 con 0x7f473c104070 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400f6b0 con 0x7f473c104070 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c19bd20 con 0x7f473c104070 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.334+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a2420 con 0x7f473c104070 2024-09-21T02:04:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.335+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c1a25f0 con 0x7f473c104070 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.337+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f47340051d0 con 0x7f473c104070 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.338+0000 7f47337fe640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47280b65a0 0x7f47280b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.338+0000 7f473a59c640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47280b65a0 0x7f47280b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.338+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47340e79d0 con 0x7f473c104070 2024-09-21T02:04:00.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.338+0000 7f473a59c640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47280b65a0 0x7f47280b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f472c031440 tx=0x7f472c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.341+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4734014030 con 0x7f473c104070 2024-09-21T02:04:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.527+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f473c0fff90 con 0x7f47280b65a0 2024-09-21T02:04:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.529+0000 7f47337fe640 1 -- 172.21.15.92:0/2173114985 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f473c0fff90 con 0x7f47280b65a0 2024-09-21T02:04:00.530 INFO:teuthology.orchestra.run.smithi092.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T02:04:00.530 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T02:04:00.530 INFO:teuthology.orchestra.run.smithi092.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T02:04:00.530 INFO:teuthology.orchestra.run.smithi092.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 2m ago Has a FileSystem, LVM detected 2024-09-21T02:04:00.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47280b65a0 msgr2=0x7f47280b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47280b65a0 0x7f47280b8ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f472c031440 tx=0x7f472c005d50 comp rx=0 tx=0).stop 2024-09-21T02:04:00.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 msgr2=0x7f473c19b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 0x7f473c19b210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f473400cad0 tx=0x7f473400cf90 comp rx=0 tx=0).stop 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 shutdown_connections 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47280b65a0 0x7f47280b8ab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f473c104070 0x7f473c19b210 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f473c1036d0 0x7f473c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 --2- 172.21.15.92:0/2173114985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c102640 0x7f473c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 >> 172.21.15.92:0/2173114985 conn(0x7f473c0fdfb0 msgr2=0x7f473c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.531+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 shutdown_connections 2024-09-21T02:04:00.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:00.532+0000 7f47408be640 1 -- 172.21.15.92:0/2173114985 wait complete. 2024-09-21T02:04:01.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:00 smithi099 ceph-mon[34404]: pgmap v331: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:04:01.180 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:00 smithi092 ceph-mon[31190]: pgmap v331: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:04:01.208 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T02:04:01.225 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:04:01.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T02:04:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:00 smithi136 ceph-mon[34754]: pgmap v331: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:04:01.472 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:01.898 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:01 smithi092 ceph-mon[31190]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:01.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 -- 172.21.15.92:0/2559989648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac104070 msgr2=0x7ff0ac10aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/2559989648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac104070 0x7ff0ac10aa10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009ca0 tx=0x7ff0a402a950 comp rx=0 tx=0).stop 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 -- 172.21.15.92:0/2559989648 shutdown_connections 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/2559989648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac104070 0x7ff0ac10aa10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/2559989648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac1036d0 0x7ff0ac103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/2559989648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ac102640 0x7ff0ac102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 -- 172.21.15.92:0/2559989648 >> 172.21.15.92:0/2559989648 conn(0x7ff0ac0fdfb0 msgr2=0x7ff0ac100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 -- 172.21.15.92:0/2559989648 shutdown_connections 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.989+0000 7ff0b2e67640 1 -- 172.21.15.92:0/2559989648 wait complete. 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 Processor -- start 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 -- start start 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 0x7ff0ac199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ac1036d0 0x7ff0ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac104070 0x7ff0ac19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10f3b0 con 0x7ff0ac102640 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10f230 con 0x7ff0ac1036d0 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b2e67640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10f530 con 0x7ff0ac104070 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 0x7ff0ac199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 0x7ff0ac199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41802/0 (socket says 172.21.15.92:41802) 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 -- 172.21.15.92:0/236159199 learned_addr learned my addr 172.21.15.92:0/236159199 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b1404640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac104070 0x7ff0ac19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 -- 172.21.15.92:0/236159199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac104070 msgr2=0x7ff0ac19b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0abfff640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ac1036d0 0x7ff0ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac104070 0x7ff0ac19b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 -- 172.21.15.92:0/236159199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ac1036d0 msgr2=0x7ff0ac19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ac1036d0 0x7ff0ac19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b0c03640 1 -- 172.21.15.92:0/236159199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac19b8f0 con 0x7ff0ac102640 2024-09-21T02:04:02.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.990+0000 7ff0b1404640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac104070 0x7ff0ac19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.991+0000 7ff0b0c03640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 0x7ff0ac199f50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c920 tx=0x7ff0a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.991+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0007c70 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.991+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a0007e10 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.991+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0018820 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.991+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac19bbe0 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.991+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1a2370 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.992+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac109820 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.993+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7ff0a000f450 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.994+0000 7ff0a9ffb640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff09c0b65a0 0x7ff09c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.994+0000 7ff0abfff640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff09c0b65a0 0x7ff09c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.994+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff0a00e7960 con 0x7ff0ac102640 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.995+0000 7ff0abfff640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff09c0b65a0 0x7ff09c0b8ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff098009aa0 tx=0x7ff098005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:01.997+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0a0014030 con 0x7ff0ac102640 2024-09-21T02:04:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:01 smithi099 ceph-mon[34404]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.213+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff0ac02e640 con 0x7ff0ac102640 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.213+0000 7ff0a9ffb640 1 -- 172.21.15.92:0/236159199 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3121 (secure 0 0 0) 0x7ff0a00ae100 con 0x7ff0ac102640 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:e28 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:btime 2024-09-21T02:02:43:311503+0000 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 28 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-21T02:02:31.545262+0000 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-09-21T02:02:42.638635+0000 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 300 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 5 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1,2,3,4 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=24637,1=24623,2=24619,3=24610,4=14799} 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data disabled 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 0 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 24637 members: 14799,24610,24623,24619,24637 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.i{0:24637} state up:active seq 6 addr [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.h{0:24593} state up:standby-replay seq 1 addr [v2:172.21.15.99:6836/3685475120,v1:172.21.15.99:6837/3685475120] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b{1:24623} state up:active seq 6 addr [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.l{1:24574} state up:standby-replay seq 1 addr [v2:172.21.15.136:6832/2879720066,v1:172.21.15.136:6833/2879720066] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c{2:24619} state up:active seq 9 addr [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e{2:24572} state up:standby-replay seq 1 addr [v2:172.21.15.99:6834/2004123702,v1:172.21.15.99:6835/2004123702] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f{3:24610} state up:active seq 10 addr [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.k{3:24551} state up:standby-replay seq 1 addr [v2:172.21.15.99:6832/3451484092,v1:172.21.15.99:6833/3451484092] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a{4:14799} state up:active seq 7 addr [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.g{4:14793} state up:standby-replay seq 1 addr [v2:172.21.15.92:6838/4068035941,v1:172.21.15.92:6839/4068035941] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:02.217 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.j{-1:14727} state up:standby seq 1 addr [v2:172.21.15.92:6834/859830568,v1:172.21.15.92:6835/859830568] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d{-1:14739} state up:standby seq 1 addr [v2:172.21.15.92:6836/67515545,v1:172.21.15.92:6837/67515545] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.216+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff09c0b65a0 msgr2=0x7ff09c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.216+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff09c0b65a0 0x7ff09c0b8ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff098009aa0 tx=0x7ff098005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:02.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.216+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 msgr2=0x7ff0ac199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.216+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 0x7ff0ac199f50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c920 tx=0x7ff0a000cde0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 shutdown_connections 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff09c0b65a0 0x7ff09c0b8ab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0ac104070 0x7ff0ac19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ac1036d0 0x7ff0ac19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 --2- 172.21.15.92:0/236159199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0ac102640 0x7ff0ac199f50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 >> 172.21.15.92:0/236159199 conn(0x7ff0ac0fdfb0 msgr2=0x7ff0ac1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 shutdown_connections 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:02.217+0000 7ff0b2e67640 1 -- 172.21.15.92:0/236159199 wait complete. 2024-09-21T02:04:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:01 smithi136 ceph-mon[34754]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:02.886 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:02 smithi092 ceph-mon[31190]: pgmap v332: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:02.886 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:02 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/236159199' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:04:02.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T02:04:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:02 smithi099 ceph-mon[34404]: pgmap v332: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:02 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/236159199' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:04:03.159 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:02 smithi136 ceph-mon[34754]: pgmap v332: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:02 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/236159199' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.636+0000 7f7c8cec1640 1 -- 172.21.15.92:0/565889820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88103720 msgr2=0x7f7c88103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/565889820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88103720 0x7f7c88103b30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009900 tx=0x7f7c7002a480 comp rx=0 tx=0).stop 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 -- 172.21.15.92:0/565889820 shutdown_connections 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/565889820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c881040c0 0x7f7c8810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/565889820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88103720 0x7f7c88103b30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/565889820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c88102690 0x7f7c88102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 -- 172.21.15.92:0/565889820 >> 172.21.15.92:0/565889820 conn(0x7f7c880fe000 msgr2=0x7f7c881004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 -- 172.21.15.92:0/565889820 shutdown_connections 2024-09-21T02:04:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 -- 172.21.15.92:0/565889820 wait complete. 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.637+0000 7f7c8cec1640 1 Processor -- start 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 -- start start 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 0x7f7c8819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c88103720 0x7f7c8819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881040c0 0x7f7c8819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8810f460 con 0x7f7c88102690 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8810f2e0 con 0x7f7c881040c0 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8cec1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8810f5e0 con 0x7f7c88103720 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 0x7f7c8819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 0x7f7c8819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51250/0 (socket says 172.21.15.92:51250) 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 -- 172.21.15.92:0/2554463689 learned_addr learned my addr 172.21.15.92:0/2554463689 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c86d9d640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881040c0 0x7f7c8819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c7dd9b640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c88103720 0x7f7c8819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 -- 172.21.15.92:0/2554463689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c88103720 msgr2=0x7f7c8819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c88103720 0x7f7c8819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 -- 172.21.15.92:0/2554463689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881040c0 msgr2=0x7f7c8819b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881040c0 0x7f7c8819b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 -- 172.21.15.92:0/2554463689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8819ba00 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c7dd9b640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c88103720 0x7f7c8819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c86d9d640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881040c0 0x7f7c8819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.638+0000 7f7c8659c640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 0x7f7c8819a040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800d950 tx=0x7f7c7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.639+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78017070 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.639+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c78009c30 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.639+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8819bcf0 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.639+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800f620 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.639+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c881a23f0 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.640+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c88109850 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.642+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f7c78010460 con 0x7f7c88102690 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.643+0000 7f7c7ffff640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7c740b65a0 0x7f7c740b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.643+0000 7f7c7dd9b640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7c740b65a0 0x7f7c740b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.643+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7c780e71d0 con 0x7f7c88102690 2024-09-21T02:04:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.643+0000 7f7c7dd9b640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7c740b65a0 0x7f7c740b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000c00 tx=0x7f7c70031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.646+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c780ad8e0 con 0x7f7c88102690 2024-09-21T02:04:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.829+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7c8802e640 con 0x7f7c88102690 2024-09-21T02:04:03.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.829+0000 7f7c7ffff640 1 -- 172.21.15.92:0/2554463689 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5330 (secure 0 0 0) 0x7f7c78009de0 con 0x7f7c88102690 2024-09-21T02:04:03.830 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 77 2024-09-21T02:04:03.830 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:04:03.830 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-21T01:54:21.538024+0000 2024-09-21T02:04:03.830 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-09-21T02:03:12.414418+0000 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 26 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client luminous 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release squid 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 12 2024-09-21T02:04:03.831 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/2450056429,v1:172.21.15.92:6803/2450056429] [v2:172.21.15.92:6804/2450056429,v1:172.21.15.92:6805/2450056429] exists,up e256750d-9725-4215-90dc-41a4dc35dfb9 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/3284764622,v1:172.21.15.99:6801/3284764622] [v2:172.21.15.99:6802/3284764622,v1:172.21.15.99:6803/3284764622] exists,up e1b3ce9c-9425-4d9c-b63f-4a3d3181f253 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 17 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/2651717363,v1:172.21.15.136:6801/2651717363] [v2:172.21.15.136:6802/2651717363,v1:172.21.15.136:6803/2651717363] exists,up d560a67b-26ab-47d8-9a94-c9f293238171 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 25 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/2531772265,v1:172.21.15.92:6811/2531772265] [v2:172.21.15.92:6812/2531772265,v1:172.21.15.92:6813/2531772265] exists,up 200ba4ba-2dd5-4dbc-a805-d0b25c4816b9 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 30 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/1012266354,v1:172.21.15.99:6809/1012266354] [v2:172.21.15.99:6810/1012266354,v1:172.21.15.99:6811/1012266354] exists,up 39775100-fc5d-4e58-b5d8-e81dbc327b31 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/659398601,v1:172.21.15.136:6809/659398601] [v2:172.21.15.136:6810/659398601,v1:172.21.15.136:6811/659398601] exists,up fa5283a3-4b38-49cb-9edc-622f2c227ae1 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 41 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/1034212183,v1:172.21.15.92:6819/1034212183] [v2:172.21.15.92:6820/1034212183,v1:172.21.15.92:6821/1034212183] exists,up 0d1d32dc-5414-40bf-824a-230ac53e834f 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 47 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/1697733163,v1:172.21.15.99:6817/1697733163] [v2:172.21.15.99:6818/1697733163,v1:172.21.15.99:6819/1697733163] exists,up 4f02b02a-3345-49b2-aa55-bd16a66a1c7a 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.8 up in weight 1 up_from 53 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/3496989963,v1:172.21.15.136:6817/3496989963] [v2:172.21.15.136:6818/3496989963,v1:172.21.15.136:6819/3496989963] exists,up 813f34cb-ed23-4d4e-b578-0f9e906c8d34 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.9 up in weight 1 up_from 58 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/1884194979,v1:172.21.15.92:6827/1884194979] [v2:172.21.15.92:6828/1884194979,v1:172.21.15.92:6829/1884194979] exists,up 574cf4c9-c764-453d-9661-a05a8f62b827 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.10 up in weight 1 up_from 64 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/3821192974,v1:172.21.15.99:6825/3821192974] [v2:172.21.15.99:6826/3821192974,v1:172.21.15.99:6827/3821192974] exists,up 769d723d-f839-4eca-8b28-0336cdb68063 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/2995433302,v1:172.21.15.136:6825/2995433302] [v2:172.21.15.136:6826/2995433302,v1:172.21.15.136:6827/2995433302] exists,up bccf2012-84f1-4ce9-a21d-9016f7d5cce6 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.b [8,11] 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.13 [2,11] 2024-09-21T02:04:03.832 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.1d [2,4] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.13 [4,9] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.17 [4,7] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.1d [4,6] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.25 [5,9] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.28 [3,8] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.34 [4,9] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.37 [5,8] 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/4144615167 expires 2024-09-22T01:55:10.932400+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2108554208 expires 2024-09-22T01:55:10.932400+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/207017941 expires 2024-09-22T01:55:10.932400+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/3151574869 expires 2024-09-22T01:55:10.932400+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/3151574869 expires 2024-09-22T01:55:10.932400+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/3466917826 expires 2024-09-22T01:54:45.168174+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/3466917826 expires 2024-09-22T01:54:45.168174+0000 2024-09-21T02:04:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/1290991696 expires 2024-09-22T01:54:45.168174+0000 2024-09-21T02:04:03.834 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2823790923 expires 2024-09-22T01:54:45.168174+0000 2024-09-21T02:04:03.834 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2207814153 expires 2024-09-22T01:54:45.168174+0000 2024-09-21T02:04:03.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7c740b65a0 msgr2=0x7f7c740b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7c740b65a0 0x7f7c740b8ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000c00 tx=0x7f7c70031400 comp rx=0 tx=0).stop 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 msgr2=0x7f7c8819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 0x7f7c8819a040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800d950 tx=0x7f7c7800de10 comp rx=0 tx=0).stop 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 shutdown_connections 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7c740b65a0 0x7f7c740b8ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881040c0 0x7f7c8819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c88103720 0x7f7c8819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 --2- 172.21.15.92:0/2554463689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c88102690 0x7f7c8819a040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 >> 172.21.15.92:0/2554463689 conn(0x7f7c880fe000 msgr2=0x7f7c88106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 shutdown_connections 2024-09-21T02:04:03.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:03.833+0000 7f7c8cec1640 1 -- 172.21.15.92:0/2554463689 wait complete. 2024-09-21T02:04:04.463 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T02:04:04.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:04:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:05.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:04 smithi099 ceph-mon[34404]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:04:05.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:04 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2554463689' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:04:05.155 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:04 smithi092 ceph-mon[31190]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:04:05.155 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:04 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2554463689' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 -- 172.21.15.92:0/3566376402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 msgr2=0x7f575c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 --2- 172.21.15.92:0/3566376402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c10f550 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5754009ca0 tx=0x7f575402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 -- 172.21.15.92:0/3566376402 shutdown_connections 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 --2- 172.21.15.92:0/3566376402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c10f550 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 --2- 172.21.15.92:0/3566376402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f575c1083d0 0x7f575c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 --2- 172.21.15.92:0/3566376402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c102a90 0x7f575c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 -- 172.21.15.92:0/3566376402 >> 172.21.15.92:0/3566376402 conn(0x7f575c079750 msgr2=0x7f575c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 -- 172.21.15.92:0/3566376402 shutdown_connections 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.243+0000 7f5760dbb640 1 -- 172.21.15.92:0/3566376402 wait complete. 2024-09-21T02:04:05.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.244+0000 7f5760dbb640 1 Processor -- start 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.244+0000 7f5760dbb640 1 -- start start 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c102a90 0x7f575c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f575c1083d0 0x7f575c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f575c113af0 con 0x7f575c108d20 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f575c113970 con 0x7f575c1083d0 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f575c113c70 con 0x7f575c102a90 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51274/0 (socket says 172.21.15.92:51274) 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 -- 172.21.15.92:0/2651518412 learned_addr learned my addr 172.21.15.92:0/2651518412 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575a59c640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c102a90 0x7f575c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5751d9b640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f575c1083d0 0x7f575c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 -- 172.21.15.92:0/2651518412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c102a90 msgr2=0x7f575c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c102a90 0x7f575c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 -- 172.21.15.92:0/2651518412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f575c1083d0 msgr2=0x7f575c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f575c1083d0 0x7f575c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 -- 172.21.15.92:0/2651518412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c1a0370 con 0x7f575c108d20 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f575ad9d640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c19fd00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5754005cc0 tx=0x7f575402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754003cf0 con 0x7f575c108d20 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.245+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c1a0600 con 0x7f575c108d20 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.246+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540044f0 con 0x7f575c108d20 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.246+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575402fc30 con 0x7f575c108d20 2024-09-21T02:04:05.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.246+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c1a0b40 con 0x7f575c108d20 2024-09-21T02:04:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.247+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f575c103d10 con 0x7f575c108d20 2024-09-21T02:04:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.249+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f57540046a0 con 0x7f575c108d20 2024-09-21T02:04:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.250+0000 7f5753fff640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57480b65d0 0x7f57480b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.250+0000 7f575a59c640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57480b65d0 0x7f57480b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.250+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5754108b00 con 0x7f575c108d20 2024-09-21T02:04:05.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.250+0000 7f575a59c640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57480b65d0 0x7f57480b8ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f574c00abe0 tx=0x7f574c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.253+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f57540cf1b0 con 0x7f575c108d20 2024-09-21T02:04:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:04 smithi136 ceph-mon[34754]: pgmap v333: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:04:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:04 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2554463689' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:04:05.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.465+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f575c102ea0 con 0x7f575c108d20 2024-09-21T02:04:05.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.466+0000 7f5753fff640 1 -- 172.21.15.92:0/2651518412 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11963 (secure 0 0 0) 0x7f57540ca070 con 0x7f575c108d20 2024-09-21T02:04:05.467 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:05.467 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"btime":"2024-09-21T02:02:43:311503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:02:42.638635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[14799,24610,24623,24619,24637]},"id":1}]} 2024-09-21T02:04:05.468 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:04:05.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57480b65d0 msgr2=0x7f57480b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57480b65d0 0x7f57480b8ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f574c00abe0 tx=0x7f574c00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 msgr2=0x7f575c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c19fd00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5754005cc0 tx=0x7f575402f470 comp rx=0 tx=0).stop 2024-09-21T02:04:05.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 shutdown_connections 2024-09-21T02:04:05.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57480b65d0 0x7f57480b8ae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f575c108d20 0x7f575c19fd00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f575c1083d0 0x7f575c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 --2- 172.21.15.92:0/2651518412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575c102a90 0x7f575c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.468+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 >> 172.21.15.92:0/2651518412 conn(0x7f575c079750 msgr2=0x7f575c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.469+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 shutdown_connections 2024-09-21T02:04:05.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:05.469+0000 7f5760dbb640 1 -- 172.21.15.92:0/2651518412 wait complete. 2024-09-21T02:04:06.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:05 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2651518412' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:06.084 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2651518412' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:06.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:04:06.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:05 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2651518412' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:06.358 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:06.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 -- 172.21.15.92:0/1086307812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 msgr2=0x7f56001035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:06.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 --2- 172.21.15.92:0/1086307812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f56001035d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009900 tx=0x7f55f002a410 comp rx=0 tx=0).stop 2024-09-21T02:04:06.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 -- 172.21.15.92:0/1086307812 shutdown_connections 2024-09-21T02:04:06.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 --2- 172.21.15.92:0/1086307812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5600108f50 0x7f560010f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:06.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 --2- 172.21.15.92:0/1086307812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5600108600 0x7f5600108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:06.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 --2- 172.21.15.92:0/1086307812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f56001035d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:06.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 -- 172.21.15.92:0/1086307812 >> 172.21.15.92:0/1086307812 conn(0x7f5600079e30 msgr2=0x7f560007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:06.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 -- 172.21.15.92:0/1086307812 shutdown_connections 2024-09-21T02:04:06.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 -- 172.21.15.92:0/1086307812 wait complete. 2024-09-21T02:04:06.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.825+0000 7f560606c640 1 Processor -- start 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 -- start start 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f560019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5600108600 0x7f560019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5600108f50 0x7f560019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5600112c10 con 0x7f56001031c0 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5600112a90 con 0x7f5600108f50 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f560606c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5600112d90 con 0x7f5600108600 2024-09-21T02:04:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f560019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f55ff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f560019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51282/0 (socket says 172.21.15.92:51282) 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f55ff7fe640 1 -- 172.21.15.92:0/1526360664 learned_addr learned my addr 172.21.15.92:0/1526360664 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f55fffff640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5600108f50 0x7f560019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.826+0000 7f55feffd640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5600108600 0x7f560019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.827+0000 7f55ff7fe640 1 -- 172.21.15.92:0/1526360664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5600108600 msgr2=0x7f560019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.827+0000 7f55ff7fe640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5600108600 0x7f560019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.827+0000 7f55ff7fe640 1 -- 172.21.15.92:0/1526360664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5600108f50 msgr2=0x7f560019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.827+0000 7f55ff7fe640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5600108f50 0x7f560019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.827+0000 7f55ff7fe640 1 -- 172.21.15.92:0/1526360664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a06e0 con 0x7f56001031c0 2024-09-21T02:04:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.827+0000 7f55ff7fe640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f560019a770 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f55f0005940 tx=0x7f55f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.830+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f003f030 con 0x7f56001031c0 2024-09-21T02:04:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.830+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0003cb0 con 0x7f56001031c0 2024-09-21T02:04:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.830+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a0970 con 0x7f56001031c0 2024-09-21T02:04:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.830+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0035070 con 0x7f56001031c0 2024-09-21T02:04:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.830+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a7130 con 0x7f56001031c0 2024-09-21T02:04:06.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.831+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56001035d0 con 0x7f56001031c0 2024-09-21T02:04:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.832+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f55f0042030 con 0x7f56001031c0 2024-09-21T02:04:06.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.833+0000 7f55fcff9640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f55e80b65a0 0x7f55e80b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:06.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.833+0000 7f55feffd640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f55e80b65a0 0x7f55e80b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:06.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.834+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f55f0107cc0 con 0x7f56001031c0 2024-09-21T02:04:06.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.834+0000 7f55feffd640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f55e80b65a0 0x7f55e80b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f560006b1c0 tx=0x7f55f400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:06.838+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55f00d27b0 con 0x7f56001031c0 2024-09-21T02:04:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:06 smithi092 ceph-mon[31190]: pgmap v334: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:04:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.057+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f560010bd30 con 0x7f56001031c0 2024-09-21T02:04:07.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.058+0000 7f55fcff9640 1 -- 172.21.15.92:0/1526360664 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11963 (secure 0 0 0) 0x7f55f00d76f0 con 0x7f56001031c0 2024-09-21T02:04:07.058 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:07.059 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":28,"btime":"2024-09-21T02:02:43:311503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:02:42.638635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[14799,24610,24623,24619,24637]},"id":1}]} 2024-09-21T02:04:07.059 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 28 2024-09-21T02:04:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f55e80b65a0 msgr2=0x7f55e80b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f55e80b65a0 0x7f55e80b8ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f560006b1c0 tx=0x7f55f400a590 comp rx=0 tx=0).stop 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 msgr2=0x7f560019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f560019a770 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f55f0005940 tx=0x7f55f00042f0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 shutdown_connections 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f55e80b65a0 0x7f55e80b8ab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5600108f50 0x7f560019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5600108600 0x7f560019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 --2- 172.21.15.92:0/1526360664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56001031c0 0x7f560019a770 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 >> 172.21.15.92:0/1526360664 conn(0x7f5600079e30 msgr2=0x7f5600101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 shutdown_connections 2024-09-21T02:04:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:07.060+0000 7f560606c640 1 -- 172.21.15.92:0/1526360664 wait complete. 2024-09-21T02:04:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:06 smithi099 ceph-mon[34404]: pgmap v334: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:04:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:06 smithi136 ceph-mon[34754]: pgmap v334: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-21T02:04:07.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:04:07.964 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:07 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1526360664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:08.084 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:07 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1526360664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:08.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:07 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1526360664' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/2609233410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08102ae0 msgr2=0x7f4c08102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/2609233410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08102ae0 0x7f4c08102ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009ca0 tx=0x7f4bf802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/2609233410 shutdown_connections 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/2609233410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c08108d20 0x7f4c0810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/2609233410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c081083d0 0x7f4c081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/2609233410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08102ae0 0x7f4c08102ef0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/2609233410 >> 172.21.15.92:0/2609233410 conn(0x7f4c08079760 msgr2=0x7f4c08079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/2609233410 shutdown_connections 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.466+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/2609233410 wait complete. 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 Processor -- start 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 -- start start 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c08102ae0 0x7f4c08199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c081083d0 0x7f4c08199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 0x7f4c0819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08112990 con 0x7f4c08108d20 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08112810 con 0x7f4c081083d0 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c0d8c8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08112b10 con 0x7f4c08102ae0 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 0x7f4c0819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 0x7f4c0819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51306/0 (socket says 172.21.15.92:51306) 2024-09-21T02:04:08.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c077fe640 1 -- 172.21.15.92:0/1957352544 learned_addr learned my addr 172.21.15.92:0/1957352544 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c067fc640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c081083d0 0x7f4c08199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.467+0000 7f4c06ffd640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c08102ae0 0x7f4c08199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c077fe640 1 -- 172.21.15.92:0/1957352544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c08102ae0 msgr2=0x7f4c08199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c077fe640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c08102ae0 0x7f4c08199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c077fe640 1 -- 172.21.15.92:0/1957352544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c081083d0 msgr2=0x7f4c08199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c077fe640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c081083d0 0x7f4c08199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c077fe640 1 -- 172.21.15.92:0/1957352544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0819f2d0 con 0x7f4c08108d20 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c077fe640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 0x7f4c0819eba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000e970 tx=0x7f4c0000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000ccc0 con 0x7f4c08108d20 2024-09-21T02:04:08.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0000ce60 con 0x7f4c08108d20 2024-09-21T02:04:08.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0819f5c0 con 0x7f4c08108d20 2024-09-21T02:04:08.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00010430 con 0x7f4c08108d20 2024-09-21T02:04:08.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.468+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a5d20 con 0x7f4c08108d20 2024-09-21T02:04:08.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.469+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08102ef0 con 0x7f4c08108d20 2024-09-21T02:04:08.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.471+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f4c00004f70 con 0x7f4c08108d20 2024-09-21T02:04:08.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.471+0000 7f4c0c8c6640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4bf00b65a0 0x7f4bf00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.472+0000 7f4c06ffd640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4bf00b65a0 0x7f4bf00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.472+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4c000e7a50 con 0x7f4c08108d20 2024-09-21T02:04:08.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.472+0000 7f4c06ffd640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4bf00b65a0 0x7f4bf00b8ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8005e30 tx=0x7f4bf8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:08.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.475+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c00056710 con 0x7f4c08108d20 2024-09-21T02:04:08.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.653+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c0819a490 con 0x7f4c08108d20 2024-09-21T02:04:08.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.655+0000 7f4c0c8c6640 1 -- 172.21.15.92:0/1957352544 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22797 (secure 0 0 0) 0x7f4c0819a490 con 0x7f4c08108d20 2024-09-21T02:04:08.656 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:08.656 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:03:12.414418+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4bf00b65a0 msgr2=0x7f4bf00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4bf00b65a0 0x7f4bf00b8ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8005e30 tx=0x7f4bf8009360 comp rx=0 tx=0).stop 2024-09-21T02:04:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 msgr2=0x7f4c0819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 0x7f4c0819eba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000e970 tx=0x7f4c0000ed70 comp rx=0 tx=0).stop 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 shutdown_connections 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4bf00b65a0 0x7f4bf00b8ab0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c08108d20 0x7f4c0819eba0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c081083d0 0x7f4c08199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 --2- 172.21.15.92:0/1957352544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c08102ae0 0x7f4c08199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.657+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 >> 172.21.15.92:0/1957352544 conn(0x7f4c08079760 msgr2=0x7f4c081013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.658+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 shutdown_connections 2024-09-21T02:04:08.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:08.658+0000 7f4c0d8c8640 1 -- 172.21.15.92:0/1957352544 wait complete. 2024-09-21T02:04:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:08 smithi092 ceph-mon[31190]: pgmap v335: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:08.973 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:08 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1957352544' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:08 smithi136 ceph-mon[34754]: pgmap v335: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:08 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1957352544' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:08 smithi099 ceph-mon[34404]: pgmap v335: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:08 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1957352544' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:09.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T02:02:31.545262+0000', 'modified': '2024-09-21T02:02:42.638635+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24637, 'mds_1': 24623, 'mds_2': 24619, 'mds_3': 24610, 'mds_4': 14799}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14793': {'gid': 14793, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6839/4068035941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 4068035941}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 4068035941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14799': {'gid': 14799, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.92:6841/2220276047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6840', 'nonce': 2220276047}, {'type': 'v1', 'addr': '172.21.15.92:6841', 'nonce': 2220276047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/3451484092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 3451484092}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 3451484092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24572': {'gid': 24572, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/2004123702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2004123702}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2004123702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24574': {'gid': 24574, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/2879720066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 2879720066}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 2879720066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24593': {'gid': 24593, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6837/3685475120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3685475120}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3685475120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24610': {'gid': 24610, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.136:6835/2933531534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 2933531534}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2933531534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24619': {'gid': 24619, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.136:6837/3734933147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 3734933147}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 3734933147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24623': {'gid': 24623, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.99:6839/2904178656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 2904178656}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 2904178656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.136:6839/419135468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 419135468}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 419135468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24637, 'qdb_cluster': [14799, 24610, 24623, 24619, 24637]} 2024-09-21T02:04:09.434 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T02:04:09.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T02:04:09.681 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 -- 172.21.15.92:0/2596605613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 msgr2=0x7f92d41080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 --2- 172.21.15.92:0/2596605613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41080f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ca0 tx=0x7f92cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 -- 172.21.15.92:0/2596605613 shutdown_connections 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 --2- 172.21.15.92:0/2596605613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41080f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 --2- 172.21.15.92:0/2596605613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d4103170 0x7f92d41076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 --2- 172.21.15.92:0/2596605613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 0x7f92d4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 -- 172.21.15.92:0/2596605613 >> 172.21.15.92:0/2596605613 conn(0x7f92d4079790 msgr2=0x7f92d4079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 -- 172.21.15.92:0/2596605613 shutdown_connections 2024-09-21T02:04:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 -- 172.21.15.92:0/2596605613 wait complete. 2024-09-21T02:04:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.192+0000 7f92dac2b640 1 Processor -- start 2024-09-21T02:04:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 -- start start 2024-09-21T02:04:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 0x7f92d41a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d4103170 0x7f92d41a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f92d411a100 con 0x7f92d4107c30 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4119f80 con 0x7f92d4102820 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92dac2b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92d411a280 con 0x7f92d4103170 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51314/0 (socket says 172.21.15.92:51314) 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 -- 172.21.15.92:0/1052732845 learned_addr learned my addr 172.21.15.92:0/1052732845 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d89c7640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 0x7f92d41a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92cbfff640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d4103170 0x7f92d41a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 -- 172.21.15.92:0/1052732845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d4103170 msgr2=0x7f92d41a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d4103170 0x7f92d41a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 -- 172.21.15.92:0/1052732845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 msgr2=0x7f92d41a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 0x7f92d41a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 -- 172.21.15.92:0/1052732845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d41a68d0 con 0x7f92d4107c30 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d89c7640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 0x7f92d41a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.193+0000 7f92d91c8640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41a6260 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009430 tx=0x7f92cc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.194+0000 7f92c9ffb640 1 -- 172.21.15.92:0/1052732845 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc003c30 con 0x7f92d4107c30 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.194+0000 7f92c9ffb640 1 -- 172.21.15.92:0/1052732845 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92cc003dd0 con 0x7f92d4107c30 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.194+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d41a6b60 con 0x7f92d4107c30 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.194+0000 7f92c9ffb640 1 -- 172.21.15.92:0/1052732845 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc02faa0 con 0x7f92d4107c30 2024-09-21T02:04:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.194+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41ad3e0 con 0x7f92d4107c30 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 Processor -- start 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92c9ffb640 1 -- 172.21.15.92:0/1052732845 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f92cc02fc40 con 0x7f92d4107c30 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 -- start start 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 0x556ed612ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6130440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6130980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556ed60a7c10 con 0x556ed5d89f60 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ed6097040 con 0x556ed5d610a0 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92e119f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ed609ae50 con 0x556ed5d69280 2024-09-21T02:04:10.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 0x556ed612ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 0x556ed612ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51320/0 (socket says 172.21.15.92:51320) 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 -- 172.21.15.92:0/3014952713 learned_addr learned my addr 172.21.15.92:0/3014952713 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 -- 172.21.15.92:0/3014952713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 msgr2=0x556ed6130980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6130980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6130440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 -- 172.21.15.92:0/3014952713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 msgr2=0x556ed6130440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6130440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92cbfff640 1 -- 172.21.15.92:0/3014952713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed5df5540 con 0x556ed5d89f60 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.197+0000 7f92c9ffb640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92bc0b6650 0x7f92bc0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.198+0000 7f92d89c7640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92bc0b6650 0x7f92bc0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.198+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 0x556ed612ff00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f92c400bb50 tx=0x7f92c400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41401999a9a23c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:10.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.198+0000 7f92c9ffb640 1 -- 172.21.15.92:0/1052732845 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f92cc108810 con 0x7f92d4107c30 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.198+0000 7f92baffd640 1 -- 172.21.15.92:0/3014952713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4005220 con 0x556ed5d89f60 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.198+0000 7f92baffd640 1 -- 172.21.15.92:0/3014952713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c400be50 con 0x556ed5d89f60 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.198+0000 7f92baffd640 1 -- 172.21.15.92:0/3014952713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40116d0 con 0x556ed5d89f60 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 -- 172.21.15.92:0/3014952713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 msgr2=0x556ed612ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92d89c7640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92bc0b6650 0x7f92bc0b8b60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x556ed612eb80 tx=0x7f92d0005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 0x556ed612ff00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f92c400bb50 tx=0x7f92c400f730 comp rx=0 tx=0).stop 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 -- 172.21.15.92:0/3014952713 shutdown_connections 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6130980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6130440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 --2- 172.21.15.92:0/3014952713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d89f60 0x556ed612ff00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 -- 172.21.15.92:0/3014952713 >> 172.21.15.92:0/3014952713 conn(0x556ed5d7e350 msgr2=0x556ed612ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 -- 172.21.15.92:0/3014952713 shutdown_connections 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 -- 172.21.15.92:0/3014952713 wait complete. 2024-09-21T02:04:10.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 -- start start 2024-09-21T02:04:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.199+0000 7f92e119f740 1 Processor -- start 2024-09-21T02:04:10.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed613b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed613bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed61479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556ed5ef85f0 con 0x556ed5d69280 2024-09-21T02:04:10.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ed6032010 con 0x556ed5d610a0 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ed613e6e0 con 0x556ed5d89f60 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92e119f740 10 client.0 _ll_register_callbacks cb 0x556ed6091340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed613bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed613bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51336/0 (socket says 172.21.15.92:51336) 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 -- 172.21.15.92:0/3811943929 learned_addr learned my addr 172.21.15.92:0/3811943929 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed613b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 -- 172.21.15.92:0/3811943929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 msgr2=0x556ed61479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed61479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 -- 172.21.15.92:0/3811943929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 msgr2=0x556ed613b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed613b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92cbfff640 1 -- 172.21.15.92:0/3811943929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed605fa00 con 0x556ed5d69280 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.200+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed613b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed613bdb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f92c400bb20 tx=0x7f92c4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d0f09ca186a2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92b3fff640 1 -- 172.21.15.92:0/3811943929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4029060 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92b3fff640 1 -- 172.21.15.92:0/3811943929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f92c4004c00 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c400be50 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92b3fff640 1 -- 172.21.15.92:0/3811943929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92c4004e00 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 10 client.15309 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ed614a5f0 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92b3fff640 1 -- 172.21.15.92:0/3811943929 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c400be50 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92b3fff640 1 -- 172.21.15.92:0/3811943929 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92c402f490 con 0x556ed5d69280 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 10 client.15309 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 10 client.15309 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 20 client.15309 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d488e0 0x556ed614ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 4 client.15309 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.201+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ed60a9b60 con 0x556ed5d488e0 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.202+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d488e0 0x556ed614ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.202+0000 7f92b3fff640 10 client.15309 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.202+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d488e0 0x556ed614ddb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62544cd58fb12352 server_cookie=d2464aa2408622a7 in_seq=0 out_seq=0 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.202+0000 7f92b3fff640 1 -- 172.21.15.92:0/3811943929 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ed60a9b60 con 0x556ed5d488e0 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.202+0000 7f92b3fff640 10 client.15309 handle_command_reply: tid=0 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 client.15309 shutdown 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 msgr2=0x556ed613bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed613bdb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f92c400bb20 tx=0x7f92c4004740 comp rx=0 tx=0).stop 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 shutdown_connections 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d488e0 0x556ed614ddb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed61479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed613bdb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 --2- 172.21.15.92:0/3811943929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed613b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92b3fff640 0 client.15309 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 >> 172.21.15.92:0/3811943929 conn(0x556ed5d7e350 msgr2=0x556ed61330e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 shutdown_connections 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 1 -- 172.21.15.92:0/3811943929 wait complete. 2024-09-21T02:04:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.203+0000 7f92e119f740 20 client.15309 trim_cache size 0 max 16384 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 Processor -- start 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 -- start start 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 0x556ed6136020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6147940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d89f60 0x556ed6147e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556ed614c0f0 con 0x556ed5d610a0 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ed614c7c0 con 0x556ed5d89f60 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92e119f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ed614ce90 con 0x556ed5d69280 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92d89c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 0x556ed6136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92d89c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 0x556ed6136020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51338/0 (socket says 172.21.15.92:51338) 2024-09-21T02:04:10.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92d89c7640 1 -- 172.21.15.92:0/755165760 learned_addr learned my addr 172.21.15.92:0/755165760 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92cbfff640 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.221+0000 7f92d91c8640 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d89f60 0x556ed6147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d89c7640 1 -- 172.21.15.92:0/755165760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 msgr2=0x556ed6147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d89c7640 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d89c7640 1 -- 172.21.15.92:0/755165760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d89f60 msgr2=0x556ed6147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d89c7640 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d89f60 0x556ed6147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d89c7640 1 -- 172.21.15.92:0/755165760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed613ca90 con 0x556ed5d610a0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d91c8640 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d89f60 0x556ed6147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92d89c7640 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 0x556ed6136020 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f92d000f730 tx=0x7f92d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56506c6d9abcbf49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92ba7fc640 1 -- 172.21.15.92:0/755165760 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d002b070 con 0x556ed5d610a0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92ba7fc640 1 -- 172.21.15.92:0/755165760 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d0011db0 con 0x556ed5d610a0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.222+0000 7f92ba7fc640 1 -- 172.21.15.92:0/755165760 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00107f0 con 0x556ed5d610a0 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 -- 172.21.15.92:0/755165760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 msgr2=0x556ed6136020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 0x556ed6136020 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f92d000f730 tx=0x7f92d0015000 comp rx=0 tx=0).stop 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 -- 172.21.15.92:0/755165760 shutdown_connections 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d89f60 0x556ed6147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d69280 0x556ed6147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 --2- 172.21.15.92:0/755165760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d610a0 0x556ed6136020 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 -- 172.21.15.92:0/755165760 >> 172.21.15.92:0/755165760 conn(0x556ed5d7e350 msgr2=0x556ed613a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 -- 172.21.15.92:0/755165760 shutdown_connections 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 -- 172.21.15.92:0/755165760 wait complete. 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 -- start start 2024-09-21T02:04:10.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.223+0000 7f92e119f740 1 Processor -- start 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6154000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed6154580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed614f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556ed614dc30 con 0x556ed5d69280 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ed614d560 con 0x556ed5d610a0 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ed614e300 con 0x556ed5d89f60 2024-09-21T02:04:10.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 10 client.0 _ll_register_callbacks cb 0x7f92c400ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed6154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed6154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51348/0 (socket says 172.21.15.92:51348) 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 -- 172.21.15.92:0/3901940485 learned_addr learned my addr 172.21.15.92:0/3901940485 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92d91c8640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed614f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 -- 172.21.15.92:0/3901940485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 msgr2=0x556ed614f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed614f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 -- 172.21.15.92:0/3901940485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 msgr2=0x556ed6154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 -- 172.21.15.92:0/3901940485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed614fd90 con 0x556ed5d69280 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92d91c8640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed614f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92cbfff640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed6154580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92c400f770 tx=0x7f92c4005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1c34ff6d4985b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92b3fff640 1 -- 172.21.15.92:0/3901940485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4029040 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92b3fff640 1 -- 172.21.15.92:0/3901940485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f92c4004c00 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.224+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d0011db0 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92b3fff640 1 -- 172.21.15.92:0/3901940485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92c403f050 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 10 client.15315 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ed61556d0 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92b3fff640 1 -- 172.21.15.92:0/3901940485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d0011db0 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92b3fff640 1 -- 172.21.15.92:0/3901940485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92c40119d0 con 0x556ed5d69280 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 10 client.15315 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 10 client.15315 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 20 client.15315 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:10.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d1dc90 0x556ed61bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 4 client.15315 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "status"}) -- 0x556ed60a5610 con 0x556ed5d1dc90 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.225+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d1dc90 0x556ed61bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92b3fff640 10 client.15315 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92d89c7640 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d1dc90 0x556ed61bc450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=87f3815c6798113c server_cookie=845cc667bc5aa3bb in_seq=0 out_seq=0 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92b3fff640 1 -- 172.21.15.92:0/3901940485 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556ed60a5610 con 0x556ed5d1dc90 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92b3fff640 10 client.15315 handle_command_reply: tid=0 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 client.15315 shutdown 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 msgr2=0x556ed6154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed6154580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92c400f770 tx=0x7f92c4005540 comp rx=0 tx=0).stop 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 shutdown_connections 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x556ed5d1dc90 0x556ed61bc450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ed5d89f60 0x556ed614f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ed5d69280 0x556ed6154580 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92b3fff640 0 client.15315 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 --2- 172.21.15.92:0/3901940485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ed5d610a0 0x556ed6154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 >> 172.21.15.92:0/3901940485 conn(0x556ed5d7e350 msgr2=0x556ed612d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 shutdown_connections 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 1 -- 172.21.15.92:0/3901940485 wait complete. 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.226+0000 7f92e119f740 20 client.15315 trim_cache size 0 max 16384 2024-09-21T02:04:10.233 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 14799, 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 88.924877660000007, 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 111.46657505 2024-09-21T02:04:10.234 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92bc0b6650 msgr2=0x7f92bc0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92bc0b6650 0x7f92bc0b8b60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x556ed612eb80 tx=0x7f92d0005e70 comp rx=0 tx=0).stop 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 msgr2=0x7f92d41a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41a6260 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009430 tx=0x7f92cc0319f0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 shutdown_connections 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f92bc0b6650 0x7f92bc0b8b60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92d4107c30 0x7f92d41a6260 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d4103170 0x7f92d41a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 --2- 172.21.15.92:0/1052732845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92d4102820 0x7f92d41a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 >> 172.21.15.92:0/1052732845 conn(0x7f92d4079790 msgr2=0x7f92d4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 shutdown_connections 2024-09-21T02:04:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:10.227+0000 7f92dac2b640 1 -- 172.21.15.92:0/1052732845 wait complete. 2024-09-21T02:04:11.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T02:04:11.282 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:10 smithi136 ceph-mon[34754]: pgmap v336: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:04:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:10 smithi092 ceph-mon[31190]: pgmap v336: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:04:11.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:10 smithi099 ceph-mon[34404]: pgmap v336: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 -- 172.21.15.92:0/683157627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 msgr2=0x7f81c8110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 --2- 172.21.15.92:0/683157627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8110630 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009ca0 tx=0x7f81c002a8e0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 -- 172.21.15.92:0/683157627 shutdown_connections 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 --2- 172.21.15.92:0/683157627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8110630 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 --2- 172.21.15.92:0/683157627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 0x7f81c81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 --2- 172.21.15.92:0/683157627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 0x7f81c8103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 -- 172.21.15.92:0/683157627 >> 172.21.15.92:0/683157627 conn(0x7f81c807b910 msgr2=0x7f81c807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 -- 172.21.15.92:0/683157627 shutdown_connections 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.782+0000 7f81d01b3640 1 -- 172.21.15.92:0/683157627 wait complete. 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 Processor -- start 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 -- start start 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 0x7f81c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 0x7f81c8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8113b80 con 0x7f81c8109e00 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8113a00 con 0x7f81c8103bc0 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81d01b3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8113d00 con 0x7f81c81094b0 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51374/0 (socket says 172.21.15.92:51374) 2024-09-21T02:04:11.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 -- 172.21.15.92:0/2273930559 learned_addr learned my addr 172.21.15.92:0/2273930559 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 0x7f81c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81cd74e640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 0x7f81c8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 -- 172.21.15.92:0/2273930559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 msgr2=0x7f81c8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 0x7f81c8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 -- 172.21.15.92:0/2273930559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 msgr2=0x7f81c8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 0x7f81c8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 -- 172.21.15.92:0/2273930559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c81a88b0 con 0x7f81c8109e00 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81cd74e640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 0x7f81c8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 0x7f81c8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.783+0000 7f81ce750640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8076190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005cc0 tx=0x7f81c0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.784+0000 7f81beffd640 1 -- 172.21.15.92:0/2273930559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0004040 con 0x7f81c8109e00 2024-09-21T02:04:11.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.784+0000 7f81beffd640 1 -- 172.21.15.92:0/2273930559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0004690 con 0x7f81c8109e00 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.784+0000 7f81beffd640 1 -- 172.21.15.92:0/2273930559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0030e70 con 0x7f81c8109e00 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.784+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c81a8b40 con 0x7f81c8109e00 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.784+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a9080 con 0x7f81c8109e00 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.786+0000 7f81d6727740 1 Processor -- start 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.786+0000 7f81beffd640 1 -- 172.21.15.92:0/2273930559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f81c00305e0 con 0x7f81c8109e00 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 -- start start 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb26ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb26f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb26f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb1e6c10 con 0x55a9caea8280 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb1d6040 con 0x55a9caec8f60 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81d6727740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb1d9e50 con 0x55a9caea00a0 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb26f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb26f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51380/0 (socket says 172.21.15.92:51380) 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 -- 172.21.15.92:0/1593181623 learned_addr learned my addr 172.21.15.92:0/1593181623 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb26ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb26f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81beffd640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81b40b65a0 0x7f81b40b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 -- 172.21.15.92:0/1593181623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 msgr2=0x55a9cb26f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb26f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 -- 172.21.15.92:0/1593181623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 msgr2=0x55a9cb26ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb26ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.787+0000 7f81cd74e640 1 -- 172.21.15.92:0/1593181623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9caf34540 con 0x55a9caea8280 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.788+0000 7f81ce750640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81b40b65a0 0x7f81b40b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.788+0000 7f81beffd640 1 -- 172.21.15.92:0/2273930559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81c0108980 con 0x7f81c8109e00 2024-09-21T02:04:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.788+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb26f980 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f81b800cfc0 tx=0x7f81b800c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3f3631c3030825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.788+0000 7f81b3fff640 1 -- 172.21.15.92:0/1593181623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b801d070 con 0x55a9caea8280 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.788+0000 7f81b3fff640 1 -- 172.21.15.92:0/1593181623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b8018b20 con 0x55a9caea8280 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.788+0000 7f81b3fff640 1 -- 172.21.15.92:0/1593181623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b800f040 con 0x55a9caea8280 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 -- 172.21.15.92:0/1593181623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 msgr2=0x55a9cb26f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81ce750640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81b40b65a0 0x7f81b40b8ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f81c0030930 tx=0x7f81c0003f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb26f980 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f81b800cfc0 tx=0x7f81b800c1d0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 -- 172.21.15.92:0/1593181623 shutdown_connections 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb26f980 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb26f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 --2- 172.21.15.92:0/1593181623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb26ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 -- 172.21.15.92:0/1593181623 >> 172.21.15.92:0/1593181623 conn(0x55a9caebd350 msgr2=0x55a9cb26cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 -- 172.21.15.92:0/1593181623 shutdown_connections 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 -- 172.21.15.92:0/1593181623 wait complete. 2024-09-21T02:04:11.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 -- start start 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.789+0000 7f81d6727740 1 Processor -- start 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 0x55a9cb27a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea8280 0x55a9cb27af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb286a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb0375f0 con 0x55a9caea00a0 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb171010 con 0x55a9caec8f60 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb27d6e0 con 0x55a9caea8280 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81d6727740 10 client.0 _ll_register_callbacks cb 0x55a9cb1d0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 0x55a9cb27a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 0x55a9cb27a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51396/0 (socket says 172.21.15.92:51396) 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 -- 172.21.15.92:0/2112364665 learned_addr learned my addr 172.21.15.92:0/2112364665 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb286a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 -- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea8280 msgr2=0x55a9cb27af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea8280 0x55a9cb27af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea8280 0x55a9cb27af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 -- 172.21.15.92:0/2112364665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 msgr2=0x55a9cb286a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb286a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 -- 172.21.15.92:0/2112364665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9cb19ea00 con 0x55a9caea00a0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea8280 0x55a9cb27af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81cd74e640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 0x55a9cb27a9d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f81b800ca70 tx=0x7f81b8016670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fa892fdaa14926 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.790+0000 7f81b17fa640 1 -- 172.21.15.92:0/2112364665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b802a060 con 0x55a9caea00a0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81b17fa640 1 -- 172.21.15.92:0/2112364665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81b8018740 con 0x55a9caea00a0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b8018b20 con 0x55a9caea00a0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81b17fa640 1 -- 172.21.15.92:0/2112364665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b801d050 con 0x55a9caea00a0 2024-09-21T02:04:11.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 10 client.15327 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9cb289660 con 0x55a9caea00a0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81b17fa640 1 -- 172.21.15.92:0/2112364665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b8018b20 con 0x55a9caea00a0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81b17fa640 1 -- 172.21.15.92:0/2112364665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81b8030760 con 0x55a9caea00a0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 10 client.15327 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 10 client.15327 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 20 client.15327 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae878e0 0x55a9cb28aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 4 client.15327 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.791+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9cb1e8b60 con 0x55a9cae878e0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.792+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae878e0 0x55a9cb28aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.792+0000 7f81b17fa640 10 client.15327 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.792+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae878e0 0x55a9cb28aa10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee5317ef69484054 server_cookie=e6c7a748e0543350 in_seq=0 out_seq=0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81b17fa640 1 -- 172.21.15.92:0/2112364665 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9cb1e8b60 con 0x55a9cae878e0 2024-09-21T02:04:11.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81b17fa640 10 client.15327 handle_command_reply: tid=0 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 client.15327 shutdown 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 msgr2=0x55a9cb27a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 0x55a9cb27a9d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f81b800ca70 tx=0x7f81b8016670 comp rx=0 tx=0).stop 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 shutdown_connections 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae878e0 0x55a9cb28aa10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb286a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea8280 0x55a9cb27af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 --2- 172.21.15.92:0/2112364665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea00a0 0x55a9cb27a9d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81b17fa640 0 client.15327 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 >> 172.21.15.92:0/2112364665 conn(0x55a9caebd350 msgr2=0x55a9cb272090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 shutdown_connections 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 1 -- 172.21.15.92:0/2112364665 wait complete. 2024-09-21T02:04:11.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.793+0000 7f81d6727740 20 client.15327 trim_cache size 0 max 16384 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 Processor -- start 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 -- start start 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caea00a0 0x55a9cb26cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb274d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caec8f60 0x55a9cb2869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb28af60 con 0x55a9caea8280 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb28b630 con 0x55a9caea00a0 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81d6727740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb28bd00 con 0x55a9caec8f60 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81cdf4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb274d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81cdf4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb274d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51404/0 (socket says 172.21.15.92:51404) 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.811+0000 7f81cdf4f640 1 -- 172.21.15.92:0/3821485346 learned_addr learned my addr 172.21.15.92:0/3821485346 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:11.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cd74e640 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caea00a0 0x55a9cb26cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cdf4f640 1 -- 172.21.15.92:0/3821485346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caec8f60 msgr2=0x55a9cb2869a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caec8f60 0x55a9cb2869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cdf4f640 1 -- 172.21.15.92:0/3821485346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caea00a0 msgr2=0x55a9cb26cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caea00a0 0x55a9cb26cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cdf4f640 1 -- 172.21.15.92:0/3821485346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9cb27ba90 con 0x55a9caea8280 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb274d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55a9cb27b530 tx=0x7f81c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433e49e2cfd18e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81b37fe640 1 -- 172.21.15.92:0/3821485346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c400ad40 con 0x55a9caea8280 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81b37fe640 1 -- 172.21.15.92:0/3821485346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c401f030 con 0x55a9caea8280 2024-09-21T02:04:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.812+0000 7f81b37fe640 1 -- 172.21.15.92:0/3821485346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4014610 con 0x55a9caea8280 2024-09-21T02:04:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 -- 172.21.15.92:0/3821485346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 msgr2=0x55a9cb274d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb274d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55a9cb27b530 tx=0x7f81c4008040 comp rx=0 tx=0).stop 2024-09-21T02:04:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 -- 172.21.15.92:0/3821485346 shutdown_connections 2024-09-21T02:04:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caec8f60 0x55a9cb2869a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb274d80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 --2- 172.21.15.92:0/3821485346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caea00a0 0x55a9cb26cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 -- 172.21.15.92:0/3821485346 >> 172.21.15.92:0/3821485346 conn(0x55a9caebd350 msgr2=0x55a9cb26c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 -- 172.21.15.92:0/3821485346 shutdown_connections 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 -- 172.21.15.92:0/3821485346 wait complete. 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 -- start start 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.813+0000 7f81d6727740 1 Processor -- start 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb28f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb28f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb2949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb28caa0 con 0x55a9caea8280 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb28c3d0 con 0x55a9caec8f60 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a9cb28d170 con 0x55a9caea00a0 2024-09-21T02:04:11.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81d6727740 10 client.0 _ll_register_callbacks cb 0x7f81b801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb28f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb28f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51420/0 (socket says 172.21.15.92:51420) 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 -- 172.21.15.92:0/1801074736 learned_addr learned my addr 172.21.15.92:0/1801074736 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb2949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb28f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 -- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 msgr2=0x55a9cb28f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb28f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 -- 172.21.15.92:0/1801074736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 msgr2=0x55a9cb2949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb2949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 -- 172.21.15.92:0/1801074736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9cb294f00 con 0x55a9caea8280 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81cdf4f640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb28f770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81c40059c0 tx=0x7f81c40059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaa8981c3acb764 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:11.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.814+0000 7f81b17fa640 1 -- 172.21.15.92:0/1801074736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c402a070 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81b17fa640 1 -- 172.21.15.92:0/1801074736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81c4005c10 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c401f030 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81b17fa640 1 -- 172.21.15.92:0/1801074736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81c4036050 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 10 client.15333 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a9cb2f8710 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81b17fa640 1 -- 172.21.15.92:0/1801074736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c401f030 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81b17fa640 1 -- 172.21.15.92:0/1801074736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81c40062e0 con 0x55a9caea8280 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 10 client.15333 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 10 client.15333 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 20 client.15333 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae5cc90 0x55a9cb2fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 4 client.15333 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.815+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "status"}) -- 0x55a9cb1e4610 con 0x55a9cae5cc90 2024-09-21T02:04:11.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae5cc90 0x55a9cb2fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81cd74e640 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae5cc90 0x55a9cb2fbc40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2607ce0efcf06cb server_cookie=4ac81fac060f7f06 in_seq=0 out_seq=0 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81b17fa640 10 client.15333 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81b17fa640 1 -- 172.21.15.92:0/1801074736 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a9cb1e4610 con 0x55a9cae5cc90 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81b17fa640 10 client.15333 handle_command_reply: tid=0 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 client.15333 shutdown 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 msgr2=0x55a9cb28f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb28f770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81c40059c0 tx=0x7f81c40059f0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 shutdown_connections 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55a9cae5cc90 0x55a9cb2fbc40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9caec8f60 0x55a9cb2949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a9caea8280 0x55a9cb28f770 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81b17fa640 0 client.15333 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 --2- 172.21.15.92:0/1801074736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a9caea00a0 0x55a9cb28f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.816+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 >> 172.21.15.92:0/1801074736 conn(0x55a9caebd350 msgr2=0x55a9cb26b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 shutdown_connections 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d6727740 1 -- 172.21.15.92:0/1801074736 wait complete. 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d6727740 20 client.15333 trim_cache size 0 max 16384 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24610, 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:04:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:04:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 90.780521152999995, 2024-09-21T02:04:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:04:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:04:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:04:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 126.148269957 2024-09-21T02:04:11.824 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81b40b65a0 msgr2=0x7f81b40b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81b40b65a0 0x7f81b40b8ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f81c0030930 tx=0x7f81c0003f90 comp rx=0 tx=0).stop 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 msgr2=0x7f81c8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8076190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005cc0 tx=0x7f81c0003ba0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 shutdown_connections 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81b40b65a0 0x7f81b40b8ab0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c8109e00 0x7f81c8076190 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c81094b0 0x7f81c8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 --2- 172.21.15.92:0/2273930559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8103bc0 0x7f81c8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 >> 172.21.15.92:0/2273930559 conn(0x7f81c807b910 msgr2=0x7f81c8107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 shutdown_connections 2024-09-21T02:04:11.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:11.817+0000 7f81d01b3640 1 -- 172.21.15.92:0/2273930559 wait complete. 2024-09-21T02:04:12.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T02:04:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:13.150 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:12 smithi092 ceph-mon[31190]: pgmap v337: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 -- 172.21.15.92:0/1012983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 msgr2=0x7f55a0110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 --2- 172.21.15.92:0/1012983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a0110650 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ca0 tx=0x7f559802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 -- 172.21.15.92:0/1012983303 shutdown_connections 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 --2- 172.21.15.92:0/1012983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a0110650 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 --2- 172.21.15.92:0/1012983303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a01094d0 0x7f55a01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 --2- 172.21.15.92:0/1012983303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a0103b90 0x7f55a0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 -- 172.21.15.92:0/1012983303 >> 172.21.15.92:0/1012983303 conn(0x7f55a00ff600 msgr2=0x7f55a0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 -- 172.21.15.92:0/1012983303 shutdown_connections 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 -- 172.21.15.92:0/1012983303 wait complete. 2024-09-21T02:04:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.245+0000 7f55a74f0640 1 Processor -- start 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 -- start start 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a0103b90 0x7f55a019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a01094d0 0x7f55a019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0113ae0 con 0x7f55a0109e20 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0113960 con 0x7f55a0103b90 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a74f0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0113c60 con 0x7f55a01094d0 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37244/0 (socket says 172.21.15.92:37244) 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 -- 172.21.15.92:0/3666665426 learned_addr learned my addr 172.21.15.92:0/3666665426 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a528c640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a0103b90 0x7f55a019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a01094d0 0x7f55a019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 -- 172.21.15.92:0/3666665426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a01094d0 msgr2=0x7f55a019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a01094d0 0x7f55a019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 -- 172.21.15.92:0/3666665426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a0103b90 msgr2=0x7f55a019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a0103b90 0x7f55a019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.246+0000 7f55a5a8d640 1 -- 172.21.15.92:0/3666665426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a01a03a0 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.247+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a019fc70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5598005cc0 tx=0x7f559802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.247+0000 7f55967fc640 1 -- 172.21.15.92:0/3666665426 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598003bf0 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.247+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a01a0630 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.247+0000 7f55967fc640 1 -- 172.21.15.92:0/3666665426 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5598003d90 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.247+0000 7f55967fc640 1 -- 172.21.15.92:0/3666665426 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559802fbc0 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.247+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a6df0 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.250+0000 7f55ada64740 1 Processor -- start 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.250+0000 7f55967fc640 1 -- 172.21.15.92:0/3666665426 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 154563+0+0 (secure 0 0 0) 0x7f559802fd60 con 0x7f55a0109e20 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.250+0000 7f55ada64740 1 -- start start 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.250+0000 7f55ada64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc71f60 0x55f8de017f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55ada64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de018440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55ada64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de018980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55ada64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8ddf8fc10 con 0x55f8ddc490a0 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55ada64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8ddf7f040 con 0x55f8ddc71f60 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55ada64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8ddf82e50 con 0x55f8ddc51280 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de018440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de018440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37254/0 (socket says 172.21.15.92:37254) 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 -- 172.21.15.92:0/2894171261 learned_addr learned my addr 172.21.15.92:0/2894171261 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de018980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc71f60 0x55f8de017f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 -- 172.21.15.92:0/2894171261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 msgr2=0x55f8de018980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de018980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 -- 172.21.15.92:0/2894171261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc71f60 msgr2=0x55f8de017f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc71f60 0x55f8de017f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 -- 172.21.15.92:0/2894171261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ddcdd540 con 0x55f8ddc490a0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc71f60 0x55f8de017f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55967fc640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f558c0b65a0 0x7f558c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f558c0b65a0 0x7f558c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55a528c640 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de018440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b760 tx=0x7f559c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1100200dfa70c123 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.251+0000 7f55967fc640 1 -- 172.21.15.92:0/3666665426 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5598108920 con 0x7f55a0109e20 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.252+0000 7f558b7fe640 1 -- 172.21.15.92:0/2894171261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c003dd0 con 0x55f8ddc490a0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.252+0000 7f558b7fe640 1 -- 172.21.15.92:0/2894171261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c015070 con 0x55f8ddc490a0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.252+0000 7f558b7fe640 1 -- 172.21.15.92:0/2894171261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c004e00 con 0x55f8ddc490a0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 -- 172.21.15.92:0/2894171261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 msgr2=0x55f8de018440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f558c0b65a0 0x7f558c0b8ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55a0102e50 tx=0x7f559000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de018440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b760 tx=0x7f559c00bc20 comp rx=0 tx=0).stop 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 -- 172.21.15.92:0/2894171261 shutdown_connections 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de018980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de018440 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 --2- 172.21.15.92:0/2894171261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc71f60 0x55f8de017f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 -- 172.21.15.92:0/2894171261 >> 172.21.15.92:0/2894171261 conn(0x55f8ddc66350 msgr2=0x55f8de015df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 -- 172.21.15.92:0/2894171261 shutdown_connections 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 -- 172.21.15.92:0/2894171261 wait complete. 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 -- start start 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.253+0000 7f55ada64740 1 Processor -- start 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de023a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de023f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc71f60 0x55f8de02fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8ddde05f0 con 0x55f8ddc490a0 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8ddf1a010 con 0x55f8ddc51280 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8de0266e0 con 0x55f8ddc71f60 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55ada64740 10 client.0 _ll_register_callbacks cb 0x55f8ddf79340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de023a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de023a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37270/0 (socket says 172.21.15.92:37270) 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 -- 172.21.15.92:0/1287324929 learned_addr learned my addr 172.21.15.92:0/1287324929 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de023f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc71f60 0x55f8de02fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 -- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc71f60 msgr2=0x55f8de02fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc71f60 0x55f8de02fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 -- 172.21.15.92:0/1287324929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 msgr2=0x55f8de023f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de023f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 -- 172.21.15.92:0/1287324929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ddf47a00 con 0x55f8ddc490a0 2024-09-21T02:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f55a528c640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de023a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f559c00bfd0 tx=0x7f559c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c351f4476896d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.254+0000 7f5588ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f5588ff9640 1 -- 172.21.15.92:0/1287324929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c003de0 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f5588ff9640 1 -- 172.21.15.92:0/1287324929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f559c011440 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c015070 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f5588ff9640 1 -- 172.21.15.92:0/1287324929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f559c011640 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 10 client.15345 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8de032600 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f5588ff9640 1 -- 172.21.15.92:0/1287324929 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c015070 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f5588ff9640 1 -- 172.21.15.92:0/1287324929 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f559c025850 con 0x55f8ddc490a0 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 10 client.15345 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 10 client.15345 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 20 client.15345 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc308e0 0x55f8de035e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 4 client.15345 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.255+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8ddf91b60 con 0x55f8ddc308e0 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.256+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc308e0 0x55f8de035e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.256+0000 7f5588ff9640 10 client.15345 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.256+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc308e0 0x55f8de035e50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=950b1c8144cee65c server_cookie=4d34ccc5d60dd907 in_seq=0 out_seq=0 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f5588ff9640 1 -- 172.21.15.92:0/1287324929 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8ddf91b60 con 0x55f8ddc308e0 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f5588ff9640 10 client.15345 handle_command_reply: tid=0 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 client.15345 shutdown 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 msgr2=0x55f8de023a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de023a00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f559c00bfd0 tx=0x7f559c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 shutdown_connections 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc308e0 0x55f8de035e50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f5588ff9640 0 client.15345 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc71f60 0x55f8de02fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de023f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 --2- 172.21.15.92:0/1287324929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc490a0 0x55f8de023a00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 >> 172.21.15.92:0/1287324929 conn(0x55f8ddc66350 msgr2=0x55f8de01b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 shutdown_connections 2024-09-21T02:04:13.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 1 -- 172.21.15.92:0/1287324929 wait complete. 2024-09-21T02:04:13.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.257+0000 7f55ada64740 20 client.15345 trim_cache size 0 max 16384 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.275+0000 7f55ada64740 1 Processor -- start 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.275+0000 7f55ada64740 1 -- start start 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.275+0000 7f55ada64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de02f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.275+0000 7f55ada64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc490a0 0x55f8de02feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55ada64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de01e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55ada64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8de034190 con 0x55f8ddc308e0 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55ada64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8de034860 con 0x55f8ddc490a0 2024-09-21T02:04:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55ada64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8de034f30 con 0x55f8ddc51280 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de02f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de02f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37280/0 (socket says 172.21.15.92:37280) 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 -- 172.21.15.92:0/457653127 learned_addr learned my addr 172.21.15.92:0/457653127 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc490a0 0x55f8de02feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de01e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 -- 172.21.15.92:0/457653127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 msgr2=0x55f8de01e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de01e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 -- 172.21.15.92:0/457653127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc490a0 msgr2=0x55f8de02feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc490a0 0x55f8de02feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 -- 172.21.15.92:0/457653127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8de010080 con 0x55f8ddc308e0 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f55a528c640 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de02f970 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b520 tx=0x7f559c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5c8f1efc00a5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.276+0000 7f558affd640 1 -- 172.21.15.92:0/457653127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c034040 con 0x55f8ddc308e0 2024-09-21T02:04:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f558affd640 1 -- 172.21.15.92:0/457653127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c015070 con 0x55f8ddc308e0 2024-09-21T02:04:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f558affd640 1 -- 172.21.15.92:0/457653127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c0198b0 con 0x55f8ddc308e0 2024-09-21T02:04:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 -- 172.21.15.92:0/457653127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 msgr2=0x55f8de02f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de02f970 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b520 tx=0x7f559c005fd0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 -- 172.21.15.92:0/457653127 shutdown_connections 2024-09-21T02:04:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc51280 0x55f8de01e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc490a0 0x55f8de02feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 --2- 172.21.15.92:0/457653127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de02f970 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.277+0000 7f55ada64740 1 -- 172.21.15.92:0/457653127 >> 172.21.15.92:0/457653127 conn(0x55f8ddc66350 msgr2=0x55f8de033a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 -- 172.21.15.92:0/457653127 shutdown_connections 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 -- 172.21.15.92:0/457653127 wait complete. 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 -- start start 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 Processor -- start 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de03e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc490a0 0x55f8de037d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de0382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8de035cd0 con 0x55f8ddc308e0 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.278+0000 7f55ada64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8de035600 con 0x55f8ddc51280 2024-09-21T02:04:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55ada64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8de0363a0 con 0x55f8ddc490a0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55ada64740 10 client.0 _ll_register_callbacks cb 0x7f559000a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de03e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de03e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37292/0 (socket says 172.21.15.92:37292) 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 -- 172.21.15.92:0/1460184871 learned_addr learned my addr 172.21.15.92:0/1460184871 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc490a0 0x55f8de037d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de0382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 -- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc490a0 msgr2=0x55f8de037d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc490a0 0x55f8de037d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 -- 172.21.15.92:0/1460184871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 msgr2=0x55f8de0382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de0382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 -- 172.21.15.92:0/1460184871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8de038a20 con 0x55f8ddc308e0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a5a8d640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de0382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55a528c640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de03e8c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b760 tx=0x7f559c004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecceb76a3ba007ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f5588ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f5588ff9640 1 -- 172.21.15.92:0/1460184871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c034070 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f5588ff9640 1 -- 172.21.15.92:0/1460184871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f559c025430 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.279+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c015070 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f5588ff9640 1 -- 172.21.15.92:0/1460184871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f559c047050 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 10 client.15351 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8de0a1320 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f5588ff9640 1 -- 172.21.15.92:0/1460184871 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c015070 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f5588ff9640 1 -- 172.21.15.92:0/1460184871 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f559c038460 con 0x55f8ddc308e0 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 10 client.15351 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 20 client.15351 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc71f60 0x55f8de0a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 4 client.15351 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:13.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "status"}) -- 0x55f8ddf8d610 con 0x55f8ddc71f60 2024-09-21T02:04:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.280+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc71f60 0x55f8de0a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f5588ff9640 10 client.15351 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:04:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55a4a8b640 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc71f60 0x55f8de0a47a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b2e219ef68c7fae server_cookie=88a658d89f8ecfcb in_seq=0 out_seq=0 2024-09-21T02:04:13.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:12 smithi136 ceph-mon[34754]: pgmap v337: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:13.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f5588ff9640 1 -- 172.21.15.92:0/1460184871 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f8ddf8d610 con 0x55f8ddc71f60 2024-09-21T02:04:13.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f5588ff9640 10 client.15351 handle_command_reply: tid=0 2024-09-21T02:04:13.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 client.15351 shutdown 2024-09-21T02:04:13.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 msgr2=0x55f8de03e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de03e8c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b760 tx=0x7f559c004aa0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 shutdown_connections 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8ddc71f60 0x55f8de0a47a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8ddc51280 0x55f8de0382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8ddc490a0 0x55f8de037d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f55ada64740 1 --2- 172.21.15.92:0/1460184871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8ddc308e0 0x55f8de03e8c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.281+0000 7f5588ff9640 0 client.15351 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 >> 172.21.15.92:0/1460184871 conn(0x55f8ddc66350 msgr2=0x55f8de01e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 shutdown_connections 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55ada64740 1 -- 172.21.15.92:0/1460184871 wait complete. 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55ada64740 20 client.15351 trim_cache size 0 max 16384 2024-09-21T02:04:13.287 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24619, 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 93.255314333000001, 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 125.224960853 2024-09-21T02:04:13.288 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:13.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f558c0b65a0 msgr2=0x7f558c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55a74f0640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f558c0b65a0 0x7f558c0b8ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55a0102e50 tx=0x7f559000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 msgr2=0x7f55a019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55a74f0640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a019fc70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5598005cc0 tx=0x7f559802f380 comp rx=0 tx=0).stop 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.282+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 shutdown_connections 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f558c0b65a0 0x7f558c0b8ab0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55a0109e20 0x7f55a019fc70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55a01094d0 0x7f55a019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 --2- 172.21.15.92:0/3666665426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a0103b90 0x7f55a019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 >> 172.21.15.92:0/3666665426 conn(0x7f55a00ff600 msgr2=0x7f55a0106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 shutdown_connections 2024-09-21T02:04:13.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:13.283+0000 7f55a74f0640 1 -- 172.21.15.92:0/3666665426 wait complete. 2024-09-21T02:04:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:12 smithi099 ceph-mon[34404]: pgmap v337: 97 pgs: 97 active+clean; 649 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:14.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T02:04:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:14.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.833+0000 7fefacef1640 1 -- 172.21.15.92:0/2348663961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 msgr2=0x7fefa8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.833+0000 7fefacef1640 1 --2- 172.21.15.92:0/2348663961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa8110620 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009ca0 tx=0x7fefa002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.833+0000 7fefacef1640 1 -- 172.21.15.92:0/2348663961 shutdown_connections 2024-09-21T02:04:14.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 --2- 172.21.15.92:0/2348663961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa8110620 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 --2- 172.21.15.92:0/2348663961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa81094a0 0x7fefa81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 --2- 172.21.15.92:0/2348663961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefa8103b60 0x7fefa8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 -- 172.21.15.92:0/2348663961 >> 172.21.15.92:0/2348663961 conn(0x7fefa807b850 msgr2=0x7fefa807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:14.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 -- 172.21.15.92:0/2348663961 shutdown_connections 2024-09-21T02:04:14.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 -- 172.21.15.92:0/2348663961 wait complete. 2024-09-21T02:04:14.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 Processor -- start 2024-09-21T02:04:14.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.834+0000 7fefacef1640 1 -- start start 2024-09-21T02:04:14.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefacef1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefa8103b60 0x7fefa819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefacef1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa81094a0 0x7fefa819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefacef1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefacef1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8113ab0 con 0x7fefa8109df0 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefacef1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8113930 con 0x7fefa8103b60 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefacef1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8113c30 con 0x7fefa81094a0 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37310/0 (socket says 172.21.15.92:37310) 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 -- 172.21.15.92:0/3629313304 learned_addr learned my addr 172.21.15.92:0/3629313304 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa81094a0 0x7fefa819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa659c640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefa8103b60 0x7fefa819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 -- 172.21.15.92:0/3629313304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa81094a0 msgr2=0x7fefa819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa81094a0 0x7fefa819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 -- 172.21.15.92:0/3629313304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefa8103b60 msgr2=0x7fefa819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefa8103b60 0x7fefa819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.835+0000 7fefa6d9d640 1 -- 172.21.15.92:0/3629313304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa81a02f0 con 0x7fefa8109df0 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.836+0000 7fefa6d9d640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa819fc80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fefa0005cc0 tx=0x7fefa002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.836+0000 7fef977fe640 1 -- 172.21.15.92:0/3629313304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003cf0 con 0x7fefa8109df0 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.836+0000 7fef977fe640 1 -- 172.21.15.92:0/3629313304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa00044f0 con 0x7fefa8109df0 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.836+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa81a0580 con 0x7fefa8109df0 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.836+0000 7fef977fe640 1 -- 172.21.15.92:0/3629313304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa002fc30 con 0x7fefa8109df0 2024-09-21T02:04:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.836+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a6e00 con 0x7fefa8109df0 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.838+0000 7fef977fe640 1 -- 172.21.15.92:0/3629313304 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fefa00046a0 con 0x7fefa8109df0 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 Processor -- start 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 -- start start 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x56370218df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x56370218e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 0x56370218e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563702105c10 con 0x563701dc7280 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5637020f5040 con 0x563701de7f60 2024-09-21T02:04:14.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefb3465740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5637020f8e50 con 0x563701dbf0a0 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 0x56370218e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 0x56370218e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37312/0 (socket says 172.21.15.92:37312) 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefa5d9b640 1 -- 172.21.15.92:0/1955133153 learned_addr learned my addr 172.21.15.92:0/1955133153 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x56370218df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fef977fe640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fef900b65c0 0x7fef900b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.839+0000 7fefa659c640 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x56370218e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa6d9d640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fef900b65c0 0x7fef900b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa5d9b640 1 -- 172.21.15.92:0/1955133153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 msgr2=0x56370218e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x56370218e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa5d9b640 1 -- 172.21.15.92:0/1955133153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 msgr2=0x56370218df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x56370218df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa5d9b640 1 -- 172.21.15.92:0/1955133153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563701e53540 con 0x563701dc7280 2024-09-21T02:04:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fef977fe640 1 -- 172.21.15.92:0/3629313304 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fefa0108680 con 0x7fefa8109df0 2024-09-21T02:04:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 0x56370218e980 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0116b0 tx=0x7fef9c00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab462aac6a4d322 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fefa6d9d640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fef900b65c0 0x7fef900b8ad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa00ce4f0 tx=0x7fefa00ff290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fef94ff9640 1 -- 172.21.15.92:0/1955133153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c004020 con 0x563701dc7280 2024-09-21T02:04:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fef94ff9640 1 -- 172.21.15.92:0/1955133153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c0054d0 con 0x563701dc7280 2024-09-21T02:04:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.840+0000 7fef94ff9640 1 -- 172.21.15.92:0/1955133153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c004d00 con 0x563701dc7280 2024-09-21T02:04:14.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 -- 172.21.15.92:0/1955133153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 msgr2=0x56370218e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 0x56370218e980 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0116b0 tx=0x7fef9c00a640 comp rx=0 tx=0).stop 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 -- 172.21.15.92:0/1955133153 shutdown_connections 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dc7280 0x56370218e980 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x56370218e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 --2- 172.21.15.92:0/1955133153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x56370218df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 -- 172.21.15.92:0/1955133153 >> 172.21.15.92:0/1955133153 conn(0x563701ddc350 msgr2=0x56370218bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 -- 172.21.15.92:0/1955133153 shutdown_connections 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.841+0000 7fefb3465740 1 -- 172.21.15.92:0/1955133153 wait complete. 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 -- start start 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 Processor -- start 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dbf0a0 0x563702199a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x563702199f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563701f565f0 con 0x563701de7f60 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563702090010 con 0x563701dbf0a0 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56370219c6e0 con 0x563701dc7280 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefb3465740 10 client.0 _ll_register_callbacks cb 0x5637020ef340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37320/0 (socket says 172.21.15.92:37320) 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 -- 172.21.15.92:0/2940080338 learned_addr learned my addr 172.21.15.92:0/2940080338 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa659c640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x563702199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dbf0a0 0x563702199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 -- 172.21.15.92:0/2940080338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 msgr2=0x563702199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x563702199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 -- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dbf0a0 msgr2=0x563702199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dbf0a0 0x563702199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.842+0000 7fefa5d9b640 1 -- 172.21.15.92:0/2940080338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637020bda00 con 0x563701de7f60 2024-09-21T02:04:14.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a59d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x56370219c060 tx=0x7fef9c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f193e176817bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fef8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fef8dffb640 1 -- 172.21.15.92:0/2940080338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00f800 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fef8dffb640 1 -- 172.21.15.92:0/2940080338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fef9c00f9a0 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef9c0054d0 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fef8dffb640 1 -- 172.21.15.92:0/2940080338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef9c00fba0 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 10 client.15363 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5637021a8640 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fef8dffb640 1 -- 172.21.15.92:0/2940080338 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c0054d0 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fef8dffb640 1 -- 172.21.15.92:0/2940080338 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fef9c01ea40 con 0x563701de7f60 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 20 client.15363 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:14.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.843+0000 7fefb3465740 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701da68e0 0x5637021abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.844+0000 7fefb3465740 4 client.15363 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.844+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563702107b60 con 0x563701da68e0 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.844+0000 7fefa659c640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701da68e0 0x5637021abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.844+0000 7fef8dffb640 10 client.15363 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.844+0000 7fefa659c640 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701da68e0 0x5637021abe10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2705d71044be05f3 server_cookie=d5fc4c1675e0edfd in_seq=0 out_seq=0 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fef8dffb640 1 -- 172.21.15.92:0/2940080338 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563702107b60 con 0x563701da68e0 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fef8dffb640 10 client.15363 handle_command_reply: tid=0 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 client.15363 shutdown 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 msgr2=0x5637021a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a59d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x56370219c060 tx=0x7fef9c004840 comp rx=0 tx=0).stop 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 shutdown_connections 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701da68e0 0x5637021abe10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a59d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x563702199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 --2- 172.21.15.92:0/2940080338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dbf0a0 0x563702199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 >> 172.21.15.92:0/2940080338 conn(0x563701ddc350 msgr2=0x5637021910e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:14.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fef8dffb640 0 client.15363 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:04:14.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 shutdown_connections 2024-09-21T02:04:14.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 1 -- 172.21.15.92:0/2940080338 wait complete. 2024-09-21T02:04:14.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.845+0000 7fefb3465740 20 client.15363 trim_cache size 0 max 16384 2024-09-21T02:04:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.863+0000 7fefb3465740 1 Processor -- start 2024-09-21T02:04:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.863+0000 7fefb3465740 1 -- start start 2024-09-21T02:04:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefb3465740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x5637021ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefb3465740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dc7280 0x563702186ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefb3465740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefb3465740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637021aa150 con 0x563701de7f60 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefb3465740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5637021aa820 con 0x563701dc7280 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefb3465740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5637021aaef0 con 0x563701dbf0a0 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37334/0 (socket says 172.21.15.92:37334) 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 -- 172.21.15.92:0/3355575380 learned_addr learned my addr 172.21.15.92:0/3355575380 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa659c640 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dc7280 0x563702186ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x5637021ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 -- 172.21.15.92:0/3355575380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 msgr2=0x5637021ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x5637021ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 -- 172.21.15.92:0/3355575380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dc7280 msgr2=0x563702186ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dc7280 0x563702186ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 -- 172.21.15.92:0/3355575380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637021950a0 con 0x563701de7f60 2024-09-21T02:04:14.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a5940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00aa10 tx=0x7fef9c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff43b8b0f119ce23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:14.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.864+0000 7fef8ffff640 1 -- 172.21.15.92:0/3355575380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00aac0 con 0x563701de7f60 2024-09-21T02:04:14.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fef8ffff640 1 -- 172.21.15.92:0/3355575380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c028040 con 0x563701de7f60 2024-09-21T02:04:14.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fef8ffff640 1 -- 172.21.15.92:0/3355575380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c011020 con 0x563701de7f60 2024-09-21T02:04:14.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 -- 172.21.15.92:0/3355575380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 msgr2=0x5637021a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a5940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00aa10 tx=0x7fef9c016040 comp rx=0 tx=0).stop 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 -- 172.21.15.92:0/3355575380 shutdown_connections 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701de7f60 0x5637021a5940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701dc7280 0x563702186ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 --2- 172.21.15.92:0/3355575380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dbf0a0 0x5637021ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.865+0000 7fefb3465740 1 -- 172.21.15.92:0/3355575380 >> 172.21.15.92:0/3355575380 conn(0x563701ddc350 msgr2=0x563702198090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 -- 172.21.15.92:0/3355575380 shutdown_connections 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 -- 172.21.15.92:0/3355575380 wait complete. 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 -- start start 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 Processor -- start 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 0x5637021b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x5637021add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x5637021ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637021abc90 con 0x563701dbf0a0 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5637021ab5c0 con 0x563701de7f60 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5637021ac360 con 0x563701dc7280 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefb3465740 10 client.0 _ll_register_callbacks cb 0x7fef9c00ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 0x5637021b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefa5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 0x5637021b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37340/0 (socket says 172.21.15.92:37340) 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.866+0000 7fefa5d9b640 1 -- 172.21.15.92:0/235747488 learned_addr learned my addr 172.21.15.92:0/235747488 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa659c640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x5637021add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa5d9b640 1 -- 172.21.15.92:0/235747488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 msgr2=0x5637021add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x5637021add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa5d9b640 1 -- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 msgr2=0x5637021ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x5637021ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa5d9b640 1 -- 172.21.15.92:0/235747488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637021aea50 con 0x563701dbf0a0 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa659c640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x5637021add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:14.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefa5d9b640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 0x5637021b48b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fef9c01ef00 tx=0x7fef9c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5f337256dae07d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fef8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fef8dffb640 1 -- 172.21.15.92:0/235747488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00fd40 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fef8dffb640 1 -- 172.21.15.92:0/235747488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fef9c011030 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef9c028040 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fef8dffb640 1 -- 172.21.15.92:0/235747488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef9c011230 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefb3465740 10 client.15369 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5637022172e0 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.867+0000 7fef8dffb640 1 -- 172.21.15.92:0/235747488 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c028040 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fef8dffb640 1 -- 172.21.15.92:0/235747488 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fef9c01e470 con 0x563701dbf0a0 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 10 client.15369 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 10 client.15369 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 20 client.15369 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701d7bc90 0x56370221a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 4 client.15369 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:14.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "status"}) -- 0x563702103610 con 0x563701d7bc90 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefa659c640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701d7bc90 0x56370221a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fef8dffb640 10 client.15369 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefa659c640 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701d7bc90 0x56370221a710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24a5dd4b78304337 server_cookie=7f57c60c39b96ffb in_seq=0 out_seq=0 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fef8dffb640 1 -- 172.21.15.92:0/235747488 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563702103610 con 0x563701d7bc90 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fef8dffb640 10 client.15369 handle_command_reply: tid=0 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.868+0000 7fefb3465740 1 client.15369 shutdown 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 msgr2=0x5637021b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 0x5637021b48b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fef9c01ef00 tx=0x7fef9c004f90 comp rx=0 tx=0).stop 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 shutdown_connections 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563701de7f60 0x5637021ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563701dc7280 0x5637021add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563701d7bc90 0x56370221a710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 --2- 172.21.15.92:0/235747488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563701dbf0a0 0x5637021b48b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fef8dffb640 0 client.15369 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 >> 172.21.15.92:0/235747488 conn(0x563701ddc350 msgr2=0x56370218a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 shutdown_connections 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 1 -- 172.21.15.92:0/235747488 wait complete. 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefb3465740 20 client.15369 trim_cache size 0 max 16384 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24623, 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:04:14.875 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 95.850415631999994, 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 113.55424304899999 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:14.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fef900b65c0 msgr2=0x7fef900b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fef900b65c0 0x7fef900b8ad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa00ce4f0 tx=0x7fefa00ff290 comp rx=0 tx=0).stop 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 msgr2=0x7fefa819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa819fc80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fefa0005cc0 tx=0x7fefa002f470 comp rx=0 tx=0).stop 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 shutdown_connections 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fef900b65c0 0x7fef900b8ad0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefa8109df0 0x7fefa819fc80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefa81094a0 0x7fefa819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 --2- 172.21.15.92:0/3629313304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefa8103b60 0x7fefa819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.869+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 >> 172.21.15.92:0/3629313304 conn(0x7fefa807b850 msgr2=0x7fefa81074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.870+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 shutdown_connections 2024-09-21T02:04:14.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:14.870+0000 7fefacef1640 1 -- 172.21.15.92:0/3629313304 wait complete. 2024-09-21T02:04:15.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:14 smithi092 ceph-mon[31190]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:04:15.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:14 smithi092 ceph-mon[31190]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T02:04:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:14 smithi136 ceph-mon[34754]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:04:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:14 smithi136 ceph-mon[34754]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T02:04:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:14 smithi099 ceph-mon[34404]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:04:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:14 smithi099 ceph-mon[34404]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-21T02:04:15.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T02:04:15.819 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 -- 172.21.15.92:0/2992200061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 msgr2=0x7f82881098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 --2- 172.21.15.92:0/2992200061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f82881098b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f827c009900 tx=0x7f827c02a410 comp rx=0 tx=0).stop 2024-09-21T02:04:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 -- 172.21.15.92:0/2992200061 shutdown_connections 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 --2- 172.21.15.92:0/2992200061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 0x7f8288110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 --2- 172.21.15.92:0/2992200061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f82881098b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 --2- 172.21.15.92:0/2992200061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 0x7f8288103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 -- 172.21.15.92:0/2992200061 >> 172.21.15.92:0/2992200061 conn(0x7f828807b850 msgr2=0x7f828807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 -- 172.21.15.92:0/2992200061 shutdown_connections 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.272+0000 7f828cfad640 1 -- 172.21.15.92:0/2992200061 wait complete. 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.273+0000 7f828cfad640 1 Processor -- start 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.273+0000 7f828cfad640 1 -- start start 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828cfad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 0x7f828819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828cfad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f828819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828cfad640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 0x7f828819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828cfad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8288113ab0 con 0x7f82881094a0 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828cfad640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8288113930 con 0x7f8288103b60 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828cfad640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8288113c30 con 0x7f8288109df0 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 0x7f828819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f828819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 0x7f828819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 0x7f828819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:56114/0 (socket says 172.21.15.92:56114) 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 0x7f828819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:37198/0 (socket says 172.21.15.92:37198) 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f828659c640 1 -- 172.21.15.92:0/1471439721 learned_addr learned my addr 172.21.15.92:0/1471439721 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f828819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37356/0 (socket says 172.21.15.92:37356) 2024-09-21T02:04:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 -- 172.21.15.92:0/1471439721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 msgr2=0x7f828819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 0x7f828819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 -- 172.21.15.92:0/1471439721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 msgr2=0x7f828819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 0x7f828819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 -- 172.21.15.92:0/1471439721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82881a0450 con 0x7f82881094a0 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.274+0000 7f8285d9b640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f828819ab40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f827c02a930 tx=0x7f827c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.275+0000 7f82777fe640 1 -- 172.21.15.92:0/1471439721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c03f070 con 0x7f82881094a0 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.275+0000 7f82777fe640 1 -- 172.21.15.92:0/1471439721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c03b460 con 0x7f82881094a0 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.275+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881a06e0 con 0x7f82881094a0 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.275+0000 7f82777fe640 1 -- 172.21.15.92:0/1471439721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c035070 con 0x7f82881094a0 2024-09-21T02:04:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.275+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881a0bc0 con 0x7f82881094a0 2024-09-21T02:04:16.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.277+0000 7f8293521740 1 Processor -- start 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 -- start start 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f82777fe640 1 -- 172.21.15.92:0/1471439721 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f827c03b610 con 0x7f82881094a0 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa912f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f240a0 0x55fa912f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa912f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa9126ac10 con 0x55fa90f4cf60 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fa9125a040 con 0x55fa90f240a0 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8293521740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa9125de50 con 0x55fa90f2c280 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa912f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa912f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37358/0 (socket says 172.21.15.92:37358) 2024-09-21T02:04:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 -- 172.21.15.92:0/471331797 learned_addr learned my addr 172.21.15.92:0/471331797 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f828659c640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f240a0 0x55fa912f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8285d9b640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa912f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 -- 172.21.15.92:0/471331797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 msgr2=0x55fa912f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa912f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 -- 172.21.15.92:0/471331797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f240a0 msgr2=0x55fa912f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f240a0 0x55fa912f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8286d9d640 1 -- 172.21.15.92:0/471331797 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa90fb8540 con 0x55fa90f4cf60 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f8285d9b640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa912f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.278+0000 7f828659c640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f240a0 0x55fa912f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:16.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f8286d9d640 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa912f2f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8280007ec0 tx=0x7f828000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d119930207a1eb8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f82777fe640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82700b6540 0x7f82700b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f8274ff9640 1 -- 172.21.15.92:0/471331797 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280010840 con 0x55fa90f4cf60 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f828659c640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82700b6540 0x7f82700b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f8274ff9640 1 -- 172.21.15.92:0/471331797 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82800183f0 con 0x55fa90f4cf60 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f8274ff9640 1 -- 172.21.15.92:0/471331797 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280017600 con 0x55fa90f4cf60 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.279+0000 7f82777fe640 1 -- 172.21.15.92:0/1471439721 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f827c107470 con 0x7f82881094a0 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 -- 172.21.15.92:0/471331797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 msgr2=0x55fa912f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f828659c640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82700b6540 0x7f82700b8a50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55fa912f1b80 tx=0x7f827800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa912f2f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8280007ec0 tx=0x7f828000f790 comp rx=0 tx=0).stop 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 -- 172.21.15.92:0/471331797 shutdown_connections 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa912f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f240a0 0x55fa912f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 --2- 172.21.15.92:0/471331797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa912f2f00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 -- 172.21.15.92:0/471331797 >> 172.21.15.92:0/471331797 conn(0x55fa90f41350 msgr2=0x55fa912f0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 -- 172.21.15.92:0/471331797 shutdown_connections 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 -- 172.21.15.92:0/471331797 wait complete. 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.280+0000 7f8293521740 1 -- start start 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 Processor -- start 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f240a0 0x55fa912fea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa912fef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa9130ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa910bb5f0 con 0x55fa90f4cf60 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fa911f5010 con 0x55fa90f2c280 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa91305af0 con 0x55fa90f240a0 2024-09-21T02:04:16.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8293521740 10 client.0 _ll_register_callbacks cb 0x55fa91254340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa9130ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa9130ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37364/0 (socket says 172.21.15.92:37364) 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 -- 172.21.15.92:0/3912153325 learned_addr learned my addr 172.21.15.92:0/3912153325 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f828659c640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f240a0 0x55fa912fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8285d9b640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa912fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 -- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f240a0 msgr2=0x55fa912fea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f240a0 0x55fa912fea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 -- 172.21.15.92:0/3912153325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 msgr2=0x55fa912fef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa912fef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f8286d9d640 1 -- 172.21.15.92:0/3912153325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa91222a00 con 0x55fa90f4cf60 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.281+0000 7f828659c640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f240a0 0x55fa912fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8286d9d640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa9130ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8280007e90 tx=0x7f8280005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7219e34c3aff56c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f826dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:16.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f826dffb640 1 -- 172.21.15.92:0/3912153325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280026020 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f826dffb640 1 -- 172.21.15.92:0/3912153325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8280003cc0 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82800183f0 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f826dffb640 1 -- 172.21.15.92:0/3912153325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8280031050 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 10 client.15381 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa913118c0 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f826dffb640 1 -- 172.21.15.92:0/3912153325 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82800183f0 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f826dffb640 1 -- 172.21.15.92:0/3912153325 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8280004160 con 0x55fa90f4cf60 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 10 client.15381 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 20 client.15381 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f0b8e0 0x55fa91314fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 4 client.15381 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.282+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9126cb60 con 0x55fa90f0b8e0 2024-09-21T02:04:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.283+0000 7f828659c640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f0b8e0 0x55fa91314fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.283+0000 7f826dffb640 10 client.15381 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.283+0000 7f828659c640 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f0b8e0 0x55fa91314fb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4edfa92ac038fd49 server_cookie=d6a04d5a11ef5a0 in_seq=0 out_seq=0 2024-09-21T02:04:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f826dffb640 1 -- 172.21.15.92:0/3912153325 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa9126cb60 con 0x55fa90f0b8e0 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f826dffb640 10 client.15381 handle_command_reply: tid=0 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 client.15381 shutdown 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 msgr2=0x55fa9130ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa9130ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8280007e90 tx=0x7f8280005a50 comp rx=0 tx=0).stop 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 shutdown_connections 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f0b8e0 0x55fa91314fb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f4cf60 0x55fa9130ede0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa912fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f826dffb640 0 client.15381 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 --2- 172.21.15.92:0/3912153325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f240a0 0x55fa912fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 >> 172.21.15.92:0/3912153325 conn(0x55fa90f41350 msgr2=0x55fa912f60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 shutdown_connections 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 1 -- 172.21.15.92:0/3912153325 wait complete. 2024-09-21T02:04:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.284+0000 7f8293521740 20 client.15381 trim_cache size 0 max 16384 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 Processor -- start 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 -- start start 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f0b8e0 0x55fa912fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa9130ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa9130f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa913132f0 con 0x55fa90f240a0 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fa913139c0 con 0x55fa90f2c280 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.302+0000 7f8293521740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa91314090 con 0x55fa90f0b8e0 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa9130ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa9130ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37380/0 (socket says 172.21.15.92:37380) 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 -- 172.21.15.92:0/695265627 learned_addr learned my addr 172.21.15.92:0/695265627 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 -- 172.21.15.92:0/695265627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f0b8e0 msgr2=0x55fa912fdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f828659c640 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f0b8e0 0x55fa912fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8286d9d640 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa9130f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f0b8e0 0x55fa912fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 -- 172.21.15.92:0/695265627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 msgr2=0x55fa9130f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa9130f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 -- 172.21.15.92:0/695265627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa912fa090 con 0x55fa90f240a0 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8286d9d640 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa9130f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f8285d9b640 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa9130ed50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f827c0cd620 tx=0x7f827c0c9df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37a267082fe0705 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f826ffff640 1 -- 172.21.15.92:0/695265627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0cce00 con 0x55fa90f240a0 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f826ffff640 1 -- 172.21.15.92:0/695265627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c0cd1a0 con 0x55fa90f240a0 2024-09-21T02:04:16.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.303+0000 7f826ffff640 1 -- 172.21.15.92:0/695265627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0d6b70 con 0x55fa90f240a0 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 -- 172.21.15.92:0/695265627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 msgr2=0x55fa9130ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa9130ed50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f827c0cd620 tx=0x7f827c0c9df0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 -- 172.21.15.92:0/695265627 shutdown_connections 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f2c280 0x55fa9130f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa9130ed50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 --2- 172.21.15.92:0/695265627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f0b8e0 0x55fa912fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 -- 172.21.15.92:0/695265627 >> 172.21.15.92:0/695265627 conn(0x55fa90f41350 msgr2=0x55fa912fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 -- 172.21.15.92:0/695265627 shutdown_connections 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 -- 172.21.15.92:0/695265627 wait complete. 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.304+0000 7f8293521740 1 -- start start 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 Processor -- start 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f0b8e0 0x55fa9131d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa91316df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa91317330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa91314e30 con 0x55fa90f240a0 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fa91314760 con 0x55fa90f0b8e0 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa91315500 con 0x55fa90f2c280 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8293521740 10 client.0 _ll_register_callbacks cb 0x7f8280007d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa91316df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa91316df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37386/0 (socket says 172.21.15.92:37386) 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 -- 172.21.15.92:0/2791769320 learned_addr learned my addr 172.21.15.92:0/2791769320 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 -- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 msgr2=0x55fa91317330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa91317330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f828659c640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f0b8e0 0x55fa9131d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 -- 172.21.15.92:0/2791769320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f0b8e0 msgr2=0x55fa9131d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f0b8e0 0x55fa9131d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f8285d9b640 1 -- 172.21.15.92:0/2791769320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa912f5790 con 0x55fa90f240a0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.305+0000 7f828659c640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f0b8e0 0x55fa9131d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8285d9b640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa91316df0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f827c107820 tx=0x7f827c0cd560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bc4b0a56d5818a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f826dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f826dffb640 1 -- 172.21.15.92:0/2791769320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0d6d70 con 0x55fa90f240a0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f826dffb640 1 -- 172.21.15.92:0/2791769320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f827c0d6040 con 0x55fa90f240a0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827c0cd1a0 con 0x55fa90f240a0 2024-09-21T02:04:16.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f826dffb640 1 -- 172.21.15.92:0/2791769320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f827c0d20c0 con 0x55fa90f240a0 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 10 client.15387 fetch_fsmap learned FSMap version 28 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa9131d0c0 con 0x55fa90f240a0 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f826dffb640 1 -- 172.21.15.92:0/2791769320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c0cd1a0 con 0x55fa90f240a0 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f826dffb640 1 -- 172.21.15.92:0/2791769320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f827c0c91e0 con 0x55fa90f240a0 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 10 client.15387 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 20 client.15387 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f4cf60 0x55fa913836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 4 client.15387 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.306+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "status"}) -- 0x55fa91268610 con 0x55fa90f4cf60 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f828659c640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f4cf60 0x55fa913836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f826dffb640 10 client.15387 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f828659c640 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f4cf60 0x55fa913836e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba8b9d895e39801 server_cookie=593aa4b3cd3d1a28 in_seq=0 out_seq=0 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f826dffb640 1 -- 172.21.15.92:0/2791769320 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fa91268610 con 0x55fa90f4cf60 2024-09-21T02:04:16.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f826dffb640 10 client.15387 handle_command_reply: tid=0 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f8293521740 1 client.15387 shutdown 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 msgr2=0x55fa91316df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f8293521740 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa91316df0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f827c107820 tx=0x7f827c0cd560 comp rx=0 tx=0).stop 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 shutdown_connections 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f8293521740 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fa90f4cf60 0x55fa913836e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.307+0000 7f8293521740 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa90f2c280 0x55fa91317330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f826dffb640 0 client.15387 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f8293521740 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa90f240a0 0x55fa91316df0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f8293521740 1 --2- 172.21.15.92:0/2791769320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa90f0b8e0 0x55fa9131d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 >> 172.21.15.92:0/2791769320 conn(0x55fa90f41350 msgr2=0x55fa912f0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 shutdown_connections 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f8293521740 1 -- 172.21.15.92:0/2791769320 wait complete. 2024-09-21T02:04:16.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f8293521740 20 client.15387 trim_cache size 0 max 16384 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "49513f9c-77bc-11ef-baf5-efdc52797490", 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24637, 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 104.33811921500001, 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 28, 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 77, 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:04:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 123.16443427999999 2024-09-21T02:04:16.315 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:16.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82700b6540 msgr2=0x7f82700b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82700b6540 0x7f82700b8a50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55fa912f1b80 tx=0x7f827800a380 comp rx=0 tx=0).stop 2024-09-21T02:04:16.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 msgr2=0x7f828819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:16.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f828819ab40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f827c02a930 tx=0x7f827c004270 comp rx=0 tx=0).stop 2024-09-21T02:04:16.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 shutdown_connections 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82700b6540 0x7f82700b8a50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8288109df0 0x7f828819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82881094a0 0x7f828819ab40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 --2- 172.21.15.92:0/1471439721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8288103b60 0x7f828819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 >> 172.21.15.92:0/1471439721 conn(0x7f828807b850 msgr2=0x7f8288105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 shutdown_connections 2024-09-21T02:04:16.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:16.308+0000 7f828cfad640 1 -- 172.21.15.92:0/1471439721 wait complete. 2024-09-21T02:04:16.963 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:16 smithi092 ceph-mon[31190]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:16.964 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T02:04:16.993 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T02:04:16.993 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T02:04:16.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:16.994 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-21T02:04:17.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:17.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-09-21T02:04:17.077 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2024-09-21T02:04:17.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:04:17.079 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-21T02:04:17.079 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:04:17.079 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:17.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T02:04:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:16 smithi136 ceph-mon[34754]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:16 smithi099 ceph-mon[34404]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 -- 172.21.15.92:0/2962408325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 msgr2=0x7fd784102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/2962408325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd784102e80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd774009ca0 tx=0x7fd77402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 -- 172.21.15.92:0/2962408325 shutdown_connections 2024-09-21T02:04:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/2962408325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784108d00 0x7fd78410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/2962408325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7841083b0 0x7fd7841087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/2962408325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd784102e80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 -- 172.21.15.92:0/2962408325 >> 172.21.15.92:0/2962408325 conn(0x7fd784079710 msgr2=0x7fd784079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 -- 172.21.15.92:0/2962408325 shutdown_connections 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.793+0000 7fd78c4ab640 1 -- 172.21.15.92:0/2962408325 wait complete. 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 Processor -- start 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 -- start start 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd7841993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7841083b0 0x7fd784199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd784108d00 0x7fd78419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd7841129c0 con 0x7fd784102a70 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd784112840 con 0x7fd7841083b0 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78c4ab640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd784112b40 con 0x7fd784108d00 2024-09-21T02:04:17.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78a247640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd7841993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78a247640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd7841993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37400/0 (socket says 172.21.15.92:37400) 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78a247640 1 -- 172.21.15.92:0/657987274 learned_addr learned my addr 172.21.15.92:0/657987274 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd789a46640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7841083b0 0x7fd784199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.794+0000 7fd78aa48640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd784108d00 0x7fd78419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78a247640 1 -- 172.21.15.92:0/657987274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd784108d00 msgr2=0x7fd78419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78a247640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd784108d00 0x7fd78419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78a247640 1 -- 172.21.15.92:0/657987274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7841083b0 msgr2=0x7fd784199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78a247640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7841083b0 0x7fd784199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78a247640 1 -- 172.21.15.92:0/657987274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78419f280 con 0x7fd784102a70 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78aa48640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd784108d00 0x7fd78419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78a247640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd7841993d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd77402ade0 tx=0x7fd774003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774003d60 con 0x7fd784102a70 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78419f510 con 0x7fd784102a70 2024-09-21T02:04:17.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7740044f0 con 0x7fd784102a70 2024-09-21T02:04:17.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77402fd10 con 0x7fd784102a70 2024-09-21T02:04:17.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.795+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78419fa50 con 0x7fd784102a70 2024-09-21T02:04:17.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.796+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd784103cf0 con 0x7fd784102a70 2024-09-21T02:04:17.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.797+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fd7740046a0 con 0x7fd784102a70 2024-09-21T02:04:17.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.798+0000 7fd7837fe640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd76c0b6540 0x7fd76c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.798+0000 7fd789a46640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd76c0b6540 0x7fd76c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.798+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd7741088a0 con 0x7fd784102a70 2024-09-21T02:04:17.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.799+0000 7fd789a46640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd76c0b6540 0x7fd76c0b8a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd778005fd0 tx=0x7fd778005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:17.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:17.803+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7740cf110 con 0x7fd784102a70 2024-09-21T02:04:18.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.018+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd78402e640 con 0x7fd784102a70 2024-09-21T02:04:18.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.019+0000 7fd7837fe640 1 -- 172.21.15.92:0/657987274 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7fd7740d34f0 con 0x7fd784102a70 2024-09-21T02:04:18.020 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd76c0b6540 msgr2=0x7fd76c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd76c0b6540 0x7fd76c0b8a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd778005fd0 tx=0x7fd778005e20 comp rx=0 tx=0).stop 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 msgr2=0x7fd7841993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd7841993d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd77402ade0 tx=0x7fd774003970 comp rx=0 tx=0).stop 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 shutdown_connections 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd76c0b6540 0x7fd76c0b8a50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd784108d00 0x7fd78419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7841083b0 0x7fd784199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 --2- 172.21.15.92:0/657987274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd784102a70 0x7fd7841993d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 >> 172.21.15.92:0/657987274 conn(0x7fd784079710 msgr2=0x7fd7841064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 shutdown_connections 2024-09-21T02:04:18.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18.021+0000 7fd78c4ab640 1 -- 172.21.15.92:0/657987274 wait complete. 2024-09-21T02:04:18.697 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:04:18.698 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:04:18.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:18.698 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:04:18.715 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42139sec preferred_lft 42139sec 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-21T02:04:18.716 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:04:18.717 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:04:18.717 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-21T02:04:18.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:18.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:18.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:18.924 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:04:18.949 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-21T02:04:18.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:18.950 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-09-21T02:04:19.014 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-09-21T02:04:19.015 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:04:19.015 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:04:19.015 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:04:19.015 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:04:19.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:19.015 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-21T02:04:19.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:04:19.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:04:19.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:04:19.016 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-21T02:04:19.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:19.113 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:18 smithi092 ceph-mon[31190]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:04:19.113 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:18 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/657987274' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T02:04:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:18 smithi136 ceph-mon[34754]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:04:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:18 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/657987274' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T02:04:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:19.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:19.310 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-21T02:04:19.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:19.327 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-09-21T02:04:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:18 smithi099 ceph-mon[34404]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:04:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:18 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/657987274' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T02:04:19.382 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:19.382 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-21T02:04:19.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:04:19.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T02:04:19.382 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-21T02:04:19.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:19.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:19.502 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T02:04:19.502 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-21T02:04:19.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:19.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:19.625 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-21T02:04:19.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-09-21T02:04:19.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:04:19.625 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-21T02:04:19.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:19.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:19.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:04:19.706 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:19.721 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:04:19.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:04:19.722 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:19.775 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T02:04:19.775 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T02:04:19.775 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/80f7c1c1-cbdd-4a71-a4cb-e351ed1898ee 2024-09-21T02:04:19.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:19.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/80f7c1c1-cbdd-4a71-a4cb-e351ed1898ee /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T02:04:19.884 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T02:04:19.885 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-21T02:04:20.033 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T02:04:20.033 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-21T02:04:20.033 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.99:3300,172.21.15.136:3300" 2024-09-21T02:04:20.033 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@49513f9c-77bc-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/80f7c1c1-cbdd-4a71-a4cb-e351ed1898ee 2024-09-21T02:04:20.033 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.92:3300/172.21.15.99:3300/172.21.15.136:3300" will pass to kernel 2024-09-21T02:04:20.034 INFO:teuthology.orchestra.run.smithi092.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T02:04:20.034 INFO:teuthology.orchestra.run.smithi092.stdout: You just mounted a file system that supports labels which does not 2024-09-21T02:04:20.034 INFO:teuthology.orchestra.run.smithi092.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T02:04:20.034 INFO:teuthology.orchestra.run.smithi092.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T02:04:20.034 INFO:teuthology.orchestra.run.smithi092.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T02:04:20.036 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T02:04:20.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:20.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:20.065 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:04:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:04:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:20.162 INFO:teuthology.orchestra.run.smithi092.stdout:client25012 2024-09-21T02:04:20.162 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T02:04:20.162 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:04:20.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:04:20.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:04:20.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:20.276 INFO:teuthology.orchestra.run.smithi092.stdout:client25012 2024-09-21T02:04:20.276 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:04:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:04:20.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:20.332 INFO:teuthology.orchestra.run.smithi092.stdout:client25012 2024-09-21T02:04:20.332 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/49513f9c-77bc-11ef-baf5-efdc52797490.client25012/status') 2024-09-21T02:04:20.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:20.390 INFO:teuthology.orchestra.run.smithi092.stdout:instance: client.25012 (3)172.21.15.92:0/549090066 2024-09-21T02:04:20.390 INFO:teuthology.orchestra.run.smithi092.stdout:blocklisted: false 2024-09-21T02:04:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2024-09-21T02:04:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2024-09-21T02:04:20.390 INFO:teuthology.orchestra.run.smithi092.stderr:70 bytes copied, 3.9476e-05 s, 1.8 MB/s 2024-09-21T02:04:20.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:20.397 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T02:04:20.407 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T02:04:20.407 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:04:20.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T02:04:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 -- 172.21.15.92:0/2493740825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 msgr2=0x7fce7c10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 --2- 172.21.15.92:0/2493740825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c10b8e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fce74009c90 tx=0x7fce7402a860 comp rx=0 tx=0).stop 2024-09-21T02:04:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 -- 172.21.15.92:0/2493740825 shutdown_connections 2024-09-21T02:04:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 --2- 172.21.15.92:0/2493740825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c10b8e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 --2- 172.21.15.92:0/2493740825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce7c1068c0 0x7fce7c108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 --2- 172.21.15.92:0/2493740825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c06b670 0x7fce7c106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 -- 172.21.15.92:0/2493740825 >> 172.21.15.92:0/2493740825 conn(0x7fce7c0fdee0 msgr2=0x7fce7c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 -- 172.21.15.92:0/2493740825 shutdown_connections 2024-09-21T02:04:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.517+0000 7fce8454a640 1 -- 172.21.15.92:0/2493740825 wait complete. 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 Processor -- start 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 -- start start 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce7c06b670 0x7fce7c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c1068c0 0x7fce7c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c10f2c0 con 0x7fce7c1093d0 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c10f140 con 0x7fce7c1068c0 2024-09-21T02:04:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.518+0000 7fce8454a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c10f440 con 0x7fce7c06b670 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37410/0 (socket says 172.21.15.92:37410) 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 -- 172.21.15.92:0/2220116550 learned_addr learned my addr 172.21.15.92:0/2220116550 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce81ae5640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c1068c0 0x7fce7c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce822e6640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce7c06b670 0x7fce7c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 -- 172.21.15.92:0/2220116550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce7c06b670 msgr2=0x7fce7c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce7c06b670 0x7fce7c199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 -- 172.21.15.92:0/2220116550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c1068c0 msgr2=0x7fce7c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c1068c0 0x7fce7c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 -- 172.21.15.92:0/2220116550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c19b950 con 0x7fce7c1093d0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce82ae7640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c19b130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fce740011e0 tx=0x7fce74004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74030560 con 0x7fce7c1093d0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c19bbe0 con 0x7fce7c1093d0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce74030bb0 con 0x7fce7c1093d0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7403cc90 con 0x7fce7c1093d0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.519+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c1a2330 con 0x7fce7c1093d0 2024-09-21T02:04:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.520+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7c106cd0 con 0x7fce7c1093d0 2024-09-21T02:04:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.522+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fce7403c410 con 0x7fce7c1093d0 2024-09-21T02:04:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.522+0000 7fce737fe640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce680b6510 0x7fce680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.522+0000 7fce822e6640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce680b6510 0x7fce680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.522+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fce74108780 con 0x7fce7c1093d0 2024-09-21T02:04:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.523+0000 7fce822e6640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce680b6510 0x7fce680b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce78005fd0 tx=0x7fce78005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.525+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce740ceff0 con 0x7fce7c1093d0 2024-09-21T02:04:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.740+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fce7c0ffa00 con 0x7fce7c1093d0 2024-09-21T02:04:20.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.857+0000 7fce737fe640 1 -- 172.21.15.92:0/2220116550 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fce740d33d0 con 0x7fce7c1093d0 2024-09-21T02:04:20.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce680b6510 msgr2=0x7fce680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce680b6510 0x7fce680b8a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce78005fd0 tx=0x7fce78005c70 comp rx=0 tx=0).stop 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 msgr2=0x7fce7c19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c19b130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fce740011e0 tx=0x7fce74004cc0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 shutdown_connections 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce680b6510 0x7fce680b8a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce7c1093d0 0x7fce7c19b130 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c1068c0 0x7fce7c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 --2- 172.21.15.92:0/2220116550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce7c06b670 0x7fce7c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 >> 172.21.15.92:0/2220116550 conn(0x7fce7c0fdee0 msgr2=0x7fce7c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 shutdown_connections 2024-09-21T02:04:20.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.860+0000 7fce8454a640 1 -- 172.21.15.92:0/2220116550 wait complete. 2024-09-21T02:04:20.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.985+0000 7f849a5c3640 1 -- 172.21.15.92:0/1057250118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84941025c0 msgr2=0x7f8494102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.985+0000 7f849a5c3640 1 --2- 172.21.15.92:0/1057250118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84941025c0 0x7f8494102a30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8484009900 tx=0x7f848402a410 comp rx=0 tx=0).stop 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.985+0000 7f849a5c3640 1 -- 172.21.15.92:0/1057250118 shutdown_connections 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.985+0000 7f849a5c3640 1 --2- 172.21.15.92:0/1057250118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8494103fe0 0x7f849410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 --2- 172.21.15.92:0/1057250118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8494103640 0x7f8494103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 --2- 172.21.15.92:0/1057250118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84941025c0 0x7f8494102a30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 -- 172.21.15.92:0/1057250118 >> 172.21.15.92:0/1057250118 conn(0x7f84940fdf00 msgr2=0x7f84941003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 -- 172.21.15.92:0/1057250118 shutdown_connections 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 -- 172.21.15.92:0/1057250118 wait complete. 2024-09-21T02:04:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 Processor -- start 2024-09-21T02:04:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.986+0000 7f849a5c3640 1 -- start start 2024-09-21T02:04:20.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941025c0 0x7f849406c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8494103640 0x7f849419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 0x7f84941a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f849410e2b0 con 0x7f8494103fe0 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f849410e130 con 0x7f84941025c0 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f849410e430 con 0x7f8494103640 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 0x7f84941a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 0x7f84941a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37428/0 (socket says 172.21.15.92:37428) 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 -- 172.21.15.92:0/3212793769 learned_addr learned my addr 172.21.15.92:0/3212793769 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8493fff640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941025c0 0x7f849406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f84937fe640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8494103640 0x7f849419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 -- 172.21.15.92:0/3212793769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8494103640 msgr2=0x7f849419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8494103640 0x7f849419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 -- 172.21.15.92:0/3212793769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941025c0 msgr2=0x7f849406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941025c0 0x7f849406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 -- 172.21.15.92:0/3212793769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84941a1af0 con 0x7f8494103fe0 2024-09-21T02:04:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f8498b60640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 0x7f84941a12d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f848c00cad0 tx=0x7f848c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c00ae20 con 0x7f8494103fe0 2024-09-21T02:04:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84941a1de0 con 0x7f8494103fe0 2024-09-21T02:04:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c005020 con 0x7f8494103fe0 2024-09-21T02:04:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c00f6b0 con 0x7f8494103fe0 2024-09-21T02:04:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.987+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941a22c0 con 0x7f8494103fe0 2024-09-21T02:04:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.989+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8494109790 con 0x7f8494103fe0 2024-09-21T02:04:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.990+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f848c0051d0 con 0x7f8494103fe0 2024-09-21T02:04:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.991+0000 7f84917fa640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f847c0b6510 0x7f847c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.991+0000 7f8493fff640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f847c0b6510 0x7f847c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.991+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f848c0e7780 con 0x7f8494103fe0 2024-09-21T02:04:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.992+0000 7f8493fff640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f847c0b6510 0x7f847c0b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8484006010 tx=0x7f84840314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:20.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:20.994+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f848c014030 con 0x7f8494103fe0 2024-09-21T02:04:21.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.199+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f84940638a0 con 0x7f847c0b6510 2024-09-21T02:04:21.207 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:20 smithi092 ceph-mon[31190]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:21.207 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:20 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2220116550' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T02:04:21.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.221+0000 7f84917fa640 1 -- 172.21.15.92:0/3212793769 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f84940638a0 con 0x7f847c0b6510 2024-09-21T02:04:21.222 INFO:teuthology.orchestra.run.smithi092.stdout:{} 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f847c0b6510 msgr2=0x7f847c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f847c0b6510 0x7f847c0b8a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8484006010 tx=0x7f84840314a0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 msgr2=0x7f84941a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 0x7f84941a12d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f848c00cad0 tx=0x7f848c00cf90 comp rx=0 tx=0).stop 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 shutdown_connections 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f847c0b6510 0x7f847c0b8a20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8494103fe0 0x7f84941a12d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8494103640 0x7f849419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 --2- 172.21.15.92:0/3212793769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941025c0 0x7f849406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 >> 172.21.15.92:0/3212793769 conn(0x7f84940fdf00 msgr2=0x7f8494106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 shutdown_connections 2024-09-21T02:04:21.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.223+0000 7f849a5c3640 1 -- 172.21.15.92:0/3212793769 wait complete. 2024-09-21T02:04:21.236 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T02:04:21.247 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T02:04:21.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:04:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:20 smithi136 ceph-mon[34754]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:20 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2220116550' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T02:04:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:20 smithi099 ceph-mon[34404]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:20 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2220116550' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 -- 172.21.15.92:0/2844896611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c106bc0 msgr2=0x7f957c108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 --2- 172.21.15.92:0/2844896611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c106bc0 0x7f957c108fe0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9570009900 tx=0x7f957002a410 comp rx=0 tx=0).stop 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 -- 172.21.15.92:0/2844896611 shutdown_connections 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 --2- 172.21.15.92:0/2844896611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f957c109520 0x7f957c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 --2- 172.21.15.92:0/2844896611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c106bc0 0x7f957c108fe0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 --2- 172.21.15.92:0/2844896611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f957c101a70 0x7f957c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 -- 172.21.15.92:0/2844896611 >> 172.21.15.92:0/2844896611 conn(0x7f957c0fd630 msgr2=0x7f957c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 -- 172.21.15.92:0/2844896611 shutdown_connections 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 -- 172.21.15.92:0/2844896611 wait complete. 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 Processor -- start 2024-09-21T02:04:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.365+0000 7f95834c5640 1 -- start start 2024-09-21T02:04:21.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f957c101a70 0x7f957c104200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f957c106bc0 0x7f957c102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 0x7f957c103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10f2c0 con 0x7f957c109520 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10f140 con 0x7f957c106bc0 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10f440 con 0x7f957c101a70 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 0x7f957c103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 0x7f957c103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37468/0 (socket says 172.21.15.92:37468) 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 -- 172.21.15.92:0/2360937395 learned_addr learned my addr 172.21.15.92:0/2360937395 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9580a60640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f957c106bc0 0x7f957c102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581261640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f957c101a70 0x7f957c104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 -- 172.21.15.92:0/2360937395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f957c101a70 msgr2=0x7f957c104200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f957c101a70 0x7f957c104200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 -- 172.21.15.92:0/2360937395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f957c106bc0 msgr2=0x7f957c102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f957c106bc0 0x7f957c102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 -- 172.21.15.92:0/2360937395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c1036a0 con 0x7f957c109520 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f9581a62640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 0x7f957c103000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f957400c970 tx=0x7f957400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574007bb0 con 0x7f957c109520 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9574007d50 con 0x7f957c109520 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c1a3fc0 con 0x7f957c109520 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574018770 con 0x7f957c109520 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.366+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a4370 con 0x7f957c109520 2024-09-21T02:04:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.367+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c0ff150 con 0x7f957c109520 2024-09-21T02:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.369+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f957400f990 con 0x7f957c109520 2024-09-21T02:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.369+0000 7f957a7fc640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95640b6510 0x7f95640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.370+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f95740e7420 con 0x7f957c109520 2024-09-21T02:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.370+0000 7f9581261640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95640b6510 0x7f95640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.370+0000 7f9581261640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95640b6510 0x7f95640b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f956c005fd0 tx=0x7f956c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.373+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9574014030 con 0x7f957c109520 2024-09-21T02:04:21.554 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:04:21.204+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:04:21.554 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:04:21.204+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:04:21.554 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:04:21.204+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:04:21.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:04:21.204+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:04:21.555 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:04:21.204+0000 7fc18ccfc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:04:21.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.554+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f957c02e810 con 0x7f957c109520 2024-09-21T02:04:21.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.556+0000 7f957a7fc640 1 -- 172.21.15.92:0/2360937395 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22797 (secure 0 0 0) 0x7f95740add20 con 0x7f957c109520 2024-09-21T02:04:21.558 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:21.558 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:03:12.414418+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95640b6510 msgr2=0x7f95640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95640b6510 0x7f95640b8a20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f956c005fd0 tx=0x7f956c009870 comp rx=0 tx=0).stop 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 msgr2=0x7f957c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 0x7f957c103000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f957400c970 tx=0x7f957400ce30 comp rx=0 tx=0).stop 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 shutdown_connections 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95640b6510 0x7f95640b8a20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f957c109520 0x7f957c103000 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f957c106bc0 0x7f957c102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 --2- 172.21.15.92:0/2360937395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f957c101a70 0x7f957c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 >> 172.21.15.92:0/2360937395 conn(0x7f957c0fd630 msgr2=0x7f957c107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 shutdown_connections 2024-09-21T02:04:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.558+0000 7f95834c5640 1 -- 172.21.15.92:0/2360937395 wait complete. 2024-09-21T02:04:21.571 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:57:59.243065+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:02:27.085163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:02:29.579582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:04:21.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 -- 172.21.15.92:0/2116413653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c1025a0 msgr2=0x7fa40c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 --2- 172.21.15.92:0/2116413653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c1025a0 0x7fa40c102a10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa408009ca0 tx=0x7fa40802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 -- 172.21.15.92:0/2116413653 shutdown_connections 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 --2- 172.21.15.92:0/2116413653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 0x7fa40c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 --2- 172.21.15.92:0/2116413653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40c103620 0x7fa40c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 --2- 172.21.15.92:0/2116413653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c1025a0 0x7fa40c102a10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 -- 172.21.15.92:0/2116413653 >> 172.21.15.92:0/2116413653 conn(0x7fa40c0fdee0 msgr2=0x7fa40c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 -- 172.21.15.92:0/2116413653 shutdown_connections 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.683+0000 7fa412c30640 1 -- 172.21.15.92:0/2116413653 wait complete. 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 Processor -- start 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 -- start start 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40c1025a0 0x7fa40c06bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 0x7fa40c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 0x7fa40c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c10e1f0 con 0x7fa40c103620 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c10e070 con 0x7fa40c1025a0 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa412c30640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c10e370 con 0x7fa40c103fc0 2024-09-21T02:04:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa403fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 0x7fa40c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa403fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 0x7fa40c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37480/0 (socket says 172.21.15.92:37480) 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.684+0000 7fa403fff640 1 -- 172.21.15.92:0/1324952377 learned_addr learned my addr 172.21.15.92:0/1324952377 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa4111cd640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 0x7fa40c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa403fff640 1 -- 172.21.15.92:0/1324952377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 msgr2=0x7fa40c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa403fff640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 0x7fa40c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa4109cc640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40c1025a0 0x7fa40c06bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa403fff640 1 -- 172.21.15.92:0/1324952377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40c1025a0 msgr2=0x7fa40c06bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa403fff640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40c1025a0 0x7fa40c06bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa403fff640 1 -- 172.21.15.92:0/1324952377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a1a00 con 0x7fa40c103620 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa4111cd640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 0x7fa40c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa403fff640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 0x7fa40c19ec50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9f0 tx=0x7fa3fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00cd80 con 0x7fa40c103620 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3fc0108e0 con 0x7fa40c103620 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a1cf0 con 0x7fa40c103620 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc018980 con 0x7fa40c103620 2024-09-21T02:04:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.685+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1a21e0 con 0x7fa40c103620 2024-09-21T02:04:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.686+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c109770 con 0x7fa40c103620 2024-09-21T02:04:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.688+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7fa3fc010430 con 0x7fa40c103620 2024-09-21T02:04:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.689+0000 7fa401ffb640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3f40b6510 0x7fa3f40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.689+0000 7fa4109cc640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3f40b6510 0x7fa3f40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.689+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa3fc0e7c20 con 0x7fa40c103620 2024-09-21T02:04:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.690+0000 7fa4109cc640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3f40b6510 0x7fa3f40b8a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4080094e0 tx=0x7fa408005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.692+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3fc0ae490 con 0x7fa40c103620 2024-09-21T02:04:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.870+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa40c063ff0 con 0x7fa40c103620 2024-09-21T02:04:21.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.871+0000 7fa401ffb640 1 -- 172.21.15.92:0/1324952377 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fa3fc0b2870 con 0x7fa40c103620 2024-09-21T02:04:21.873 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-09-21T02:04:21.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.873+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3f40b6510 msgr2=0x7fa3f40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3f40b6510 0x7fa3f40b8a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4080094e0 tx=0x7fa408005fd0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 msgr2=0x7fa40c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 0x7fa40c19ec50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9f0 tx=0x7fa3fc00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 shutdown_connections 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa3f40b6510 0x7fa3f40b8a20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa40c103fc0 0x7fa40c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c103620 0x7fa40c19ec50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 --2- 172.21.15.92:0/1324952377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40c1025a0 0x7fa40c06bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 >> 172.21.15.92:0/1324952377 conn(0x7fa40c0fdee0 msgr2=0x7fa40c106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 shutdown_connections 2024-09-21T02:04:21.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.874+0000 7fa412c30640 1 -- 172.21.15.92:0/1324952377 wait complete. 2024-09-21T02:04:21.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 -- 172.21.15.92:0/1746759824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8109450 msgr2=0x7f09b810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 --2- 172.21.15.92:0/1746759824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8109450 0x7f09b810b8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009ca0 tx=0x7f09b002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 -- 172.21.15.92:0/1746759824 shutdown_connections 2024-09-21T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 --2- 172.21.15.92:0/1746759824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8109450 0x7f09b810b8f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 --2- 172.21.15.92:0/1746759824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09b8106af0 0x7f09b8108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 --2- 172.21.15.92:0/1746759824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b8101a60 0x7f09b8101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 -- 172.21.15.92:0/1746759824 >> 172.21.15.92:0/1746759824 conn(0x7f09b80fd660 msgr2=0x7f09b80ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 -- 172.21.15.92:0/1746759824 shutdown_connections 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.998+0000 7f09bed69640 1 -- 172.21.15.92:0/1746759824 wait complete. 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 Processor -- start 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 -- start start 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 0x7f09b8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09b8106af0 0x7f09b819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b8109450 0x7f09b819b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09b810f340 con 0x7f09b8101a60 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09b810f1c0 con 0x7f09b8106af0 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bed69640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f09b810f4c0 con 0x7f09b8109450 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bcb05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 0x7f09b8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bcb05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 0x7f09b8199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37500/0 (socket says 172.21.15.92:37500) 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09bcb05640 1 -- 172.21.15.92:0/1788495091 learned_addr learned my addr 172.21.15.92:0/1788495091 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:21.999+0000 7f09b7fff640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09b8106af0 0x7f09b819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bcb05640 1 -- 172.21.15.92:0/1788495091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b8109450 msgr2=0x7f09b819b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bcb05640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b8109450 0x7f09b819b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bcb05640 1 -- 172.21.15.92:0/1788495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09b8106af0 msgr2=0x7f09b819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bcb05640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09b8106af0 0x7f09b819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bcb05640 1 -- 172.21.15.92:0/1788495091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b819b940 con 0x7f09b8101a60 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bcb05640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 0x7f09b8199f00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00b4a0 tx=0x7f09ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac014070 con 0x7f09b8101a60 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b819bc30 con 0x7f09b8101a60 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ac010ac0 con 0x7f09b8101a60 2024-09-21T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac017c30 con 0x7f09b8101a60 2024-09-21T02:04:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.000+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81a2320 con 0x7f09b8101a60 2024-09-21T02:04:22.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.001+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b80ff180 con 0x7f09b8101a60 2024-09-21T02:04:22.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.003+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f09ac017070 con 0x7f09b8101a60 2024-09-21T02:04:22.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.003+0000 7f09b5ffb640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09a80b6510 0x7f09a80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.004+0000 7f09b7fff640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09a80b6510 0x7f09a80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.004+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f09ac0e6850 con 0x7f09b8101a60 2024-09-21T02:04:22.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.004+0000 7f09b7fff640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09a80b6510 0x7f09a80b8a20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09a400bbe0 tx=0x7f09a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:22.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.007+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09ac0ad0c0 con 0x7f09b8101a60 2024-09-21T02:04:22.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.186+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09b8063fc0 con 0x7f09b8101a60 2024-09-21T02:04:22.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.186+0000 7f09b5ffb640 1 -- 172.21.15.92:0/1788495091 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f09ac0b14a0 con 0x7f09b8101a60 2024-09-21T02:04:22.188 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09a80b6510 msgr2=0x7f09a80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09a80b6510 0x7f09a80b8a20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09a400bbe0 tx=0x7f09a400b390 comp rx=0 tx=0).stop 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 msgr2=0x7f09b8199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 0x7f09b8199f00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00b4a0 tx=0x7f09ac00b960 comp rx=0 tx=0).stop 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 shutdown_connections 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f09a80b6510 0x7f09a80b8a20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b8109450 0x7f09b819b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09b8106af0 0x7f09b819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 --2- 172.21.15.92:0/1788495091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8101a60 0x7f09b8199f00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 >> 172.21.15.92:0/1788495091 conn(0x7f09b80fd660 msgr2=0x7f09b81078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 shutdown_connections 2024-09-21T02:04:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.189+0000 7f09bed69640 1 -- 172.21.15.92:0/1788495091 wait complete. 2024-09-21T02:04:22.201 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2220116550' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T02:04:22.201 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-mon[31190]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:22.201 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:21 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2360937395' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:22.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:04:22.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:21 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2220116550' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T02:04:22.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:21 smithi136 ceph-mon[34754]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:22.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:21 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2360937395' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 -- 172.21.15.92:0/4041890157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c102c40 msgr2=0x7ff91c105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 --2- 172.21.15.92:0/4041890157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c102c40 0x7ff91c105060 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009900 tx=0x7ff90c02a410 comp rx=0 tx=0).stop 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 -- 172.21.15.92:0/4041890157 shutdown_connections 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 --2- 172.21.15.92:0/4041890157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff91c1082c0 0x7ff91c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 --2- 172.21.15.92:0/4041890157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c1056a0 0x7ff91c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 --2- 172.21.15.92:0/4041890157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c102c40 0x7ff91c105060 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 -- 172.21.15.92:0/4041890157 >> 172.21.15.92:0/4041890157 conn(0x7ff91c0fd690 msgr2=0x7ff91c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 -- 172.21.15.92:0/4041890157 shutdown_connections 2024-09-21T02:04:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.312+0000 7ff921aa2640 1 -- 172.21.15.92:0/4041890157 wait complete. 2024-09-21T02:04:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 Processor -- start 2024-09-21T02:04:22.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 -- start start 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c102c40 0x7ff91c1020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff91c1056a0 0x7ff91c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 0x7ff91c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10e1d0 con 0x7ff91c1082c0 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10e050 con 0x7ff91c102c40 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff921aa2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10e350 con 0x7ff91c1056a0 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 0x7ff91c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 0x7ff91c1a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37522/0 (socket says 172.21.15.92:37522) 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 -- 172.21.15.92:0/1583253753 learned_addr learned my addr 172.21.15.92:0/1583253753 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91b7fe640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c102c40 0x7ff91c1020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff913fff640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff91c1056a0 0x7ff91c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 -- 172.21.15.92:0/1583253753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff91c1056a0 msgr2=0x7ff91c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff91c1056a0 0x7ff91c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 -- 172.21.15.92:0/1583253753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c102c40 msgr2=0x7ff91c1020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c102c40 0x7ff91c1020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 -- 172.21.15.92:0/1583253753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c1a19b0 con 0x7ff91c1082c0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91b7fe640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c102c40 0x7ff91c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.313+0000 7ff91bfff640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 0x7ff91c1a11b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff91400eaa0 tx=0x7ff91400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.314+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91400cdb0 con 0x7ff91c1082c0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.314+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff914005020 con 0x7ff91c1082c0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.314+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140116d0 con 0x7ff91c1082c0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.314+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c1a1ca0 con 0x7ff91c1082c0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.314+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1a2190 con 0x7ff91c1082c0 2024-09-21T02:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.315+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91c0ff1b0 con 0x7ff91c1082c0 2024-09-21T02:04:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.316+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7ff9140051d0 con 0x7ff91c1082c0 2024-09-21T02:04:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.317+0000 7ff9197fa640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff9080b6590 0x7ff9080b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.317+0000 7ff91b7fe640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff9080b6590 0x7ff9080b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.317+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff9140e7450 con 0x7ff91c1082c0 2024-09-21T02:04:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.317+0000 7ff91b7fe640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff9080b6590 0x7ff9080b8aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff90c031440 tx=0x7ff90c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.321+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff914016030 con 0x7ff91c1082c0 2024-09-21T02:04:22.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:21 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2220116550' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T02:04:22.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:21 smithi099 ceph-mon[34404]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:04:22.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:21 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2360937395' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:22.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.500+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff91c063ff0 con 0x7ff91c1082c0 2024-09-21T02:04:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.500+0000 7ff9197fa640 1 -- 172.21.15.92:0/1583253753 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7ff9140adcc0 con 0x7ff91c1082c0 2024-09-21T02:04:22.502 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-09-21T02:04:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.502+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff9080b6590 msgr2=0x7ff9080b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff9080b6590 0x7ff9080b8aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff90c031440 tx=0x7ff90c005d50 comp rx=0 tx=0).stop 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 msgr2=0x7ff91c1a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 0x7ff91c1a11b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff91400eaa0 tx=0x7ff91400ef60 comp rx=0 tx=0).stop 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 shutdown_connections 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff9080b6590 0x7ff9080b8aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91c1082c0 0x7ff91c1a11b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff91c1056a0 0x7ff91c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 --2- 172.21.15.92:0/1583253753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff91c102c40 0x7ff91c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 >> 172.21.15.92:0/1583253753 conn(0x7ff91c0fd690 msgr2=0x7ff91c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 shutdown_connections 2024-09-21T02:04:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:22.503+0000 7ff921aa2640 1 -- 172.21.15.92:0/1583253753 wait complete. 2024-09-21T02:04:22.516 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T02:04:22.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:04:22.761 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:23.157 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:22 smithi092 ceph-mon[31190]: pgmap v342: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T02:04:23.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:22 smithi092 ceph-mon[31190]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:23.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:22 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1324952377' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:22 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1788495091' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:22 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1583253753' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:22 smithi136 ceph-mon[34754]: pgmap v342: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T02:04:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:22 smithi136 ceph-mon[34754]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:22 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1324952377' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:22 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1788495091' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:22 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1583253753' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 -- 172.21.15.92:0/2916038307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 msgr2=0x7f1a081098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/2916038307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a081098b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009900 tx=0x7f19fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 -- 172.21.15.92:0/2916038307 shutdown_connections 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/2916038307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a08109df0 0x7f1a08110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/2916038307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a081098b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/2916038307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a08103b60 0x7f1a08103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 -- 172.21.15.92:0/2916038307 >> 172.21.15.92:0/2916038307 conn(0x7f1a0807b850 msgr2=0x7f1a0807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 -- 172.21.15.92:0/2916038307 shutdown_connections 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.248+0000 7f1a0ca94640 1 -- 172.21.15.92:0/2916038307 wait complete. 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 Processor -- start 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 -- start start 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a08103b60 0x7f1a0819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a0819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a08109df0 0x7f1a0819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08113ad0 con 0x7f1a081094a0 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08113950 con 0x7f1a08103b60 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0ca94640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08113c50 con 0x7f1a08109df0 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a05d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a0819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a05d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a0819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46490/0 (socket says 172.21.15.92:46490) 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a05d9b640 1 -- 172.21.15.92:0/3111599955 learned_addr learned my addr 172.21.15.92:0/3111599955 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a0659c640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a08103b60 0x7f1a0819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.249+0000 7f1a06d9d640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a08109df0 0x7f1a0819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a05d9b640 1 -- 172.21.15.92:0/3111599955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a08109df0 msgr2=0x7f1a0819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a05d9b640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a08109df0 0x7f1a0819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a05d9b640 1 -- 172.21.15.92:0/3111599955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a08103b60 msgr2=0x7f1a0819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a05d9b640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a08103b60 0x7f1a0819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a05d9b640 1 -- 172.21.15.92:0/3111599955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a081a0370 con 0x7f1a081094a0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a05d9b640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a0819aa60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f19fc02a930 tx=0x7f19fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc03f030 con 0x7f1a081094a0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19fc003cb0 con 0x7f1a081094a0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a081a0600 con 0x7f1a081094a0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc035070 con 0x7f1a081094a0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.250+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a081a6e80 con 0x7f1a081094a0 2024-09-21T02:04:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.251+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a08104de0 con 0x7f1a081094a0 2024-09-21T02:04:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.254+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 154512+0+0 (secure 0 0 0) 0x7f19fc042030 con 0x7f1a081094a0 2024-09-21T02:04:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.254+0000 7f19f77fe640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19f00b65c0 0x7f19f00b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.254+0000 7f1a0659c640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19f00b65c0 0x7f19f00b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.254+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f19fc107d50 con 0x7f1a081094a0 2024-09-21T02:04:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.255+0000 7f1a0659c640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19f00b65c0 0x7f19f00b8ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005e10 tx=0x7f19f8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.258+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19fc0ce5c0 con 0x7f1a081094a0 2024-09-21T02:04:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:22 smithi099 ceph-mon[34404]: pgmap v342: 97 pgs: 97 active+clean; 650 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T02:04:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:22 smithi099 ceph-mon[34404]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:04:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:22 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1324952377' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:22 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1788495091' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:22 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1583253753' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:04:23.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.474+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a08103f70 con 0x7f1a081094a0 2024-09-21T02:04:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.475+0000 7f19f77fe640 1 -- 172.21.15.92:0/3111599955 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f1a08103f70 con 0x7f1a081094a0 2024-09-21T02:04:23.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:23.477 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:04:23.478 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:04:23.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19f00b65c0 msgr2=0x7f19f00b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19f00b65c0 0x7f19f00b8ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005e10 tx=0x7f19f8005da0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 msgr2=0x7f1a0819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a0819aa60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f19fc02a930 tx=0x7f19fc004270 comp rx=0 tx=0).stop 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 shutdown_connections 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19f00b65c0 0x7f19f00b8ad0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a08109df0 0x7f1a0819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a081094a0 0x7f1a0819aa60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 --2- 172.21.15.92:0/3111599955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a08103b60 0x7f1a0819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 >> 172.21.15.92:0/3111599955 conn(0x7f1a0807b850 msgr2=0x7f1a08107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 shutdown_connections 2024-09-21T02:04:23.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:23.478+0000 7f1a0ca94640 1 -- 172.21.15.92:0/3111599955 wait complete. 2024-09-21T02:04:24.162 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:23 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3111599955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:24.189 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T02:04:24.189 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T02:04:24.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:24.232 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:23 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3111599955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:23 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3111599955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:24.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 -- 172.21.15.92:0/4030051216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 msgr2=0x7fe154108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 --2- 172.21.15.92:0/4030051216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe154108cc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe144009900 tx=0x7fe14402a410 comp rx=0 tx=0).stop 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 -- 172.21.15.92:0/4030051216 shutdown_connections 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 --2- 172.21.15.92:0/4030051216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1541093b0 0x7fe15410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 --2- 172.21.15.92:0/4030051216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe154108cc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 --2- 172.21.15.92:0/4030051216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe15406c3c0 0x7fe154106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 -- 172.21.15.92:0/4030051216 >> 172.21.15.92:0/4030051216 conn(0x7fe1540fdee0 msgr2=0x7fe154100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 -- 172.21.15.92:0/4030051216 shutdown_connections 2024-09-21T02:04:24.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.299+0000 7fe15bf32640 1 -- 172.21.15.92:0/4030051216 wait complete. 2024-09-21T02:04:24.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 Processor -- start 2024-09-21T02:04:24.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 -- start start 2024-09-21T02:04:24.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe15406c3c0 0x7fe154199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe15419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1541093b0 0x7fe15419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe15410f2c0 con 0x7fe1541068a0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe15410f140 con 0x7fe1541093b0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15bf32640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe15410f440 con 0x7fe15406c3c0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe15419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe15419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46504/0 (socket says 172.21.15.92:46504) 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 -- 172.21.15.92:0/2714317017 learned_addr learned my addr 172.21.15.92:0/2714317017 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe15a4cf640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1541093b0 0x7fe15419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe159cce640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe15406c3c0 0x7fe154199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 -- 172.21.15.92:0/2714317017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe15406c3c0 msgr2=0x7fe154199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe15406c3c0 0x7fe154199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 -- 172.21.15.92:0/2714317017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1541093b0 msgr2=0x7fe15419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1541093b0 0x7fe15419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 -- 172.21.15.92:0/2714317017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15419b8c0 con 0x7fe1541068a0 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe159cce640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe15406c3c0 0x7fe154199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:24.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe1594cd640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe15419a580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe144006010 tx=0x7fe144004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:24.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.300+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14403f030 con 0x7fe1541068a0 2024-09-21T02:04:24.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.301+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe144003cb0 con 0x7fe1541068a0 2024-09-21T02:04:24.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.301+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15419bb50 con 0x7fe1541068a0 2024-09-21T02:04:24.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.301+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144035070 con 0x7fe1541068a0 2024-09-21T02:04:24.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.301+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a2150 con 0x7fe1541068a0 2024-09-21T02:04:24.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.302+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1540ffa00 con 0x7fe1541068a0 2024-09-21T02:04:24.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.303+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fe144042030 con 0x7fe1541068a0 2024-09-21T02:04:24.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.304+0000 7fe14affd640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1400b65a0 0x7fe1400b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.304+0000 7fe159cce640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1400b65a0 0x7fe1400b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.304+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe144108310 con 0x7fe1541068a0 2024-09-21T02:04:24.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.304+0000 7fe159cce640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1400b65a0 0x7fe1400b8ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe15000baa0 tx=0x7fe150005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:24.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.307+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1440cea20 con 0x7fe1541068a0 2024-09-21T02:04:24.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.487+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe154063800 con 0x7fe1400b65a0 2024-09-21T02:04:24.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.492+0000 7fe14affd640 1 -- 172.21.15.92:0/2714317017 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258882 (secure 0 0 0) 0x7fe154063800 con 0x7fe1400b65a0 2024-09-21T02:04:24.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:24.497 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:04:24.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.495+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1400b65a0 msgr2=0x7fe1400b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.495+0000 7fe15bf32640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1400b65a0 0x7fe1400b8ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe15000baa0 tx=0x7fe150005d90 comp rx=0 tx=0).stop 2024-09-21T02:04:24.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.495+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 msgr2=0x7fe15419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.495+0000 7fe15bf32640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe15419a580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe144006010 tx=0x7fe144004340 comp rx=0 tx=0).stop 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 shutdown_connections 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1400b65a0 0x7fe1400b8ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1541093b0 0x7fe15419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1541068a0 0x7fe15419a580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 --2- 172.21.15.92:0/2714317017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe15406c3c0 0x7fe154199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 >> 172.21.15.92:0/2714317017 conn(0x7fe1540fdee0 msgr2=0x7fe154104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 shutdown_connections 2024-09-21T02:04:24.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.496+0000 7fe15bf32640 1 -- 172.21.15.92:0/2714317017 wait complete. 2024-09-21T02:04:24.509 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-21T02:04:23.051281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666240,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1110,"num_read_kb":664,"num_write":394,"num_write_kb":2313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2951908,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407188,"kb_used_data":12180,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534636,"statfs":{"total":1151940427776,"available":1151523467264,"internally_reserved":0,"allocated":12472320,"data_stored":6860278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1204,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":61,"num_write":20,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266526+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:04:11.266526+0000","last_peered":"2024-09-21T02:04:11.266526+0000","last_clean":"2024-09-21T02:04:11.266526+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:04:11.266526+0000","last_undegraded":"2024-09-21T02:04:11.266526+0000","last_fullsized":"2024-09-21T02:04:11.266526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261131+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:04:11.261131+0000","last_peered":"2024-09-21T02:04:11.261131+0000","last_clean":"2024-09-21T02:04:11.261131+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:04:11.261131+0000","last_undegraded":"2024-09-21T02:04:11.261131+0000","last_fullsized":"2024-09-21T02:04:11.261131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271538+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:04:11.271538+0000","last_peered":"2024-09-21T02:04:11.271538+0000","last_clean":"2024-09-21T02:04:11.271538+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:04:11.271538+0000","last_undegraded":"2024-09-21T02:04:11.271538+0000","last_fullsized":"2024-09-21T02:04:11.271538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261297+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:04:11.261297+0000","last_peered":"2024-09-21T02:04:11.261297+0000","last_clean":"2024-09-21T02:04:11.261297+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:04:11.261297+0000","last_undegraded":"2024-09-21T02:04:11.261297+0000","last_fullsized":"2024-09-21T02:04:11.261297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.257187+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:04:11.257187+0000","last_peered":"2024-09-21T02:04:11.257187+0000","last_clean":"2024-09-21T02:04:11.257187+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:04:11.257187+0000","last_undegraded":"2024-09-21T02:04:11.257187+0000","last_fullsized":"2024-09-21T02:04:11.257187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262567+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:04:11.262567+0000","last_peered":"2024-09-21T02:04:11.262567+0000","last_clean":"2024-09-21T02:04:11.262567+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:04:11.262567+0000","last_undegraded":"2024-09-21T02:04:11.262567+0000","last_fullsized":"2024-09-21T02:04:11.262567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271583+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:04:11.271583+0000","last_peered":"2024-09-21T02:04:11.271583+0000","last_clean":"2024-09-21T02:04:11.271583+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:04:11.271583+0000","last_undegraded":"2024-09-21T02:04:11.271583+0000","last_fullsized":"2024-09-21T02:04:11.271583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269269+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:04:11.269269+0000","last_peered":"2024-09-21T02:04:11.269269+0000","last_clean":"2024-09-21T02:04:11.269269+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:04:11.269269+0000","last_undegraded":"2024-09-21T02:04:11.269269+0000","last_fullsized":"2024-09-21T02:04:11.269269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269202+0000","last_change":"2024-09-21T02:03:12.428541+0000","last_active":"2024-09-21T02:04:11.269202+0000","last_peered":"2024-09-21T02:04:11.269202+0000","last_clean":"2024-09-21T02:04:11.269202+0000","last_became_active":"2024-09-21T02:03:12.428177+0000","last_became_peered":"2024-09-21T02:03:12.428177+0000","last_unstale":"2024-09-21T02:04:11.269202+0000","last_undegraded":"2024-09-21T02:04:11.269202+0000","last_fullsized":"2024-09-21T02:04:11.269202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:42.955294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263252+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:04:11.263252+0000","last_peered":"2024-09-21T02:04:11.263252+0000","last_clean":"2024-09-21T02:04:11.263252+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:04:11.263252+0000","last_undegraded":"2024-09-21T02:04:11.263252+0000","last_fullsized":"2024-09-21T02:04:11.263252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264394+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:04:11.264394+0000","last_peered":"2024-09-21T02:04:11.264394+0000","last_clean":"2024-09-21T02:04:11.264394+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:04:11.264394+0000","last_undegraded":"2024-09-21T02:04:11.264394+0000","last_fullsized":"2024-09-21T02:04:11.264394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260057+0000","last_change":"2024-09-21T02:03:12.426083+0000","last_active":"2024-09-21T02:04:11.260057+0000","last_peered":"2024-09-21T02:04:11.260057+0000","last_clean":"2024-09-21T02:04:11.260057+0000","last_became_active":"2024-09-21T02:03:12.425716+0000","last_became_peered":"2024-09-21T02:03:12.425716+0000","last_unstale":"2024-09-21T02:04:11.260057+0000","last_undegraded":"2024-09-21T02:04:11.260057+0000","last_fullsized":"2024-09-21T02:04:11.260057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263538+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:04:11.263538+0000","last_peered":"2024-09-21T02:04:11.263538+0000","last_clean":"2024-09-21T02:04:11.263538+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:04:11.263538+0000","last_undegraded":"2024-09-21T02:04:11.263538+0000","last_fullsized":"2024-09-21T02:04:11.263538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260466+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:04:11.260466+0000","last_peered":"2024-09-21T02:04:11.260466+0000","last_clean":"2024-09-21T02:04:11.260466+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:04:11.260466+0000","last_undegraded":"2024-09-21T02:04:11.260466+0000","last_fullsized":"2024-09-21T02:04:11.260466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262755+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:04:11.262755+0000","last_peered":"2024-09-21T02:04:11.262755+0000","last_clean":"2024-09-21T02:04:11.262755+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:04:11.262755+0000","last_undegraded":"2024-09-21T02:04:11.262755+0000","last_fullsized":"2024-09-21T02:04:11.262755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270135+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:04:11.270135+0000","last_peered":"2024-09-21T02:04:11.270135+0000","last_clean":"2024-09-21T02:04:11.270135+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:04:11.270135+0000","last_undegraded":"2024-09-21T02:04:11.270135+0000","last_fullsized":"2024-09-21T02:04:11.270135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267528+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:04:11.267528+0000","last_peered":"2024-09-21T02:04:11.267528+0000","last_clean":"2024-09-21T02:04:11.267528+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:04:11.267528+0000","last_undegraded":"2024-09-21T02:04:11.267528+0000","last_fullsized":"2024-09-21T02:04:11.267528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272663+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:04:11.272663+0000","last_peered":"2024-09-21T02:04:11.272663+0000","last_clean":"2024-09-21T02:04:11.272663+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:04:11.272663+0000","last_undegraded":"2024-09-21T02:04:11.272663+0000","last_fullsized":"2024-09-21T02:04:11.272663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259439+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:04:11.259439+0000","last_peered":"2024-09-21T02:04:11.259439+0000","last_clean":"2024-09-21T02:04:11.259439+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:04:11.259439+0000","last_undegraded":"2024-09-21T02:04:11.259439+0000","last_fullsized":"2024-09-21T02:04:11.259439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.265880+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:04:11.265880+0000","last_peered":"2024-09-21T02:04:11.265880+0000","last_clean":"2024-09-21T02:04:11.265880+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:04:11.265880+0000","last_undegraded":"2024-09-21T02:04:11.265880+0000","last_fullsized":"2024-09-21T02:04:11.265880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260041+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:04:11.260041+0000","last_peered":"2024-09-21T02:04:11.260041+0000","last_clean":"2024-09-21T02:04:11.260041+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:04:11.260041+0000","last_undegraded":"2024-09-21T02:04:11.260041+0000","last_fullsized":"2024-09-21T02:04:11.260041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271873+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:04:11.271873+0000","last_peered":"2024-09-21T02:04:11.271873+0000","last_clean":"2024-09-21T02:04:11.271873+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:04:11.271873+0000","last_undegraded":"2024-09-21T02:04:11.271873+0000","last_fullsized":"2024-09-21T02:04:11.271873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264705+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:04:11.264705+0000","last_peered":"2024-09-21T02:04:11.264705+0000","last_clean":"2024-09-21T02:04:11.264705+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:04:11.264705+0000","last_undegraded":"2024-09-21T02:04:11.264705+0000","last_fullsized":"2024-09-21T02:04:11.264705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.247934+0000","last_change":"2024-09-21T02:03:12.426598+0000","last_active":"2024-09-21T02:04:11.247934+0000","last_peered":"2024-09-21T02:04:11.247934+0000","last_clean":"2024-09-21T02:04:11.247934+0000","last_became_active":"2024-09-21T02:03:12.425552+0000","last_became_peered":"2024-09-21T02:03:12.425552+0000","last_unstale":"2024-09-21T02:04:11.247934+0000","last_undegraded":"2024-09-21T02:04:11.247934+0000","last_fullsized":"2024-09-21T02:04:11.247934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:40.991213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.242197+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:04:11.242197+0000","last_peered":"2024-09-21T02:04:11.242197+0000","last_clean":"2024-09-21T02:04:11.242197+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:04:11.242197+0000","last_undegraded":"2024-09-21T02:04:11.242197+0000","last_fullsized":"2024-09-21T02:04:11.242197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.275028+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:04:11.275028+0000","last_peered":"2024-09-21T02:04:11.275028+0000","last_clean":"2024-09-21T02:04:11.275028+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:04:11.275028+0000","last_undegraded":"2024-09-21T02:04:11.275028+0000","last_fullsized":"2024-09-21T02:04:11.275028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.265779+0000","last_change":"2024-09-21T02:03:12.428701+0000","last_active":"2024-09-21T02:04:11.265779+0000","last_peered":"2024-09-21T02:04:11.265779+0000","last_clean":"2024-09-21T02:04:11.265779+0000","last_became_active":"2024-09-21T02:03:12.428283+0000","last_became_peered":"2024-09-21T02:03:12.428283+0000","last_unstale":"2024-09-21T02:04:11.265779+0000","last_undegraded":"2024-09-21T02:04:11.265779+0000","last_fullsized":"2024-09-21T02:04:11.265779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:43.972970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261710+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:04:11.261710+0000","last_peered":"2024-09-21T02:04:11.261710+0000","last_clean":"2024-09-21T02:04:11.261710+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:04:11.261710+0000","last_undegraded":"2024-09-21T02:04:11.261710+0000","last_fullsized":"2024-09-21T02:04:11.261710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263686+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:04:11.263686+0000","last_peered":"2024-09-21T02:04:11.263686+0000","last_clean":"2024-09-21T02:04:11.263686+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:04:11.263686+0000","last_undegraded":"2024-09-21T02:04:11.263686+0000","last_fullsized":"2024-09-21T02:04:11.263686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259738+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:04:11.259738+0000","last_peered":"2024-09-21T02:04:11.259738+0000","last_clean":"2024-09-21T02:04:11.259738+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:04:11.259738+0000","last_undegraded":"2024-09-21T02:04:11.259738+0000","last_fullsized":"2024-09-21T02:04:11.259738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271600+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:04:11.271600+0000","last_peered":"2024-09-21T02:04:11.271600+0000","last_clean":"2024-09-21T02:04:11.271600+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:04:11.271600+0000","last_undegraded":"2024-09-21T02:04:11.271600+0000","last_fullsized":"2024-09-21T02:04:11.271600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268792+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:04:11.268792+0000","last_peered":"2024-09-21T02:04:11.268792+0000","last_clean":"2024-09-21T02:04:11.268792+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:04:11.268792+0000","last_undegraded":"2024-09-21T02:04:11.268792+0000","last_fullsized":"2024-09-21T02:04:11.268792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.254147+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:04:11.254147+0000","last_peered":"2024-09-21T02:04:11.254147+0000","last_clean":"2024-09-21T02:04:11.254147+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:04:11.254147+0000","last_undegraded":"2024-09-21T02:04:11.254147+0000","last_fullsized":"2024-09-21T02:04:11.254147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'8","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.460283+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:04:21.460283+0000","last_peered":"2024-09-21T02:04:21.460283+0000","last_clean":"2024-09-21T02:04:21.460283+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:04:21.460283+0000","last_undegraded":"2024-09-21T02:04:21.460283+0000","last_fullsized":"2024-09-21T02:04:21.460283+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270227+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:04:11.270227+0000","last_peered":"2024-09-21T02:04:11.270227+0000","last_clean":"2024-09-21T02:04:11.270227+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:04:11.270227+0000","last_undegraded":"2024-09-21T02:04:11.270227+0000","last_fullsized":"2024-09-21T02:04:11.270227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424950+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:03:12.424950+0000","last_peered":"2024-09-21T02:03:12.424950+0000","last_clean":"2024-09-21T02:03:12.424950+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:03:12.424950+0000","last_undegraded":"2024-09-21T02:03:12.424950+0000","last_fullsized":"2024-09-21T02:03:12.424950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641291+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:03:12.641291+0000","last_peered":"2024-09-21T02:03:12.641291+0000","last_clean":"2024-09-21T02:03:12.641291+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:03:12.641291+0000","last_undegraded":"2024-09-21T02:03:12.641291+0000","last_fullsized":"2024-09-21T02:03:12.641291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264738+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:04:11.264738+0000","last_peered":"2024-09-21T02:04:11.264738+0000","last_clean":"2024-09-21T02:04:11.264738+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:04:11.264738+0000","last_undegraded":"2024-09-21T02:04:11.264738+0000","last_fullsized":"2024-09-21T02:04:11.264738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419546+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:03:12.419546+0000","last_peered":"2024-09-21T02:03:12.419546+0000","last_clean":"2024-09-21T02:03:12.419546+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:03:12.419546+0000","last_undegraded":"2024-09-21T02:03:12.419546+0000","last_fullsized":"2024-09-21T02:03:12.419546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259983+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:04:11.259983+0000","last_peered":"2024-09-21T02:04:11.259983+0000","last_clean":"2024-09-21T02:04:11.259983+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:04:11.259983+0000","last_undegraded":"2024-09-21T02:04:11.259983+0000","last_fullsized":"2024-09-21T02:04:11.259983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":225,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:18.454054+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:04:18.454054+0000","last_peered":"2024-09-21T02:04:18.454054+0000","last_clean":"2024-09-21T02:04:18.454054+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:04:18.454054+0000","last_undegraded":"2024-09-21T02:04:18.454054+0000","last_fullsized":"2024-09-21T02:04:18.454054+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.274258+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:04:11.274258+0000","last_peered":"2024-09-21T02:04:11.274258+0000","last_clean":"2024-09-21T02:04:11.274258+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:04:11.274258+0000","last_undegraded":"2024-09-21T02:04:11.274258+0000","last_fullsized":"2024-09-21T02:04:11.274258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421592+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:03:12.421592+0000","last_peered":"2024-09-21T02:03:12.421592+0000","last_clean":"2024-09-21T02:03:12.421592+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:03:12.421592+0000","last_undegraded":"2024-09-21T02:03:12.421592+0000","last_fullsized":"2024-09-21T02:03:12.421592+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271720+0000","last_change":"2024-09-21T02:03:12.423870+0000","last_active":"2024-09-21T02:04:11.271720+0000","last_peered":"2024-09-21T02:04:11.271720+0000","last_clean":"2024-09-21T02:04:11.271720+0000","last_became_active":"2024-09-21T02:03:12.423441+0000","last_became_peered":"2024-09-21T02:03:12.423441+0000","last_unstale":"2024-09-21T02:04:11.271720+0000","last_undegraded":"2024-09-21T02:04:11.271720+0000","last_fullsized":"2024-09-21T02:04:11.271720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:59.071213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419481+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:03:12.419481+0000","last_peered":"2024-09-21T02:03:12.419481+0000","last_clean":"2024-09-21T02:03:12.419481+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:03:12.419481+0000","last_undegraded":"2024-09-21T02:03:12.419481+0000","last_fullsized":"2024-09-21T02:03:12.419481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269103+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:04:11.269103+0000","last_peered":"2024-09-21T02:04:11.269103+0000","last_clean":"2024-09-21T02:04:11.269103+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:04:11.269103+0000","last_undegraded":"2024-09-21T02:04:11.269103+0000","last_fullsized":"2024-09-21T02:04:11.269103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":265,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:18.454028+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:04:18.454028+0000","last_peered":"2024-09-21T02:04:18.454028+0000","last_clean":"2024-09-21T02:04:18.454028+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:04:18.454028+0000","last_undegraded":"2024-09-21T02:04:18.454028+0000","last_fullsized":"2024-09-21T02:04:18.454028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66697,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":168,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263855+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:04:11.263855+0000","last_peered":"2024-09-21T02:04:11.263855+0000","last_clean":"2024-09-21T02:04:11.263855+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:04:11.263855+0000","last_undegraded":"2024-09-21T02:04:11.263855+0000","last_fullsized":"2024-09-21T02:04:11.263855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442392+0000","last_change":"2024-09-21T02:03:12.441759+0000","last_active":"2024-09-21T02:03:12.442392+0000","last_peered":"2024-09-21T02:03:12.442392+0000","last_clean":"2024-09-21T02:03:12.442392+0000","last_became_active":"2024-09-21T02:03:12.426827+0000","last_became_peered":"2024-09-21T02:03:12.426827+0000","last_unstale":"2024-09-21T02:03:12.442392+0000","last_undegraded":"2024-09-21T02:03:12.442392+0000","last_fullsized":"2024-09-21T02:03:12.442392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:33.917276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259856+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:04:11.259856+0000","last_peered":"2024-09-21T02:04:11.259856+0000","last_clean":"2024-09-21T02:04:11.259856+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:04:11.259856+0000","last_undegraded":"2024-09-21T02:04:11.259856+0000","last_fullsized":"2024-09-21T02:04:11.259856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418936+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:03:12.418936+0000","last_peered":"2024-09-21T02:03:12.418936+0000","last_clean":"2024-09-21T02:03:12.418936+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:03:12.418936+0000","last_undegraded":"2024-09-21T02:03:12.418936+0000","last_fullsized":"2024-09-21T02:03:12.418936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263535+0000","last_change":"2024-09-21T02:03:12.643022+0000","last_active":"2024-09-21T02:04:11.263535+0000","last_peered":"2024-09-21T02:04:11.263535+0000","last_clean":"2024-09-21T02:04:11.263535+0000","last_became_active":"2024-09-21T02:03:12.642501+0000","last_became_peered":"2024-09-21T02:03:12.642501+0000","last_unstale":"2024-09-21T02:04:11.263535+0000","last_undegraded":"2024-09-21T02:04:11.263535+0000","last_fullsized":"2024-09-21T02:04:11.263535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:29.600445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'7","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:18.490771+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:04:18.490771+0000","last_peered":"2024-09-21T02:04:18.490771+0000","last_clean":"2024-09-21T02:04:18.490771+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:04:18.490771+0000","last_undegraded":"2024-09-21T02:04:18.490771+0000","last_fullsized":"2024-09-21T02:04:18.490771+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271795+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:04:11.271795+0000","last_peered":"2024-09-21T02:04:11.271795+0000","last_clean":"2024-09-21T02:04:11.271795+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:04:11.271795+0000","last_undegraded":"2024-09-21T02:04:11.271795+0000","last_fullsized":"2024-09-21T02:04:11.271795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423964+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:03:12.423964+0000","last_peered":"2024-09-21T02:03:12.423964+0000","last_clean":"2024-09-21T02:03:12.423964+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:03:12.423964+0000","last_undegraded":"2024-09-21T02:03:12.423964+0000","last_fullsized":"2024-09-21T02:03:12.423964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270043+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:04:11.270043+0000","last_peered":"2024-09-21T02:04:11.270043+0000","last_clean":"2024-09-21T02:04:11.270043+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:04:11.270043+0000","last_undegraded":"2024-09-21T02:04:11.270043+0000","last_fullsized":"2024-09-21T02:04:11.270043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425082+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:03:12.425082+0000","last_peered":"2024-09-21T02:03:12.425082+0000","last_clean":"2024-09-21T02:03:12.425082+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:03:12.425082+0000","last_undegraded":"2024-09-21T02:03:12.425082+0000","last_fullsized":"2024-09-21T02:03:12.425082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268289+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:04:11.268289+0000","last_peered":"2024-09-21T02:04:11.268289+0000","last_clean":"2024-09-21T02:04:11.268289+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:04:11.268289+0000","last_undegraded":"2024-09-21T02:04:11.268289+0000","last_fullsized":"2024-09-21T02:04:11.268289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641780+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:03:12.641780+0000","last_peered":"2024-09-21T02:03:12.641780+0000","last_clean":"2024-09-21T02:03:12.641780+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:03:12.641780+0000","last_undegraded":"2024-09-21T02:03:12.641780+0000","last_fullsized":"2024-09-21T02:03:12.641780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266485+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:04:11.266485+0000","last_peered":"2024-09-21T02:04:11.266485+0000","last_clean":"2024-09-21T02:04:11.266485+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:04:11.266485+0000","last_undegraded":"2024-09-21T02:04:11.266485+0000","last_fullsized":"2024-09-21T02:04:11.266485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419713+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:03:12.419713+0000","last_peered":"2024-09-21T02:03:12.419713+0000","last_clean":"2024-09-21T02:03:12.419713+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:03:12.419713+0000","last_undegraded":"2024-09-21T02:03:12.419713+0000","last_fullsized":"2024-09-21T02:03:12.419713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272095+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:04:11.272095+0000","last_peered":"2024-09-21T02:04:11.272095+0000","last_clean":"2024-09-21T02:04:11.272095+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:04:11.272095+0000","last_undegraded":"2024-09-21T02:04:11.272095+0000","last_fullsized":"2024-09-21T02:04:11.272095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:17.453155+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:04:17.453155+0000","last_peered":"2024-09-21T02:04:17.453155+0000","last_clean":"2024-09-21T02:04:17.453155+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:04:17.453155+0000","last_undegraded":"2024-09-21T02:04:17.453155+0000","last_fullsized":"2024-09-21T02:04:17.453155+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266415+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:04:11.266415+0000","last_peered":"2024-09-21T02:04:11.266415+0000","last_clean":"2024-09-21T02:04:11.266415+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:04:11.266415+0000","last_undegraded":"2024-09-21T02:04:11.266415+0000","last_fullsized":"2024-09-21T02:04:11.266415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422172+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:03:12.422172+0000","last_peered":"2024-09-21T02:03:12.422172+0000","last_clean":"2024-09-21T02:03:12.422172+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:03:12.422172+0000","last_undegraded":"2024-09-21T02:03:12.422172+0000","last_fullsized":"2024-09-21T02:03:12.422172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419020+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:03:12.419020+0000","last_peered":"2024-09-21T02:03:12.419020+0000","last_clean":"2024-09-21T02:03:12.419020+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:03:12.419020+0000","last_undegraded":"2024-09-21T02:03:12.419020+0000","last_fullsized":"2024-09-21T02:03:12.419020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271556+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:04:11.271556+0000","last_peered":"2024-09-21T02:04:11.271556+0000","last_clean":"2024-09-21T02:04:11.271556+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:04:11.271556+0000","last_undegraded":"2024-09-21T02:04:11.271556+0000","last_fullsized":"2024-09-21T02:04:11.271556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421843+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:03:12.421843+0000","last_peered":"2024-09-21T02:03:12.421843+0000","last_clean":"2024-09-21T02:03:12.421843+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:03:12.421843+0000","last_undegraded":"2024-09-21T02:03:12.421843+0000","last_fullsized":"2024-09-21T02:03:12.421843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.258590+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:04:11.258590+0000","last_peered":"2024-09-21T02:04:11.258590+0000","last_clean":"2024-09-21T02:04:11.258590+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:04:11.258590+0000","last_undegraded":"2024-09-21T02:04:11.258590+0000","last_fullsized":"2024-09-21T02:04:11.258590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:17.488576+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:04:17.488576+0000","last_peered":"2024-09-21T02:04:17.488576+0000","last_clean":"2024-09-21T02:04:17.488576+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:04:17.488576+0000","last_undegraded":"2024-09-21T02:04:17.488576+0000","last_fullsized":"2024-09-21T02:04:17.488576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270528+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:04:11.270528+0000","last_peered":"2024-09-21T02:04:11.270528+0000","last_clean":"2024-09-21T02:04:11.270528+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:04:11.270528+0000","last_undegraded":"2024-09-21T02:04:11.270528+0000","last_fullsized":"2024-09-21T02:04:11.270528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":129,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:16.450780+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:04:16.450780+0000","last_peered":"2024-09-21T02:04:16.450780+0000","last_clean":"2024-09-21T02:04:16.450780+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:04:16.450780+0000","last_undegraded":"2024-09-21T02:04:16.450780+0000","last_fullsized":"2024-09-21T02:04:16.450780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259912+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:04:11.259912+0000","last_peered":"2024-09-21T02:04:11.259912+0000","last_clean":"2024-09-21T02:04:11.259912+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:04:11.259912+0000","last_undegraded":"2024-09-21T02:04:11.259912+0000","last_fullsized":"2024-09-21T02:04:11.259912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417918+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:03:12.417918+0000","last_peered":"2024-09-21T02:03:12.417918+0000","last_clean":"2024-09-21T02:03:12.417918+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:03:12.417918+0000","last_undegraded":"2024-09-21T02:03:12.417918+0000","last_fullsized":"2024-09-21T02:03:12.417918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.255582+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:04:11.255582+0000","last_peered":"2024-09-21T02:04:11.255582+0000","last_clean":"2024-09-21T02:04:11.255582+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:04:11.255582+0000","last_undegraded":"2024-09-21T02:04:11.255582+0000","last_fullsized":"2024-09-21T02:04:11.255582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425092+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:03:12.425092+0000","last_peered":"2024-09-21T02:03:12.425092+0000","last_clean":"2024-09-21T02:03:12.425092+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:03:12.425092+0000","last_undegraded":"2024-09-21T02:03:12.425092+0000","last_fullsized":"2024-09-21T02:03:12.425092+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261100+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:04:11.261100+0000","last_peered":"2024-09-21T02:04:11.261100+0000","last_clean":"2024-09-21T02:04:11.261100+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:04:11.261100+0000","last_undegraded":"2024-09-21T02:04:11.261100+0000","last_fullsized":"2024-09-21T02:04:11.261100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'19","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:20.991825+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:04:20.991825+0000","last_peered":"2024-09-21T02:04:20.991825+0000","last_clean":"2024-09-21T02:04:20.991825+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:04:20.991825+0000","last_undegraded":"2024-09-21T02:04:20.991825+0000","last_fullsized":"2024-09-21T02:04:20.991825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267166+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:04:11.267166+0000","last_peered":"2024-09-21T02:04:11.267166+0000","last_clean":"2024-09-21T02:04:11.267166+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:04:11.267166+0000","last_undegraded":"2024-09-21T02:04:11.267166+0000","last_fullsized":"2024-09-21T02:04:11.267166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419259+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:03:12.419259+0000","last_peered":"2024-09-21T02:03:12.419259+0000","last_clean":"2024-09-21T02:03:12.419259+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:03:12.419259+0000","last_undegraded":"2024-09-21T02:03:12.419259+0000","last_fullsized":"2024-09-21T02:03:12.419259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262737+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:04:11.262737+0000","last_peered":"2024-09-21T02:04:11.262737+0000","last_clean":"2024-09-21T02:04:11.262737+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:04:11.262737+0000","last_undegraded":"2024-09-21T02:04:11.262737+0000","last_fullsized":"2024-09-21T02:04:11.262737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422234+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:03:12.422234+0000","last_peered":"2024-09-21T02:03:12.422234+0000","last_clean":"2024-09-21T02:03:12.422234+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:03:12.422234+0000","last_undegraded":"2024-09-21T02:03:12.422234+0000","last_fullsized":"2024-09-21T02:03:12.422234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.223134+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:04:11.223134+0000","last_peered":"2024-09-21T02:04:11.223134+0000","last_clean":"2024-09-21T02:04:11.223134+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:04:11.223134+0000","last_undegraded":"2024-09-21T02:04:11.223134+0000","last_fullsized":"2024-09-21T02:04:11.223134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424628+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:03:12.424628+0000","last_peered":"2024-09-21T02:03:12.424628+0000","last_clean":"2024-09-21T02:03:12.424628+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:03:12.424628+0000","last_undegraded":"2024-09-21T02:03:12.424628+0000","last_fullsized":"2024-09-21T02:03:12.424628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264940+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:04:11.264940+0000","last_peered":"2024-09-21T02:04:11.264940+0000","last_clean":"2024-09-21T02:04:11.264940+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:04:11.264940+0000","last_undegraded":"2024-09-21T02:04:11.264940+0000","last_fullsized":"2024-09-21T02:04:11.264940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426547+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:03:12.426547+0000","last_peered":"2024-09-21T02:03:12.426547+0000","last_clean":"2024-09-21T02:03:12.426547+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:03:12.426547+0000","last_undegraded":"2024-09-21T02:03:12.426547+0000","last_fullsized":"2024-09-21T02:03:12.426547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.255303+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:04:11.255303+0000","last_peered":"2024-09-21T02:04:11.255303+0000","last_clean":"2024-09-21T02:04:11.255303+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:04:11.255303+0000","last_undegraded":"2024-09-21T02:04:11.255303+0000","last_fullsized":"2024-09-21T02:04:11.255303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.446014+0000","last_change":"2024-09-21T02:03:12.445516+0000","last_active":"2024-09-21T02:03:12.446014+0000","last_peered":"2024-09-21T02:03:12.446014+0000","last_clean":"2024-09-21T02:03:12.446014+0000","last_became_active":"2024-09-21T02:03:12.430149+0000","last_became_peered":"2024-09-21T02:03:12.430149+0000","last_unstale":"2024-09-21T02:03:12.446014+0000","last_undegraded":"2024-09-21T02:03:12.446014+0000","last_fullsized":"2024-09-21T02:03:12.446014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:23.573018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264521+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:04:11.264521+0000","last_peered":"2024-09-21T02:04:11.264521+0000","last_clean":"2024-09-21T02:04:11.264521+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:04:11.264521+0000","last_undegraded":"2024-09-21T02:04:11.264521+0000","last_fullsized":"2024-09-21T02:04:11.264521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267355+0000","last_change":"2024-09-21T02:03:12.642935+0000","last_active":"2024-09-21T02:04:11.267355+0000","last_peered":"2024-09-21T02:04:11.267355+0000","last_clean":"2024-09-21T02:04:11.267355+0000","last_became_active":"2024-09-21T02:03:12.642520+0000","last_became_peered":"2024-09-21T02:03:12.642520+0000","last_unstale":"2024-09-21T02:04:11.267355+0000","last_undegraded":"2024-09-21T02:04:11.267355+0000","last_fullsized":"2024-09-21T02:04:11.267355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:32.147679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424736+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:03:12.424736+0000","last_peered":"2024-09-21T02:03:12.424736+0000","last_clean":"2024-09-21T02:03:12.424736+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:03:12.424736+0000","last_undegraded":"2024-09-21T02:03:12.424736+0000","last_fullsized":"2024-09-21T02:03:12.424736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272647+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:04:11.272647+0000","last_peered":"2024-09-21T02:04:11.272647+0000","last_clean":"2024-09-21T02:04:11.272647+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:04:11.272647+0000","last_undegraded":"2024-09-21T02:04:11.272647+0000","last_fullsized":"2024-09-21T02:04:11.272647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'13","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442095+0000","last_change":"2024-09-21T02:03:12.441361+0000","last_active":"2024-09-21T02:03:12.442095+0000","last_peered":"2024-09-21T02:03:12.442095+0000","last_clean":"2024-09-21T02:03:12.442095+0000","last_became_active":"2024-09-21T02:03:12.428877+0000","last_became_peered":"2024-09-21T02:03:12.428877+0000","last_unstale":"2024-09-21T02:03:12.442095+0000","last_undegraded":"2024-09-21T02:03:12.442095+0000","last_fullsized":"2024-09-21T02:03:12.442095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.464134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271360+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:04:11.271360+0000","last_peered":"2024-09-21T02:04:11.271360+0000","last_clean":"2024-09-21T02:04:11.271360+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:04:11.271360+0000","last_undegraded":"2024-09-21T02:04:11.271360+0000","last_fullsized":"2024-09-21T02:04:11.271360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424866+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:03:12.424866+0000","last_peered":"2024-09-21T02:03:12.424866+0000","last_clean":"2024-09-21T02:03:12.424866+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:03:12.424866+0000","last_undegraded":"2024-09-21T02:03:12.424866+0000","last_fullsized":"2024-09-21T02:03:12.424866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268705+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:04:11.268705+0000","last_peered":"2024-09-21T02:04:11.268705+0000","last_clean":"2024-09-21T02:04:11.268705+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:04:11.268705+0000","last_undegraded":"2024-09-21T02:04:11.268705+0000","last_fullsized":"2024-09-21T02:04:11.268705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.458260+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:04:21.458260+0000","last_peered":"2024-09-21T02:04:21.458260+0000","last_clean":"2024-09-21T02:04:21.458260+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:04:21.458260+0000","last_undegraded":"2024-09-21T02:04:21.458260+0000","last_fullsized":"2024-09-21T02:04:21.458260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75638,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1034,"num_read_kb":600,"num_write":269,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":68,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":406987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1064645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":406317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":406695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1470464,"data_stored":1000418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":406566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36792,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708360,"statfs":{"total":95995035648,"available":95957360640,"internally_reserved":0,"allocated":1482752,"data_stored":999423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":406427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":405859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":474860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":474176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":407905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:24.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 -- 172.21.15.92:0/1691480117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1082d0 msgr2=0x7f503c10a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 --2- 172.21.15.92:0/1691480117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1082d0 0x7f503c10a7e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5034009ca0 tx=0x7f503402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 -- 172.21.15.92:0/1691480117 shutdown_connections 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 --2- 172.21.15.92:0/1691480117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1082d0 0x7f503c10a7e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 --2- 172.21.15.92:0/1691480117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f503c1056b0 0x7f503c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 --2- 172.21.15.92:0/1691480117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503c102bf0 0x7f503c105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 -- 172.21.15.92:0/1691480117 >> 172.21.15.92:0/1691480117 conn(0x7f503c0fd660 msgr2=0x7f503c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 -- 172.21.15.92:0/1691480117 shutdown_connections 2024-09-21T02:04:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 -- 172.21.15.92:0/1691480117 wait complete. 2024-09-21T02:04:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 Processor -- start 2024-09-21T02:04:24.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.658+0000 7f5044494640 1 -- start start 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503c102bf0 0x7f503c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 0x7f503c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f503c1082d0 0x7f503c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f503c10e1f0 con 0x7f503c1056b0 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f503c10e070 con 0x7f503c102bf0 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f503c10e370 con 0x7f503c1082d0 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 0x7f503c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 0x7f503c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46516/0 (socket says 172.21.15.92:46516) 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 -- 172.21.15.92:0/2606622085 learned_addr learned my addr 172.21.15.92:0/2606622085 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5042230640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503c102bf0 0x7f503c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5042a31640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f503c1082d0 0x7f503c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 -- 172.21.15.92:0/2606622085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f503c1082d0 msgr2=0x7f503c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f503c1082d0 0x7f503c199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 -- 172.21.15.92:0/2606622085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503c102bf0 msgr2=0x7f503c198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503c102bf0 0x7f503c198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 -- 172.21.15.92:0/2606622085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c19a800 con 0x7f503c1056b0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5041a2f640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 0x7f503c199430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d770 tx=0x7f502c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c017070 con 0x7f503c1056b0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0054d0 con 0x7f503c1056b0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c19aaf0 con 0x7f503c1056b0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c004c30 con 0x7f503c1056b0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.659+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c1a11e0 con 0x7f503c1056b0 2024-09-21T02:04:24.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.660+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503c0ff180 con 0x7f503c1056b0 2024-09-21T02:04:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.662+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f502c004dd0 con 0x7f503c1056b0 2024-09-21T02:04:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.663+0000 7f50337fe640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50280b6620 0x7f50280b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.663+0000 7f5042230640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50280b6620 0x7f50280b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.663+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f502c0e7300 con 0x7f503c1056b0 2024-09-21T02:04:24.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.663+0000 7f5042230640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50280b6620 0x7f50280b8b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f503800bbe0 tx=0x7f503800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:24.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.666+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f502c0ada90 con 0x7f503c1056b0 2024-09-21T02:04:24.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.841+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f503c063800 con 0x7f50280b6620 2024-09-21T02:04:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.845+0000 7f50337fe640 1 -- 172.21.15.92:0/2606622085 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258882 (secure 0 0 0) 0x7f503c063800 con 0x7f50280b6620 2024-09-21T02:04:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:04:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50280b6620 msgr2=0x7f50280b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50280b6620 0x7f50280b8b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f503800bbe0 tx=0x7f503800b390 comp rx=0 tx=0).stop 2024-09-21T02:04:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 msgr2=0x7f503c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 0x7f503c199430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d770 tx=0x7f502c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:04:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 shutdown_connections 2024-09-21T02:04:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f50280b6620 0x7f50280b8b30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f503c1082d0 0x7f503c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f503c1056b0 0x7f503c199430 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 --2- 172.21.15.92:0/2606622085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f503c102bf0 0x7f503c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 >> 172.21.15.92:0/2606622085 conn(0x7f503c0fd660 msgr2=0x7f503c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 shutdown_connections 2024-09-21T02:04:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:24.849+0000 7f5044494640 1 -- 172.21.15.92:0/2606622085 wait complete. 2024-09-21T02:04:24.862 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-21T02:04:23.051281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666240,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1110,"num_read_kb":664,"num_write":394,"num_write_kb":2313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2951908,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407188,"kb_used_data":12180,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534636,"statfs":{"total":1151940427776,"available":1151523467264,"internally_reserved":0,"allocated":12472320,"data_stored":6860278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1204,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":61,"num_write":20,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266526+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:04:11.266526+0000","last_peered":"2024-09-21T02:04:11.266526+0000","last_clean":"2024-09-21T02:04:11.266526+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:04:11.266526+0000","last_undegraded":"2024-09-21T02:04:11.266526+0000","last_fullsized":"2024-09-21T02:04:11.266526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261131+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:04:11.261131+0000","last_peered":"2024-09-21T02:04:11.261131+0000","last_clean":"2024-09-21T02:04:11.261131+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:04:11.261131+0000","last_undegraded":"2024-09-21T02:04:11.261131+0000","last_fullsized":"2024-09-21T02:04:11.261131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271538+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:04:11.271538+0000","last_peered":"2024-09-21T02:04:11.271538+0000","last_clean":"2024-09-21T02:04:11.271538+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:04:11.271538+0000","last_undegraded":"2024-09-21T02:04:11.271538+0000","last_fullsized":"2024-09-21T02:04:11.271538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261297+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:04:11.261297+0000","last_peered":"2024-09-21T02:04:11.261297+0000","last_clean":"2024-09-21T02:04:11.261297+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:04:11.261297+0000","last_undegraded":"2024-09-21T02:04:11.261297+0000","last_fullsized":"2024-09-21T02:04:11.261297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.257187+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:04:11.257187+0000","last_peered":"2024-09-21T02:04:11.257187+0000","last_clean":"2024-09-21T02:04:11.257187+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:04:11.257187+0000","last_undegraded":"2024-09-21T02:04:11.257187+0000","last_fullsized":"2024-09-21T02:04:11.257187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262567+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:04:11.262567+0000","last_peered":"2024-09-21T02:04:11.262567+0000","last_clean":"2024-09-21T02:04:11.262567+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:04:11.262567+0000","last_undegraded":"2024-09-21T02:04:11.262567+0000","last_fullsized":"2024-09-21T02:04:11.262567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271583+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:04:11.271583+0000","last_peered":"2024-09-21T02:04:11.271583+0000","last_clean":"2024-09-21T02:04:11.271583+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:04:11.271583+0000","last_undegraded":"2024-09-21T02:04:11.271583+0000","last_fullsized":"2024-09-21T02:04:11.271583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269269+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:04:11.269269+0000","last_peered":"2024-09-21T02:04:11.269269+0000","last_clean":"2024-09-21T02:04:11.269269+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:04:11.269269+0000","last_undegraded":"2024-09-21T02:04:11.269269+0000","last_fullsized":"2024-09-21T02:04:11.269269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269202+0000","last_change":"2024-09-21T02:03:12.428541+0000","last_active":"2024-09-21T02:04:11.269202+0000","last_peered":"2024-09-21T02:04:11.269202+0000","last_clean":"2024-09-21T02:04:11.269202+0000","last_became_active":"2024-09-21T02:03:12.428177+0000","last_became_peered":"2024-09-21T02:03:12.428177+0000","last_unstale":"2024-09-21T02:04:11.269202+0000","last_undegraded":"2024-09-21T02:04:11.269202+0000","last_fullsized":"2024-09-21T02:04:11.269202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:42.955294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263252+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:04:11.263252+0000","last_peered":"2024-09-21T02:04:11.263252+0000","last_clean":"2024-09-21T02:04:11.263252+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:04:11.263252+0000","last_undegraded":"2024-09-21T02:04:11.263252+0000","last_fullsized":"2024-09-21T02:04:11.263252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264394+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:04:11.264394+0000","last_peered":"2024-09-21T02:04:11.264394+0000","last_clean":"2024-09-21T02:04:11.264394+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:04:11.264394+0000","last_undegraded":"2024-09-21T02:04:11.264394+0000","last_fullsized":"2024-09-21T02:04:11.264394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260057+0000","last_change":"2024-09-21T02:03:12.426083+0000","last_active":"2024-09-21T02:04:11.260057+0000","last_peered":"2024-09-21T02:04:11.260057+0000","last_clean":"2024-09-21T02:04:11.260057+0000","last_became_active":"2024-09-21T02:03:12.425716+0000","last_became_peered":"2024-09-21T02:03:12.425716+0000","last_unstale":"2024-09-21T02:04:11.260057+0000","last_undegraded":"2024-09-21T02:04:11.260057+0000","last_fullsized":"2024-09-21T02:04:11.260057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263538+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:04:11.263538+0000","last_peered":"2024-09-21T02:04:11.263538+0000","last_clean":"2024-09-21T02:04:11.263538+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:04:11.263538+0000","last_undegraded":"2024-09-21T02:04:11.263538+0000","last_fullsized":"2024-09-21T02:04:11.263538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260466+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:04:11.260466+0000","last_peered":"2024-09-21T02:04:11.260466+0000","last_clean":"2024-09-21T02:04:11.260466+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:04:11.260466+0000","last_undegraded":"2024-09-21T02:04:11.260466+0000","last_fullsized":"2024-09-21T02:04:11.260466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262755+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:04:11.262755+0000","last_peered":"2024-09-21T02:04:11.262755+0000","last_clean":"2024-09-21T02:04:11.262755+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:04:11.262755+0000","last_undegraded":"2024-09-21T02:04:11.262755+0000","last_fullsized":"2024-09-21T02:04:11.262755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270135+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:04:11.270135+0000","last_peered":"2024-09-21T02:04:11.270135+0000","last_clean":"2024-09-21T02:04:11.270135+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:04:11.270135+0000","last_undegraded":"2024-09-21T02:04:11.270135+0000","last_fullsized":"2024-09-21T02:04:11.270135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267528+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:04:11.267528+0000","last_peered":"2024-09-21T02:04:11.267528+0000","last_clean":"2024-09-21T02:04:11.267528+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:04:11.267528+0000","last_undegraded":"2024-09-21T02:04:11.267528+0000","last_fullsized":"2024-09-21T02:04:11.267528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272663+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:04:11.272663+0000","last_peered":"2024-09-21T02:04:11.272663+0000","last_clean":"2024-09-21T02:04:11.272663+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:04:11.272663+0000","last_undegraded":"2024-09-21T02:04:11.272663+0000","last_fullsized":"2024-09-21T02:04:11.272663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259439+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:04:11.259439+0000","last_peered":"2024-09-21T02:04:11.259439+0000","last_clean":"2024-09-21T02:04:11.259439+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:04:11.259439+0000","last_undegraded":"2024-09-21T02:04:11.259439+0000","last_fullsized":"2024-09-21T02:04:11.259439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.265880+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:04:11.265880+0000","last_peered":"2024-09-21T02:04:11.265880+0000","last_clean":"2024-09-21T02:04:11.265880+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:04:11.265880+0000","last_undegraded":"2024-09-21T02:04:11.265880+0000","last_fullsized":"2024-09-21T02:04:11.265880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260041+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:04:11.260041+0000","last_peered":"2024-09-21T02:04:11.260041+0000","last_clean":"2024-09-21T02:04:11.260041+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:04:11.260041+0000","last_undegraded":"2024-09-21T02:04:11.260041+0000","last_fullsized":"2024-09-21T02:04:11.260041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271873+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:04:11.271873+0000","last_peered":"2024-09-21T02:04:11.271873+0000","last_clean":"2024-09-21T02:04:11.271873+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:04:11.271873+0000","last_undegraded":"2024-09-21T02:04:11.271873+0000","last_fullsized":"2024-09-21T02:04:11.271873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264705+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:04:11.264705+0000","last_peered":"2024-09-21T02:04:11.264705+0000","last_clean":"2024-09-21T02:04:11.264705+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:04:11.264705+0000","last_undegraded":"2024-09-21T02:04:11.264705+0000","last_fullsized":"2024-09-21T02:04:11.264705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.247934+0000","last_change":"2024-09-21T02:03:12.426598+0000","last_active":"2024-09-21T02:04:11.247934+0000","last_peered":"2024-09-21T02:04:11.247934+0000","last_clean":"2024-09-21T02:04:11.247934+0000","last_became_active":"2024-09-21T02:03:12.425552+0000","last_became_peered":"2024-09-21T02:03:12.425552+0000","last_unstale":"2024-09-21T02:04:11.247934+0000","last_undegraded":"2024-09-21T02:04:11.247934+0000","last_fullsized":"2024-09-21T02:04:11.247934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:40.991213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.242197+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:04:11.242197+0000","last_peered":"2024-09-21T02:04:11.242197+0000","last_clean":"2024-09-21T02:04:11.242197+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:04:11.242197+0000","last_undegraded":"2024-09-21T02:04:11.242197+0000","last_fullsized":"2024-09-21T02:04:11.242197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.275028+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:04:11.275028+0000","last_peered":"2024-09-21T02:04:11.275028+0000","last_clean":"2024-09-21T02:04:11.275028+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:04:11.275028+0000","last_undegraded":"2024-09-21T02:04:11.275028+0000","last_fullsized":"2024-09-21T02:04:11.275028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.265779+0000","last_change":"2024-09-21T02:03:12.428701+0000","last_active":"2024-09-21T02:04:11.265779+0000","last_peered":"2024-09-21T02:04:11.265779+0000","last_clean":"2024-09-21T02:04:11.265779+0000","last_became_active":"2024-09-21T02:03:12.428283+0000","last_became_peered":"2024-09-21T02:03:12.428283+0000","last_unstale":"2024-09-21T02:04:11.265779+0000","last_undegraded":"2024-09-21T02:04:11.265779+0000","last_fullsized":"2024-09-21T02:04:11.265779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:43.972970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261710+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:04:11.261710+0000","last_peered":"2024-09-21T02:04:11.261710+0000","last_clean":"2024-09-21T02:04:11.261710+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:04:11.261710+0000","last_undegraded":"2024-09-21T02:04:11.261710+0000","last_fullsized":"2024-09-21T02:04:11.261710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263686+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:04:11.263686+0000","last_peered":"2024-09-21T02:04:11.263686+0000","last_clean":"2024-09-21T02:04:11.263686+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:04:11.263686+0000","last_undegraded":"2024-09-21T02:04:11.263686+0000","last_fullsized":"2024-09-21T02:04:11.263686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259738+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:04:11.259738+0000","last_peered":"2024-09-21T02:04:11.259738+0000","last_clean":"2024-09-21T02:04:11.259738+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:04:11.259738+0000","last_undegraded":"2024-09-21T02:04:11.259738+0000","last_fullsized":"2024-09-21T02:04:11.259738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271600+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:04:11.271600+0000","last_peered":"2024-09-21T02:04:11.271600+0000","last_clean":"2024-09-21T02:04:11.271600+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:04:11.271600+0000","last_undegraded":"2024-09-21T02:04:11.271600+0000","last_fullsized":"2024-09-21T02:04:11.271600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268792+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:04:11.268792+0000","last_peered":"2024-09-21T02:04:11.268792+0000","last_clean":"2024-09-21T02:04:11.268792+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:04:11.268792+0000","last_undegraded":"2024-09-21T02:04:11.268792+0000","last_fullsized":"2024-09-21T02:04:11.268792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.254147+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:04:11.254147+0000","last_peered":"2024-09-21T02:04:11.254147+0000","last_clean":"2024-09-21T02:04:11.254147+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:04:11.254147+0000","last_undegraded":"2024-09-21T02:04:11.254147+0000","last_fullsized":"2024-09-21T02:04:11.254147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'8","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.460283+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:04:21.460283+0000","last_peered":"2024-09-21T02:04:21.460283+0000","last_clean":"2024-09-21T02:04:21.460283+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:04:21.460283+0000","last_undegraded":"2024-09-21T02:04:21.460283+0000","last_fullsized":"2024-09-21T02:04:21.460283+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270227+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:04:11.270227+0000","last_peered":"2024-09-21T02:04:11.270227+0000","last_clean":"2024-09-21T02:04:11.270227+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:04:11.270227+0000","last_undegraded":"2024-09-21T02:04:11.270227+0000","last_fullsized":"2024-09-21T02:04:11.270227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424950+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:03:12.424950+0000","last_peered":"2024-09-21T02:03:12.424950+0000","last_clean":"2024-09-21T02:03:12.424950+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:03:12.424950+0000","last_undegraded":"2024-09-21T02:03:12.424950+0000","last_fullsized":"2024-09-21T02:03:12.424950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641291+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:03:12.641291+0000","last_peered":"2024-09-21T02:03:12.641291+0000","last_clean":"2024-09-21T02:03:12.641291+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:03:12.641291+0000","last_undegraded":"2024-09-21T02:03:12.641291+0000","last_fullsized":"2024-09-21T02:03:12.641291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264738+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:04:11.264738+0000","last_peered":"2024-09-21T02:04:11.264738+0000","last_clean":"2024-09-21T02:04:11.264738+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:04:11.264738+0000","last_undegraded":"2024-09-21T02:04:11.264738+0000","last_fullsized":"2024-09-21T02:04:11.264738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419546+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:03:12.419546+0000","last_peered":"2024-09-21T02:03:12.419546+0000","last_clean":"2024-09-21T02:03:12.419546+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:03:12.419546+0000","last_undegraded":"2024-09-21T02:03:12.419546+0000","last_fullsized":"2024-09-21T02:03:12.419546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259983+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:04:11.259983+0000","last_peered":"2024-09-21T02:04:11.259983+0000","last_clean":"2024-09-21T02:04:11.259983+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:04:11.259983+0000","last_undegraded":"2024-09-21T02:04:11.259983+0000","last_fullsized":"2024-09-21T02:04:11.259983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":225,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:18.454054+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:04:18.454054+0000","last_peered":"2024-09-21T02:04:18.454054+0000","last_clean":"2024-09-21T02:04:18.454054+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:04:18.454054+0000","last_undegraded":"2024-09-21T02:04:18.454054+0000","last_fullsized":"2024-09-21T02:04:18.454054+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.274258+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:04:11.274258+0000","last_peered":"2024-09-21T02:04:11.274258+0000","last_clean":"2024-09-21T02:04:11.274258+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:04:11.274258+0000","last_undegraded":"2024-09-21T02:04:11.274258+0000","last_fullsized":"2024-09-21T02:04:11.274258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421592+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:03:12.421592+0000","last_peered":"2024-09-21T02:03:12.421592+0000","last_clean":"2024-09-21T02:03:12.421592+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:03:12.421592+0000","last_undegraded":"2024-09-21T02:03:12.421592+0000","last_fullsized":"2024-09-21T02:03:12.421592+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271720+0000","last_change":"2024-09-21T02:03:12.423870+0000","last_active":"2024-09-21T02:04:11.271720+0000","last_peered":"2024-09-21T02:04:11.271720+0000","last_clean":"2024-09-21T02:04:11.271720+0000","last_became_active":"2024-09-21T02:03:12.423441+0000","last_became_peered":"2024-09-21T02:03:12.423441+0000","last_unstale":"2024-09-21T02:04:11.271720+0000","last_undegraded":"2024-09-21T02:04:11.271720+0000","last_fullsized":"2024-09-21T02:04:11.271720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:59.071213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419481+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:03:12.419481+0000","last_peered":"2024-09-21T02:03:12.419481+0000","last_clean":"2024-09-21T02:03:12.419481+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:03:12.419481+0000","last_undegraded":"2024-09-21T02:03:12.419481+0000","last_fullsized":"2024-09-21T02:03:12.419481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269103+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:04:11.269103+0000","last_peered":"2024-09-21T02:04:11.269103+0000","last_clean":"2024-09-21T02:04:11.269103+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:04:11.269103+0000","last_undegraded":"2024-09-21T02:04:11.269103+0000","last_fullsized":"2024-09-21T02:04:11.269103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":265,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:18.454028+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:04:18.454028+0000","last_peered":"2024-09-21T02:04:18.454028+0000","last_clean":"2024-09-21T02:04:18.454028+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:04:18.454028+0000","last_undegraded":"2024-09-21T02:04:18.454028+0000","last_fullsized":"2024-09-21T02:04:18.454028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66697,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":168,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263855+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:04:11.263855+0000","last_peered":"2024-09-21T02:04:11.263855+0000","last_clean":"2024-09-21T02:04:11.263855+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:04:11.263855+0000","last_undegraded":"2024-09-21T02:04:11.263855+0000","last_fullsized":"2024-09-21T02:04:11.263855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442392+0000","last_change":"2024-09-21T02:03:12.441759+0000","last_active":"2024-09-21T02:03:12.442392+0000","last_peered":"2024-09-21T02:03:12.442392+0000","last_clean":"2024-09-21T02:03:12.442392+0000","last_became_active":"2024-09-21T02:03:12.426827+0000","last_became_peered":"2024-09-21T02:03:12.426827+0000","last_unstale":"2024-09-21T02:03:12.442392+0000","last_undegraded":"2024-09-21T02:03:12.442392+0000","last_fullsized":"2024-09-21T02:03:12.442392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:33.917276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259856+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:04:11.259856+0000","last_peered":"2024-09-21T02:04:11.259856+0000","last_clean":"2024-09-21T02:04:11.259856+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:04:11.259856+0000","last_undegraded":"2024-09-21T02:04:11.259856+0000","last_fullsized":"2024-09-21T02:04:11.259856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418936+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:03:12.418936+0000","last_peered":"2024-09-21T02:03:12.418936+0000","last_clean":"2024-09-21T02:03:12.418936+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:03:12.418936+0000","last_undegraded":"2024-09-21T02:03:12.418936+0000","last_fullsized":"2024-09-21T02:03:12.418936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263535+0000","last_change":"2024-09-21T02:03:12.643022+0000","last_active":"2024-09-21T02:04:11.263535+0000","last_peered":"2024-09-21T02:04:11.263535+0000","last_clean":"2024-09-21T02:04:11.263535+0000","last_became_active":"2024-09-21T02:03:12.642501+0000","last_became_peered":"2024-09-21T02:03:12.642501+0000","last_unstale":"2024-09-21T02:04:11.263535+0000","last_undegraded":"2024-09-21T02:04:11.263535+0000","last_fullsized":"2024-09-21T02:04:11.263535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:29.600445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'7","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:18.490771+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:04:18.490771+0000","last_peered":"2024-09-21T02:04:18.490771+0000","last_clean":"2024-09-21T02:04:18.490771+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:04:18.490771+0000","last_undegraded":"2024-09-21T02:04:18.490771+0000","last_fullsized":"2024-09-21T02:04:18.490771+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271795+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:04:11.271795+0000","last_peered":"2024-09-21T02:04:11.271795+0000","last_clean":"2024-09-21T02:04:11.271795+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:04:11.271795+0000","last_undegraded":"2024-09-21T02:04:11.271795+0000","last_fullsized":"2024-09-21T02:04:11.271795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423964+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:03:12.423964+0000","last_peered":"2024-09-21T02:03:12.423964+0000","last_clean":"2024-09-21T02:03:12.423964+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:03:12.423964+0000","last_undegraded":"2024-09-21T02:03:12.423964+0000","last_fullsized":"2024-09-21T02:03:12.423964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270043+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:04:11.270043+0000","last_peered":"2024-09-21T02:04:11.270043+0000","last_clean":"2024-09-21T02:04:11.270043+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:04:11.270043+0000","last_undegraded":"2024-09-21T02:04:11.270043+0000","last_fullsized":"2024-09-21T02:04:11.270043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425082+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:03:12.425082+0000","last_peered":"2024-09-21T02:03:12.425082+0000","last_clean":"2024-09-21T02:03:12.425082+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:03:12.425082+0000","last_undegraded":"2024-09-21T02:03:12.425082+0000","last_fullsized":"2024-09-21T02:03:12.425082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268289+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:04:11.268289+0000","last_peered":"2024-09-21T02:04:11.268289+0000","last_clean":"2024-09-21T02:04:11.268289+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:04:11.268289+0000","last_undegraded":"2024-09-21T02:04:11.268289+0000","last_fullsized":"2024-09-21T02:04:11.268289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641780+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:03:12.641780+0000","last_peered":"2024-09-21T02:03:12.641780+0000","last_clean":"2024-09-21T02:03:12.641780+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:03:12.641780+0000","last_undegraded":"2024-09-21T02:03:12.641780+0000","last_fullsized":"2024-09-21T02:03:12.641780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266485+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:04:11.266485+0000","last_peered":"2024-09-21T02:04:11.266485+0000","last_clean":"2024-09-21T02:04:11.266485+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:04:11.266485+0000","last_undegraded":"2024-09-21T02:04:11.266485+0000","last_fullsized":"2024-09-21T02:04:11.266485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419713+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:03:12.419713+0000","last_peered":"2024-09-21T02:03:12.419713+0000","last_clean":"2024-09-21T02:03:12.419713+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:03:12.419713+0000","last_undegraded":"2024-09-21T02:03:12.419713+0000","last_fullsized":"2024-09-21T02:03:12.419713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272095+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:04:11.272095+0000","last_peered":"2024-09-21T02:04:11.272095+0000","last_clean":"2024-09-21T02:04:11.272095+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:04:11.272095+0000","last_undegraded":"2024-09-21T02:04:11.272095+0000","last_fullsized":"2024-09-21T02:04:11.272095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:17.453155+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:04:17.453155+0000","last_peered":"2024-09-21T02:04:17.453155+0000","last_clean":"2024-09-21T02:04:17.453155+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:04:17.453155+0000","last_undegraded":"2024-09-21T02:04:17.453155+0000","last_fullsized":"2024-09-21T02:04:17.453155+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266415+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:04:11.266415+0000","last_peered":"2024-09-21T02:04:11.266415+0000","last_clean":"2024-09-21T02:04:11.266415+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:04:11.266415+0000","last_undegraded":"2024-09-21T02:04:11.266415+0000","last_fullsized":"2024-09-21T02:04:11.266415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422172+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:03:12.422172+0000","last_peered":"2024-09-21T02:03:12.422172+0000","last_clean":"2024-09-21T02:03:12.422172+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:03:12.422172+0000","last_undegraded":"2024-09-21T02:03:12.422172+0000","last_fullsized":"2024-09-21T02:03:12.422172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419020+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:03:12.419020+0000","last_peered":"2024-09-21T02:03:12.419020+0000","last_clean":"2024-09-21T02:03:12.419020+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:03:12.419020+0000","last_undegraded":"2024-09-21T02:03:12.419020+0000","last_fullsized":"2024-09-21T02:03:12.419020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271556+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:04:11.271556+0000","last_peered":"2024-09-21T02:04:11.271556+0000","last_clean":"2024-09-21T02:04:11.271556+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:04:11.271556+0000","last_undegraded":"2024-09-21T02:04:11.271556+0000","last_fullsized":"2024-09-21T02:04:11.271556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421843+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:03:12.421843+0000","last_peered":"2024-09-21T02:03:12.421843+0000","last_clean":"2024-09-21T02:03:12.421843+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:03:12.421843+0000","last_undegraded":"2024-09-21T02:03:12.421843+0000","last_fullsized":"2024-09-21T02:03:12.421843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.258590+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:04:11.258590+0000","last_peered":"2024-09-21T02:04:11.258590+0000","last_clean":"2024-09-21T02:04:11.258590+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:04:11.258590+0000","last_undegraded":"2024-09-21T02:04:11.258590+0000","last_fullsized":"2024-09-21T02:04:11.258590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":128,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:17.488576+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:04:17.488576+0000","last_peered":"2024-09-21T02:04:17.488576+0000","last_clean":"2024-09-21T02:04:17.488576+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:04:17.488576+0000","last_undegraded":"2024-09-21T02:04:17.488576+0000","last_fullsized":"2024-09-21T02:04:17.488576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270528+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:04:11.270528+0000","last_peered":"2024-09-21T02:04:11.270528+0000","last_clean":"2024-09-21T02:04:11.270528+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:04:11.270528+0000","last_undegraded":"2024-09-21T02:04:11.270528+0000","last_fullsized":"2024-09-21T02:04:11.270528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":129,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:16.450780+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:04:16.450780+0000","last_peered":"2024-09-21T02:04:16.450780+0000","last_clean":"2024-09-21T02:04:16.450780+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:04:16.450780+0000","last_undegraded":"2024-09-21T02:04:16.450780+0000","last_fullsized":"2024-09-21T02:04:16.450780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259912+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:04:11.259912+0000","last_peered":"2024-09-21T02:04:11.259912+0000","last_clean":"2024-09-21T02:04:11.259912+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:04:11.259912+0000","last_undegraded":"2024-09-21T02:04:11.259912+0000","last_fullsized":"2024-09-21T02:04:11.259912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417918+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:03:12.417918+0000","last_peered":"2024-09-21T02:03:12.417918+0000","last_clean":"2024-09-21T02:03:12.417918+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:03:12.417918+0000","last_undegraded":"2024-09-21T02:03:12.417918+0000","last_fullsized":"2024-09-21T02:03:12.417918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.255582+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:04:11.255582+0000","last_peered":"2024-09-21T02:04:11.255582+0000","last_clean":"2024-09-21T02:04:11.255582+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:04:11.255582+0000","last_undegraded":"2024-09-21T02:04:11.255582+0000","last_fullsized":"2024-09-21T02:04:11.255582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425092+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:03:12.425092+0000","last_peered":"2024-09-21T02:03:12.425092+0000","last_clean":"2024-09-21T02:03:12.425092+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:03:12.425092+0000","last_undegraded":"2024-09-21T02:03:12.425092+0000","last_fullsized":"2024-09-21T02:03:12.425092+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261100+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:04:11.261100+0000","last_peered":"2024-09-21T02:04:11.261100+0000","last_clean":"2024-09-21T02:04:11.261100+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:04:11.261100+0000","last_undegraded":"2024-09-21T02:04:11.261100+0000","last_fullsized":"2024-09-21T02:04:11.261100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'19","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:20.991825+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:04:20.991825+0000","last_peered":"2024-09-21T02:04:20.991825+0000","last_clean":"2024-09-21T02:04:20.991825+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:04:20.991825+0000","last_undegraded":"2024-09-21T02:04:20.991825+0000","last_fullsized":"2024-09-21T02:04:20.991825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267166+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:04:11.267166+0000","last_peered":"2024-09-21T02:04:11.267166+0000","last_clean":"2024-09-21T02:04:11.267166+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:04:11.267166+0000","last_undegraded":"2024-09-21T02:04:11.267166+0000","last_fullsized":"2024-09-21T02:04:11.267166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419259+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:03:12.419259+0000","last_peered":"2024-09-21T02:03:12.419259+0000","last_clean":"2024-09-21T02:03:12.419259+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:03:12.419259+0000","last_undegraded":"2024-09-21T02:03:12.419259+0000","last_fullsized":"2024-09-21T02:03:12.419259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262737+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:04:11.262737+0000","last_peered":"2024-09-21T02:04:11.262737+0000","last_clean":"2024-09-21T02:04:11.262737+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:04:11.262737+0000","last_undegraded":"2024-09-21T02:04:11.262737+0000","last_fullsized":"2024-09-21T02:04:11.262737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422234+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:03:12.422234+0000","last_peered":"2024-09-21T02:03:12.422234+0000","last_clean":"2024-09-21T02:03:12.422234+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:03:12.422234+0000","last_undegraded":"2024-09-21T02:03:12.422234+0000","last_fullsized":"2024-09-21T02:03:12.422234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.223134+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:04:11.223134+0000","last_peered":"2024-09-21T02:04:11.223134+0000","last_clean":"2024-09-21T02:04:11.223134+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:04:11.223134+0000","last_undegraded":"2024-09-21T02:04:11.223134+0000","last_fullsized":"2024-09-21T02:04:11.223134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424628+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:03:12.424628+0000","last_peered":"2024-09-21T02:03:12.424628+0000","last_clean":"2024-09-21T02:03:12.424628+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:03:12.424628+0000","last_undegraded":"2024-09-21T02:03:12.424628+0000","last_fullsized":"2024-09-21T02:03:12.424628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264940+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:04:11.264940+0000","last_peered":"2024-09-21T02:04:11.264940+0000","last_clean":"2024-09-21T02:04:11.264940+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:04:11.264940+0000","last_undegraded":"2024-09-21T02:04:11.264940+0000","last_fullsized":"2024-09-21T02:04:11.264940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426547+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:03:12.426547+0000","last_peered":"2024-09-21T02:03:12.426547+0000","last_clean":"2024-09-21T02:03:12.426547+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:03:12.426547+0000","last_undegraded":"2024-09-21T02:03:12.426547+0000","last_fullsized":"2024-09-21T02:03:12.426547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.255303+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:04:11.255303+0000","last_peered":"2024-09-21T02:04:11.255303+0000","last_clean":"2024-09-21T02:04:11.255303+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:04:11.255303+0000","last_undegraded":"2024-09-21T02:04:11.255303+0000","last_fullsized":"2024-09-21T02:04:11.255303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.446014+0000","last_change":"2024-09-21T02:03:12.445516+0000","last_active":"2024-09-21T02:03:12.446014+0000","last_peered":"2024-09-21T02:03:12.446014+0000","last_clean":"2024-09-21T02:03:12.446014+0000","last_became_active":"2024-09-21T02:03:12.430149+0000","last_became_peered":"2024-09-21T02:03:12.430149+0000","last_unstale":"2024-09-21T02:03:12.446014+0000","last_undegraded":"2024-09-21T02:03:12.446014+0000","last_fullsized":"2024-09-21T02:03:12.446014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:23.573018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264521+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:04:11.264521+0000","last_peered":"2024-09-21T02:04:11.264521+0000","last_clean":"2024-09-21T02:04:11.264521+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:04:11.264521+0000","last_undegraded":"2024-09-21T02:04:11.264521+0000","last_fullsized":"2024-09-21T02:04:11.264521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267355+0000","last_change":"2024-09-21T02:03:12.642935+0000","last_active":"2024-09-21T02:04:11.267355+0000","last_peered":"2024-09-21T02:04:11.267355+0000","last_clean":"2024-09-21T02:04:11.267355+0000","last_became_active":"2024-09-21T02:03:12.642520+0000","last_became_peered":"2024-09-21T02:03:12.642520+0000","last_unstale":"2024-09-21T02:04:11.267355+0000","last_undegraded":"2024-09-21T02:04:11.267355+0000","last_fullsized":"2024-09-21T02:04:11.267355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:32.147679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424736+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:03:12.424736+0000","last_peered":"2024-09-21T02:03:12.424736+0000","last_clean":"2024-09-21T02:03:12.424736+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:03:12.424736+0000","last_undegraded":"2024-09-21T02:03:12.424736+0000","last_fullsized":"2024-09-21T02:03:12.424736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272647+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:04:11.272647+0000","last_peered":"2024-09-21T02:04:11.272647+0000","last_clean":"2024-09-21T02:04:11.272647+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:04:11.272647+0000","last_undegraded":"2024-09-21T02:04:11.272647+0000","last_fullsized":"2024-09-21T02:04:11.272647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'13","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442095+0000","last_change":"2024-09-21T02:03:12.441361+0000","last_active":"2024-09-21T02:03:12.442095+0000","last_peered":"2024-09-21T02:03:12.442095+0000","last_clean":"2024-09-21T02:03:12.442095+0000","last_became_active":"2024-09-21T02:03:12.428877+0000","last_became_peered":"2024-09-21T02:03:12.428877+0000","last_unstale":"2024-09-21T02:03:12.442095+0000","last_undegraded":"2024-09-21T02:03:12.442095+0000","last_fullsized":"2024-09-21T02:03:12.442095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.464134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271360+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:04:11.271360+0000","last_peered":"2024-09-21T02:04:11.271360+0000","last_clean":"2024-09-21T02:04:11.271360+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:04:11.271360+0000","last_undegraded":"2024-09-21T02:04:11.271360+0000","last_fullsized":"2024-09-21T02:04:11.271360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424866+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:03:12.424866+0000","last_peered":"2024-09-21T02:03:12.424866+0000","last_clean":"2024-09-21T02:03:12.424866+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:03:12.424866+0000","last_undegraded":"2024-09-21T02:03:12.424866+0000","last_fullsized":"2024-09-21T02:03:12.424866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268705+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:04:11.268705+0000","last_peered":"2024-09-21T02:04:11.268705+0000","last_clean":"2024-09-21T02:04:11.268705+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:04:11.268705+0000","last_undegraded":"2024-09-21T02:04:11.268705+0000","last_fullsized":"2024-09-21T02:04:11.268705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.458260+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:04:21.458260+0000","last_peered":"2024-09-21T02:04:21.458260+0000","last_clean":"2024-09-21T02:04:21.458260+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:04:21.458260+0000","last_undegraded":"2024-09-21T02:04:21.458260+0000","last_fullsized":"2024-09-21T02:04:21.458260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75638,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1034,"num_read_kb":600,"num_write":269,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":68,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":406987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1064645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":406317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":406695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1470464,"data_stored":1000418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":406566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36792,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708360,"statfs":{"total":95995035648,"available":95957360640,"internally_reserved":0,"allocated":1482752,"data_stored":999423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":406427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":405859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444111,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":474860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":474176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":407905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:24.866 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T02:04:24.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:25.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3287936665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5481082d0 msgr2=0x7fb54810a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3287936665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5481082d0 0x7fb54810a7e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb538009900 tx=0x7fb53802a410 comp rx=0 tx=0).stop 2024-09-21T02:04:25.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3287936665 shutdown_connections 2024-09-21T02:04:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3287936665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5481082d0 0x7fb54810a7e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3287936665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5481056b0 0x7fb548107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3287936665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548102c30 0x7fb548105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3287936665 >> 172.21.15.92:0/3287936665 conn(0x7fb5480fd660 msgr2=0x7fb5480ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3287936665 shutdown_connections 2024-09-21T02:04:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.010+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3287936665 wait complete. 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 Processor -- start 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 -- start start 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 0x7fb5481020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5481056b0 0x7fb54819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5481082d0 0x7fb5481a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb54810e1e0 con 0x7fb548102c30 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb54810e060 con 0x7fb5481082d0 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54cdc6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb54810e360 con 0x7fb5481056b0 2024-09-21T02:04:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.011+0000 7fb54659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 0x7fb5481020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 0x7fb5481020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46534/0 (socket says 172.21.15.92:46534) 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 -- 172.21.15.92:0/3919051958 learned_addr learned my addr 172.21.15.92:0/3919051958 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 -- 172.21.15.92:0/3919051958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5481056b0 msgr2=0x7fb54819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5481056b0 0x7fb54819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 -- 172.21.15.92:0/3919051958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5481082d0 msgr2=0x7fb5481a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5481082d0 0x7fb5481a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 -- 172.21.15.92:0/3919051958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5481a18d0 con 0x7fb548102c30 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54659c640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 0x7fb5481020d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb54000ea80 tx=0x7fb54000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54000ce40 con 0x7fb548102c30 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb540005020 con 0x7fb548102c30 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5481a1bc0 con 0x7fb548102c30 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400116d0 con 0x7fb548102c30 2024-09-21T02:04:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.012+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5481a20c0 con 0x7fb548102c30 2024-09-21T02:04:25.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.013+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5480ff180 con 0x7fb548102c30 2024-09-21T02:04:25.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.015+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fb5400054d0 con 0x7fb548102c30 2024-09-21T02:04:25.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.015+0000 7fb53f7fe640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb5340b65a0 0x7fb5340b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.016+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb5400e7c10 con 0x7fb548102c30 2024-09-21T02:04:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.016+0000 7fb545d9b640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb5340b65a0 0x7fb5340b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.016+0000 7fb545d9b640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb5340b65a0 0x7fb5340b8ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb53000a990 tx=0x7fb530005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.019+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb540016030 con 0x7fb548102c30 2024-09-21T02:04:25.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:24 smithi136 ceph-mon[34754]: pgmap v343: 97 pgs: 97 active+clean; 651 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-21T02:04:25.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:24 smithi136 ceph-mon[34754]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-21T02:04:25.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.194+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 --> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb548063800 con 0x7fb5340b65a0 2024-09-21T02:04:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.199+0000 7fb53f7fe640 1 -- 172.21.15.92:0/3919051958 <== mgr.14150 v2:172.21.15.92:6800/1870152537 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258885 (secure 0 0 0) 0x7fb548063800 con 0x7fb5340b65a0 2024-09-21T02:04:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb5340b65a0 msgr2=0x7fb5340b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb5340b65a0 0x7fb5340b8ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb53000a990 tx=0x7fb530005c70 comp rx=0 tx=0).stop 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 msgr2=0x7fb5481020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 0x7fb5481020d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb54000ea80 tx=0x7fb54000ef40 comp rx=0 tx=0).stop 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 shutdown_connections 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb5340b65a0 0x7fb5340b8ab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5481082d0 0x7fb5481a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5481056b0 0x7fb54819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 --2- 172.21.15.92:0/3919051958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb548102c30 0x7fb5481020d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 >> 172.21.15.92:0/3919051958 conn(0x7fb5480fd660 msgr2=0x7fb548103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 shutdown_connections 2024-09-21T02:04:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.202+0000 7fb54cdc6640 1 -- 172.21.15.92:0/3919051958 wait complete. 2024-09-21T02:04:25.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:24 smithi092 ceph-mon[31190]: pgmap v343: 97 pgs: 97 active+clean; 651 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-21T02:04:25.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:24 smithi092 ceph-mon[31190]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-21T02:04:25.215 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-21T02:04:25.051780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":674116,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1154,"num_read_kb":694,"num_write":400,"num_write_kb":2324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2951908,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":829,"ondisk_log_size":829,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407200,"kb_used_data":12192,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534624,"statfs":{"total":1151940427776,"available":1151523454976,"internally_reserved":0,"allocated":12484608,"data_stored":6884688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8939,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":85,"num_write":22,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002948"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266526+0000","last_change":"2024-09-21T02:02:30.971852+0000","last_active":"2024-09-21T02:04:11.266526+0000","last_peered":"2024-09-21T02:04:11.266526+0000","last_clean":"2024-09-21T02:04:11.266526+0000","last_became_active":"2024-09-21T02:02:30.971499+0000","last_became_peered":"2024-09-21T02:02:30.971499+0000","last_unstale":"2024-09-21T02:04:11.266526+0000","last_undegraded":"2024-09-21T02:04:11.266526+0000","last_fullsized":"2024-09-21T02:04:11.266526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:38.906685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261131+0000","last_change":"2024-09-21T02:02:30.970865+0000","last_active":"2024-09-21T02:04:11.261131+0000","last_peered":"2024-09-21T02:04:11.261131+0000","last_clean":"2024-09-21T02:04:11.261131+0000","last_became_active":"2024-09-21T02:02:30.970517+0000","last_became_peered":"2024-09-21T02:02:30.970517+0000","last_unstale":"2024-09-21T02:04:11.261131+0000","last_undegraded":"2024-09-21T02:04:11.261131+0000","last_fullsized":"2024-09-21T02:04:11.261131+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271538+0000","last_change":"2024-09-21T02:02:30.983757+0000","last_active":"2024-09-21T02:04:11.271538+0000","last_peered":"2024-09-21T02:04:11.271538+0000","last_clean":"2024-09-21T02:04:11.271538+0000","last_became_active":"2024-09-21T02:02:30.983338+0000","last_became_peered":"2024-09-21T02:02:30.983338+0000","last_unstale":"2024-09-21T02:04:11.271538+0000","last_undegraded":"2024-09-21T02:04:11.271538+0000","last_fullsized":"2024-09-21T02:04:11.271538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:20.808440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261297+0000","last_change":"2024-09-21T02:02:30.979652+0000","last_active":"2024-09-21T02:04:11.261297+0000","last_peered":"2024-09-21T02:04:11.261297+0000","last_clean":"2024-09-21T02:04:11.261297+0000","last_became_active":"2024-09-21T02:02:30.979241+0000","last_became_peered":"2024-09-21T02:02:30.979241+0000","last_unstale":"2024-09-21T02:04:11.261297+0000","last_undegraded":"2024-09-21T02:04:11.261297+0000","last_fullsized":"2024-09-21T02:04:11.261297+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:43.637825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.257187+0000","last_change":"2024-09-21T02:02:30.970983+0000","last_active":"2024-09-21T02:04:11.257187+0000","last_peered":"2024-09-21T02:04:11.257187+0000","last_clean":"2024-09-21T02:04:11.257187+0000","last_became_active":"2024-09-21T02:02:30.970567+0000","last_became_peered":"2024-09-21T02:02:30.970567+0000","last_unstale":"2024-09-21T02:04:11.257187+0000","last_undegraded":"2024-09-21T02:04:11.257187+0000","last_fullsized":"2024-09-21T02:04:11.257187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:08.026440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262567+0000","last_change":"2024-09-21T02:02:30.983736+0000","last_active":"2024-09-21T02:04:11.262567+0000","last_peered":"2024-09-21T02:04:11.262567+0000","last_clean":"2024-09-21T02:04:11.262567+0000","last_became_active":"2024-09-21T02:02:30.983346+0000","last_became_peered":"2024-09-21T02:02:30.983346+0000","last_unstale":"2024-09-21T02:04:11.262567+0000","last_undegraded":"2024-09-21T02:04:11.262567+0000","last_fullsized":"2024-09-21T02:04:11.262567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.194403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271583+0000","last_change":"2024-09-21T02:02:30.967559+0000","last_active":"2024-09-21T02:04:11.271583+0000","last_peered":"2024-09-21T02:04:11.271583+0000","last_clean":"2024-09-21T02:04:11.271583+0000","last_became_active":"2024-09-21T02:02:30.967162+0000","last_became_peered":"2024-09-21T02:02:30.967162+0000","last_unstale":"2024-09-21T02:04:11.271583+0000","last_undegraded":"2024-09-21T02:04:11.271583+0000","last_fullsized":"2024-09-21T02:04:11.271583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:49.235683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269269+0000","last_change":"2024-09-21T02:02:30.974465+0000","last_active":"2024-09-21T02:04:11.269269+0000","last_peered":"2024-09-21T02:04:11.269269+0000","last_clean":"2024-09-21T02:04:11.269269+0000","last_became_active":"2024-09-21T02:02:30.973858+0000","last_became_peered":"2024-09-21T02:02:30.973858+0000","last_unstale":"2024-09-21T02:04:11.269269+0000","last_undegraded":"2024-09-21T02:04:11.269269+0000","last_fullsized":"2024-09-21T02:04:11.269269+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:25.621653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269202+0000","last_change":"2024-09-21T02:03:12.428541+0000","last_active":"2024-09-21T02:04:11.269202+0000","last_peered":"2024-09-21T02:04:11.269202+0000","last_clean":"2024-09-21T02:04:11.269202+0000","last_became_active":"2024-09-21T02:03:12.428177+0000","last_became_peered":"2024-09-21T02:03:12.428177+0000","last_unstale":"2024-09-21T02:04:11.269202+0000","last_undegraded":"2024-09-21T02:04:11.269202+0000","last_fullsized":"2024-09-21T02:04:11.269202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:42.955294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263252+0000","last_change":"2024-09-21T02:02:30.981941+0000","last_active":"2024-09-21T02:04:11.263252+0000","last_peered":"2024-09-21T02:04:11.263252+0000","last_clean":"2024-09-21T02:04:11.263252+0000","last_became_active":"2024-09-21T02:02:30.981586+0000","last_became_peered":"2024-09-21T02:02:30.981586+0000","last_unstale":"2024-09-21T02:04:11.263252+0000","last_undegraded":"2024-09-21T02:04:11.263252+0000","last_fullsized":"2024-09-21T02:04:11.263252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:12.348230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264394+0000","last_change":"2024-09-21T02:02:30.983838+0000","last_active":"2024-09-21T02:04:11.264394+0000","last_peered":"2024-09-21T02:04:11.264394+0000","last_clean":"2024-09-21T02:04:11.264394+0000","last_became_active":"2024-09-21T02:02:30.983486+0000","last_became_peered":"2024-09-21T02:02:30.983486+0000","last_unstale":"2024-09-21T02:04:11.264394+0000","last_undegraded":"2024-09-21T02:04:11.264394+0000","last_fullsized":"2024-09-21T02:04:11.264394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:41.787429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260057+0000","last_change":"2024-09-21T02:03:12.426083+0000","last_active":"2024-09-21T02:04:11.260057+0000","last_peered":"2024-09-21T02:04:11.260057+0000","last_clean":"2024-09-21T02:04:11.260057+0000","last_became_active":"2024-09-21T02:03:12.425716+0000","last_became_peered":"2024-09-21T02:03:12.425716+0000","last_unstale":"2024-09-21T02:04:11.260057+0000","last_undegraded":"2024-09-21T02:04:11.260057+0000","last_fullsized":"2024-09-21T02:04:11.260057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:54.219661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263538+0000","last_change":"2024-09-21T02:02:30.970405+0000","last_active":"2024-09-21T02:04:11.263538+0000","last_peered":"2024-09-21T02:04:11.263538+0000","last_clean":"2024-09-21T02:04:11.263538+0000","last_became_active":"2024-09-21T02:02:30.970081+0000","last_became_peered":"2024-09-21T02:02:30.970081+0000","last_unstale":"2024-09-21T02:04:11.263538+0000","last_undegraded":"2024-09-21T02:04:11.263538+0000","last_fullsized":"2024-09-21T02:04:11.263538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.856116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260466+0000","last_change":"2024-09-21T02:02:30.975579+0000","last_active":"2024-09-21T02:04:11.260466+0000","last_peered":"2024-09-21T02:04:11.260466+0000","last_clean":"2024-09-21T02:04:11.260466+0000","last_became_active":"2024-09-21T02:02:30.974973+0000","last_became_peered":"2024-09-21T02:02:30.974973+0000","last_unstale":"2024-09-21T02:04:11.260466+0000","last_undegraded":"2024-09-21T02:04:11.260466+0000","last_fullsized":"2024-09-21T02:04:11.260466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:06.395675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262755+0000","last_change":"2024-09-21T02:02:30.977959+0000","last_active":"2024-09-21T02:04:11.262755+0000","last_peered":"2024-09-21T02:04:11.262755+0000","last_clean":"2024-09-21T02:04:11.262755+0000","last_became_active":"2024-09-21T02:02:30.977574+0000","last_became_peered":"2024-09-21T02:02:30.977574+0000","last_unstale":"2024-09-21T02:04:11.262755+0000","last_undegraded":"2024-09-21T02:04:11.262755+0000","last_fullsized":"2024-09-21T02:04:11.262755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.699851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270135+0000","last_change":"2024-09-21T02:02:30.986437+0000","last_active":"2024-09-21T02:04:11.270135+0000","last_peered":"2024-09-21T02:04:11.270135+0000","last_clean":"2024-09-21T02:04:11.270135+0000","last_became_active":"2024-09-21T02:02:30.985894+0000","last_became_peered":"2024-09-21T02:02:30.985894+0000","last_unstale":"2024-09-21T02:04:11.270135+0000","last_undegraded":"2024-09-21T02:04:11.270135+0000","last_fullsized":"2024-09-21T02:04:11.270135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:21.278756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267528+0000","last_change":"2024-09-21T02:02:30.971695+0000","last_active":"2024-09-21T02:04:11.267528+0000","last_peered":"2024-09-21T02:04:11.267528+0000","last_clean":"2024-09-21T02:04:11.267528+0000","last_became_active":"2024-09-21T02:02:30.971320+0000","last_became_peered":"2024-09-21T02:02:30.971320+0000","last_unstale":"2024-09-21T02:04:11.267528+0000","last_undegraded":"2024-09-21T02:04:11.267528+0000","last_fullsized":"2024-09-21T02:04:11.267528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:09.102023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272663+0000","last_change":"2024-09-21T02:02:30.974064+0000","last_active":"2024-09-21T02:04:11.272663+0000","last_peered":"2024-09-21T02:04:11.272663+0000","last_clean":"2024-09-21T02:04:11.272663+0000","last_became_active":"2024-09-21T02:02:30.973671+0000","last_became_peered":"2024-09-21T02:02:30.973671+0000","last_unstale":"2024-09-21T02:04:11.272663+0000","last_undegraded":"2024-09-21T02:04:11.272663+0000","last_fullsized":"2024-09-21T02:04:11.272663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:44:42.033373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259439+0000","last_change":"2024-09-21T02:02:30.982965+0000","last_active":"2024-09-21T02:04:11.259439+0000","last_peered":"2024-09-21T02:04:11.259439+0000","last_clean":"2024-09-21T02:04:11.259439+0000","last_became_active":"2024-09-21T02:02:30.982332+0000","last_became_peered":"2024-09-21T02:02:30.982332+0000","last_unstale":"2024-09-21T02:04:11.259439+0000","last_undegraded":"2024-09-21T02:04:11.259439+0000","last_fullsized":"2024-09-21T02:04:11.259439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:03.868905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.265880+0000","last_change":"2024-09-21T02:02:30.974426+0000","last_active":"2024-09-21T02:04:11.265880+0000","last_peered":"2024-09-21T02:04:11.265880+0000","last_clean":"2024-09-21T02:04:11.265880+0000","last_became_active":"2024-09-21T02:02:30.974005+0000","last_became_peered":"2024-09-21T02:02:30.974005+0000","last_unstale":"2024-09-21T02:04:11.265880+0000","last_undegraded":"2024-09-21T02:04:11.265880+0000","last_fullsized":"2024-09-21T02:04:11.265880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:37.700208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.260041+0000","last_change":"2024-09-21T02:02:30.968840+0000","last_active":"2024-09-21T02:04:11.260041+0000","last_peered":"2024-09-21T02:04:11.260041+0000","last_clean":"2024-09-21T02:04:11.260041+0000","last_became_active":"2024-09-21T02:02:30.968471+0000","last_became_peered":"2024-09-21T02:02:30.968471+0000","last_unstale":"2024-09-21T02:04:11.260041+0000","last_undegraded":"2024-09-21T02:04:11.260041+0000","last_fullsized":"2024-09-21T02:04:11.260041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:27.639465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271873+0000","last_change":"2024-09-21T02:02:30.967980+0000","last_active":"2024-09-21T02:04:11.271873+0000","last_peered":"2024-09-21T02:04:11.271873+0000","last_clean":"2024-09-21T02:04:11.271873+0000","last_became_active":"2024-09-21T02:02:30.967605+0000","last_became_peered":"2024-09-21T02:02:30.967605+0000","last_unstale":"2024-09-21T02:04:11.271873+0000","last_undegraded":"2024-09-21T02:04:11.271873+0000","last_fullsized":"2024-09-21T02:04:11.271873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:42.222977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264705+0000","last_change":"2024-09-21T02:02:30.980663+0000","last_active":"2024-09-21T02:04:11.264705+0000","last_peered":"2024-09-21T02:04:11.264705+0000","last_clean":"2024-09-21T02:04:11.264705+0000","last_became_active":"2024-09-21T02:02:30.979953+0000","last_became_peered":"2024-09-21T02:02:30.979953+0000","last_unstale":"2024-09-21T02:04:11.264705+0000","last_undegraded":"2024-09-21T02:04:11.264705+0000","last_fullsized":"2024-09-21T02:04:11.264705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:12.402307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.247934+0000","last_change":"2024-09-21T02:03:12.426598+0000","last_active":"2024-09-21T02:04:11.247934+0000","last_peered":"2024-09-21T02:04:11.247934+0000","last_clean":"2024-09-21T02:04:11.247934+0000","last_became_active":"2024-09-21T02:03:12.425552+0000","last_became_peered":"2024-09-21T02:03:12.425552+0000","last_unstale":"2024-09-21T02:04:11.247934+0000","last_undegraded":"2024-09-21T02:04:11.247934+0000","last_fullsized":"2024-09-21T02:04:11.247934+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:40.991213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.242197+0000","last_change":"2024-09-21T02:02:30.970243+0000","last_active":"2024-09-21T02:04:11.242197+0000","last_peered":"2024-09-21T02:04:11.242197+0000","last_clean":"2024-09-21T02:04:11.242197+0000","last_became_active":"2024-09-21T02:02:30.968523+0000","last_became_peered":"2024-09-21T02:02:30.968523+0000","last_unstale":"2024-09-21T02:04:11.242197+0000","last_undegraded":"2024-09-21T02:04:11.242197+0000","last_fullsized":"2024-09-21T02:04:11.242197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:01.210448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.275028+0000","last_change":"2024-09-21T02:02:30.981321+0000","last_active":"2024-09-21T02:04:11.275028+0000","last_peered":"2024-09-21T02:04:11.275028+0000","last_clean":"2024-09-21T02:04:11.275028+0000","last_became_active":"2024-09-21T02:02:30.980676+0000","last_became_peered":"2024-09-21T02:02:30.980676+0000","last_unstale":"2024-09-21T02:04:11.275028+0000","last_undegraded":"2024-09-21T02:04:11.275028+0000","last_fullsized":"2024-09-21T02:04:11.275028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:47.654017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.265779+0000","last_change":"2024-09-21T02:03:12.428701+0000","last_active":"2024-09-21T02:04:11.265779+0000","last_peered":"2024-09-21T02:04:11.265779+0000","last_clean":"2024-09-21T02:04:11.265779+0000","last_became_active":"2024-09-21T02:03:12.428283+0000","last_became_peered":"2024-09-21T02:03:12.428283+0000","last_unstale":"2024-09-21T02:04:11.265779+0000","last_undegraded":"2024-09-21T02:04:11.265779+0000","last_fullsized":"2024-09-21T02:04:11.265779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:43.972970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261710+0000","last_change":"2024-09-21T02:02:30.969331+0000","last_active":"2024-09-21T02:04:11.261710+0000","last_peered":"2024-09-21T02:04:11.261710+0000","last_clean":"2024-09-21T02:04:11.261710+0000","last_became_active":"2024-09-21T02:02:30.969011+0000","last_became_peered":"2024-09-21T02:02:30.969011+0000","last_unstale":"2024-09-21T02:04:11.261710+0000","last_undegraded":"2024-09-21T02:04:11.261710+0000","last_fullsized":"2024-09-21T02:04:11.261710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:34.967225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263686+0000","last_change":"2024-09-21T02:02:30.969603+0000","last_active":"2024-09-21T02:04:11.263686+0000","last_peered":"2024-09-21T02:04:11.263686+0000","last_clean":"2024-09-21T02:04:11.263686+0000","last_became_active":"2024-09-21T02:02:30.969260+0000","last_became_peered":"2024-09-21T02:02:30.969260+0000","last_unstale":"2024-09-21T02:04:11.263686+0000","last_undegraded":"2024-09-21T02:04:11.263686+0000","last_fullsized":"2024-09-21T02:04:11.263686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:18.229371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259738+0000","last_change":"2024-09-21T02:02:30.979067+0000","last_active":"2024-09-21T02:04:11.259738+0000","last_peered":"2024-09-21T02:04:11.259738+0000","last_clean":"2024-09-21T02:04:11.259738+0000","last_became_active":"2024-09-21T02:02:30.978631+0000","last_became_peered":"2024-09-21T02:02:30.978631+0000","last_unstale":"2024-09-21T02:04:11.259738+0000","last_undegraded":"2024-09-21T02:04:11.259738+0000","last_fullsized":"2024-09-21T02:04:11.259738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:01.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271600+0000","last_change":"2024-09-21T02:02:30.979175+0000","last_active":"2024-09-21T02:04:11.271600+0000","last_peered":"2024-09-21T02:04:11.271600+0000","last_clean":"2024-09-21T02:04:11.271600+0000","last_became_active":"2024-09-21T02:02:30.978055+0000","last_became_peered":"2024-09-21T02:02:30.978055+0000","last_unstale":"2024-09-21T02:04:11.271600+0000","last_undegraded":"2024-09-21T02:04:11.271600+0000","last_fullsized":"2024-09-21T02:04:11.271600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.322398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268792+0000","last_change":"2024-09-21T02:02:30.978373+0000","last_active":"2024-09-21T02:04:11.268792+0000","last_peered":"2024-09-21T02:04:11.268792+0000","last_clean":"2024-09-21T02:04:11.268792+0000","last_became_active":"2024-09-21T02:02:30.978006+0000","last_became_peered":"2024-09-21T02:02:30.978006+0000","last_unstale":"2024-09-21T02:04:11.268792+0000","last_undegraded":"2024-09-21T02:04:11.268792+0000","last_fullsized":"2024-09-21T02:04:11.268792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:51.217596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.254147+0000","last_change":"2024-09-21T02:02:30.973142+0000","last_active":"2024-09-21T02:04:11.254147+0000","last_peered":"2024-09-21T02:04:11.254147+0000","last_clean":"2024-09-21T02:04:11.254147+0000","last_became_active":"2024-09-21T02:02:30.972770+0000","last_became_peered":"2024-09-21T02:02:30.972770+0000","last_unstale":"2024-09-21T02:04:11.254147+0000","last_undegraded":"2024-09-21T02:04:11.254147+0000","last_fullsized":"2024-09-21T02:04:11.254147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:47.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'8","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.460283+0000","last_change":"2024-09-21T02:02:28.957670+0000","last_active":"2024-09-21T02:04:21.460283+0000","last_peered":"2024-09-21T02:04:21.460283+0000","last_clean":"2024-09-21T02:04:21.460283+0000","last_became_active":"2024-09-21T02:02:28.957307+0000","last_became_peered":"2024-09-21T02:02:28.957307+0000","last_unstale":"2024-09-21T02:04:21.460283+0000","last_undegraded":"2024-09-21T02:04:21.460283+0000","last_fullsized":"2024-09-21T02:04:21.460283+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:09:54.910113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270227+0000","last_change":"2024-09-21T02:02:30.971464+0000","last_active":"2024-09-21T02:04:11.270227+0000","last_peered":"2024-09-21T02:04:11.270227+0000","last_clean":"2024-09-21T02:04:11.270227+0000","last_became_active":"2024-09-21T02:02:30.971132+0000","last_became_peered":"2024-09-21T02:02:30.971132+0000","last_unstale":"2024-09-21T02:04:11.270227+0000","last_undegraded":"2024-09-21T02:04:11.270227+0000","last_fullsized":"2024-09-21T02:04:11.270227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:42.728331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424950+0000","last_change":"2024-09-21T02:02:28.953010+0000","last_active":"2024-09-21T02:03:12.424950+0000","last_peered":"2024-09-21T02:03:12.424950+0000","last_clean":"2024-09-21T02:03:12.424950+0000","last_became_active":"2024-09-21T02:02:28.952536+0000","last_became_peered":"2024-09-21T02:02:28.952536+0000","last_unstale":"2024-09-21T02:03:12.424950+0000","last_undegraded":"2024-09-21T02:03:12.424950+0000","last_fullsized":"2024-09-21T02:03:12.424950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:55.221490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641291+0000","last_change":"2024-09-21T02:02:28.946240+0000","last_active":"2024-09-21T02:03:12.641291+0000","last_peered":"2024-09-21T02:03:12.641291+0000","last_clean":"2024-09-21T02:03:12.641291+0000","last_became_active":"2024-09-21T02:02:28.945813+0000","last_became_peered":"2024-09-21T02:02:28.945813+0000","last_unstale":"2024-09-21T02:03:12.641291+0000","last_undegraded":"2024-09-21T02:03:12.641291+0000","last_fullsized":"2024-09-21T02:03:12.641291+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:48.592645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264738+0000","last_change":"2024-09-21T02:02:30.970927+0000","last_active":"2024-09-21T02:04:11.264738+0000","last_peered":"2024-09-21T02:04:11.264738+0000","last_clean":"2024-09-21T02:04:11.264738+0000","last_became_active":"2024-09-21T02:02:30.969599+0000","last_became_peered":"2024-09-21T02:02:30.969599+0000","last_unstale":"2024-09-21T02:04:11.264738+0000","last_undegraded":"2024-09-21T02:04:11.264738+0000","last_fullsized":"2024-09-21T02:04:11.264738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:33.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419546+0000","last_change":"2024-09-21T02:02:28.951592+0000","last_active":"2024-09-21T02:03:12.419546+0000","last_peered":"2024-09-21T02:03:12.419546+0000","last_clean":"2024-09-21T02:03:12.419546+0000","last_became_active":"2024-09-21T02:02:28.951035+0000","last_became_peered":"2024-09-21T02:02:28.951035+0000","last_unstale":"2024-09-21T02:03:12.419546+0000","last_undegraded":"2024-09-21T02:03:12.419546+0000","last_fullsized":"2024-09-21T02:03:12.419546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:04.026105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259983+0000","last_change":"2024-09-21T02:02:30.974898+0000","last_active":"2024-09-21T02:04:11.259983+0000","last_peered":"2024-09-21T02:04:11.259983+0000","last_clean":"2024-09-21T02:04:11.259983+0000","last_became_active":"2024-09-21T02:02:30.974556+0000","last_became_peered":"2024-09-21T02:02:30.974556+0000","last_unstale":"2024-09-21T02:04:11.259983+0000","last_undegraded":"2024-09-21T02:04:11.259983+0000","last_fullsized":"2024-09-21T02:04:11.259983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:32.900240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":235,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:23.461674+0000","last_change":"2024-09-21T02:02:28.953620+0000","last_active":"2024-09-21T02:04:23.461674+0000","last_peered":"2024-09-21T02:04:23.461674+0000","last_clean":"2024-09-21T02:04:23.461674+0000","last_became_active":"2024-09-21T02:02:28.952874+0000","last_became_peered":"2024-09-21T02:02:28.952874+0000","last_unstale":"2024-09-21T02:04:23.461674+0000","last_undegraded":"2024-09-21T02:04:23.461674+0000","last_fullsized":"2024-09-21T02:04:23.461674+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:46.510692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.274258+0000","last_change":"2024-09-21T02:02:30.970700+0000","last_active":"2024-09-21T02:04:11.274258+0000","last_peered":"2024-09-21T02:04:11.274258+0000","last_clean":"2024-09-21T02:04:11.274258+0000","last_became_active":"2024-09-21T02:02:30.970317+0000","last_became_peered":"2024-09-21T02:02:30.970317+0000","last_unstale":"2024-09-21T02:04:11.274258+0000","last_undegraded":"2024-09-21T02:04:11.274258+0000","last_fullsized":"2024-09-21T02:04:11.274258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:03.521894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421592+0000","last_change":"2024-09-21T02:02:28.955521+0000","last_active":"2024-09-21T02:03:12.421592+0000","last_peered":"2024-09-21T02:03:12.421592+0000","last_clean":"2024-09-21T02:03:12.421592+0000","last_became_active":"2024-09-21T02:02:28.955131+0000","last_became_peered":"2024-09-21T02:02:28.955131+0000","last_unstale":"2024-09-21T02:03:12.421592+0000","last_undegraded":"2024-09-21T02:03:12.421592+0000","last_fullsized":"2024-09-21T02:03:12.421592+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:02.805096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271720+0000","last_change":"2024-09-21T02:03:12.423870+0000","last_active":"2024-09-21T02:04:11.271720+0000","last_peered":"2024-09-21T02:04:11.271720+0000","last_clean":"2024-09-21T02:04:11.271720+0000","last_became_active":"2024-09-21T02:03:12.423441+0000","last_became_peered":"2024-09-21T02:03:12.423441+0000","last_unstale":"2024-09-21T02:04:11.271720+0000","last_undegraded":"2024-09-21T02:04:11.271720+0000","last_fullsized":"2024-09-21T02:04:11.271720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:59.071213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419481+0000","last_change":"2024-09-21T02:02:28.976954+0000","last_active":"2024-09-21T02:03:12.419481+0000","last_peered":"2024-09-21T02:03:12.419481+0000","last_clean":"2024-09-21T02:03:12.419481+0000","last_became_active":"2024-09-21T02:02:28.976416+0000","last_became_peered":"2024-09-21T02:02:28.976416+0000","last_unstale":"2024-09-21T02:03:12.419481+0000","last_undegraded":"2024-09-21T02:03:12.419481+0000","last_fullsized":"2024-09-21T02:03:12.419481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:31.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.269103+0000","last_change":"2024-09-21T02:02:30.974866+0000","last_active":"2024-09-21T02:04:11.269103+0000","last_peered":"2024-09-21T02:04:11.269103+0000","last_clean":"2024-09-21T02:04:11.269103+0000","last_became_active":"2024-09-21T02:02:30.974487+0000","last_became_peered":"2024-09-21T02:02:30.974487+0000","last_unstale":"2024-09-21T02:04:11.269103+0000","last_undegraded":"2024-09-21T02:04:11.269103+0000","last_fullsized":"2024-09-21T02:04:11.269103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:50.274328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'38","reported_seq":281,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:23.462739+0000","last_change":"2024-09-21T02:02:28.956463+0000","last_active":"2024-09-21T02:04:23.462739+0000","last_peered":"2024-09-21T02:04:23.462739+0000","last_clean":"2024-09-21T02:04:23.462739+0000","last_became_active":"2024-09-21T02:02:28.956102+0000","last_became_peered":"2024-09-21T02:02:28.956102+0000","last_unstale":"2024-09-21T02:04:23.462739+0000","last_undegraded":"2024-09-21T02:04:23.462739+0000","last_fullsized":"2024-09-21T02:04:23.462739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:32.528519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74265,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":182,"num_write":40,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263855+0000","last_change":"2024-09-21T02:02:30.982814+0000","last_active":"2024-09-21T02:04:11.263855+0000","last_peered":"2024-09-21T02:04:11.263855+0000","last_clean":"2024-09-21T02:04:11.263855+0000","last_became_active":"2024-09-21T02:02:30.982425+0000","last_became_peered":"2024-09-21T02:02:30.982425+0000","last_unstale":"2024-09-21T02:04:11.263855+0000","last_undegraded":"2024-09-21T02:04:11.263855+0000","last_fullsized":"2024-09-21T02:04:11.263855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:55.177237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442392+0000","last_change":"2024-09-21T02:03:12.441759+0000","last_active":"2024-09-21T02:03:12.442392+0000","last_peered":"2024-09-21T02:03:12.442392+0000","last_clean":"2024-09-21T02:03:12.442392+0000","last_became_active":"2024-09-21T02:03:12.426827+0000","last_became_peered":"2024-09-21T02:03:12.426827+0000","last_unstale":"2024-09-21T02:03:12.442392+0000","last_undegraded":"2024-09-21T02:03:12.442392+0000","last_fullsized":"2024-09-21T02:03:12.442392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:33.917276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259856+0000","last_change":"2024-09-21T02:02:30.967195+0000","last_active":"2024-09-21T02:04:11.259856+0000","last_peered":"2024-09-21T02:04:11.259856+0000","last_clean":"2024-09-21T02:04:11.259856+0000","last_became_active":"2024-09-21T02:02:30.966697+0000","last_became_peered":"2024-09-21T02:02:30.966697+0000","last_unstale":"2024-09-21T02:04:11.259856+0000","last_undegraded":"2024-09-21T02:04:11.259856+0000","last_fullsized":"2024-09-21T02:04:11.259856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:03.831225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.418936+0000","last_change":"2024-09-21T02:02:28.957586+0000","last_active":"2024-09-21T02:03:12.418936+0000","last_peered":"2024-09-21T02:03:12.418936+0000","last_clean":"2024-09-21T02:03:12.418936+0000","last_became_active":"2024-09-21T02:02:28.957233+0000","last_became_peered":"2024-09-21T02:02:28.957233+0000","last_unstale":"2024-09-21T02:03:12.418936+0000","last_undegraded":"2024-09-21T02:03:12.418936+0000","last_fullsized":"2024-09-21T02:03:12.418936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:22.080633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.263535+0000","last_change":"2024-09-21T02:03:12.643022+0000","last_active":"2024-09-21T02:04:11.263535+0000","last_peered":"2024-09-21T02:04:11.263535+0000","last_clean":"2024-09-21T02:04:11.263535+0000","last_became_active":"2024-09-21T02:03:12.642501+0000","last_became_peered":"2024-09-21T02:03:12.642501+0000","last_unstale":"2024-09-21T02:04:11.263535+0000","last_undegraded":"2024-09-21T02:04:11.263535+0000","last_fullsized":"2024-09-21T02:04:11.263535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:29.600445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'7","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:23.498623+0000","last_change":"2024-09-21T02:02:28.954310+0000","last_active":"2024-09-21T02:04:23.498623+0000","last_peered":"2024-09-21T02:04:23.498623+0000","last_clean":"2024-09-21T02:04:23.498623+0000","last_became_active":"2024-09-21T02:02:28.953520+0000","last_became_peered":"2024-09-21T02:02:28.953520+0000","last_unstale":"2024-09-21T02:04:23.498623+0000","last_undegraded":"2024-09-21T02:04:23.498623+0000","last_fullsized":"2024-09-21T02:04:23.498623+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.613486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271795+0000","last_change":"2024-09-21T02:02:30.977538+0000","last_active":"2024-09-21T02:04:11.271795+0000","last_peered":"2024-09-21T02:04:11.271795+0000","last_clean":"2024-09-21T02:04:11.271795+0000","last_became_active":"2024-09-21T02:02:30.976726+0000","last_became_peered":"2024-09-21T02:02:30.976726+0000","last_unstale":"2024-09-21T02:04:11.271795+0000","last_undegraded":"2024-09-21T02:04:11.271795+0000","last_fullsized":"2024-09-21T02:04:11.271795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:38:05.351663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.423964+0000","last_change":"2024-09-21T02:02:28.976675+0000","last_active":"2024-09-21T02:03:12.423964+0000","last_peered":"2024-09-21T02:03:12.423964+0000","last_clean":"2024-09-21T02:03:12.423964+0000","last_became_active":"2024-09-21T02:02:28.976273+0000","last_became_peered":"2024-09-21T02:02:28.976273+0000","last_unstale":"2024-09-21T02:03:12.423964+0000","last_undegraded":"2024-09-21T02:03:12.423964+0000","last_fullsized":"2024-09-21T02:03:12.423964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:00.537799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270043+0000","last_change":"2024-09-21T02:02:30.977394+0000","last_active":"2024-09-21T02:04:11.270043+0000","last_peered":"2024-09-21T02:04:11.270043+0000","last_clean":"2024-09-21T02:04:11.270043+0000","last_became_active":"2024-09-21T02:02:30.977021+0000","last_became_peered":"2024-09-21T02:02:30.977021+0000","last_unstale":"2024-09-21T02:04:11.270043+0000","last_undegraded":"2024-09-21T02:04:11.270043+0000","last_fullsized":"2024-09-21T02:04:11.270043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:29.572039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425082+0000","last_change":"2024-09-21T02:02:28.955650+0000","last_active":"2024-09-21T02:03:12.425082+0000","last_peered":"2024-09-21T02:03:12.425082+0000","last_clean":"2024-09-21T02:03:12.425082+0000","last_became_active":"2024-09-21T02:02:28.955221+0000","last_became_peered":"2024-09-21T02:02:28.955221+0000","last_unstale":"2024-09-21T02:03:12.425082+0000","last_undegraded":"2024-09-21T02:03:12.425082+0000","last_fullsized":"2024-09-21T02:03:12.425082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:07.335825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268289+0000","last_change":"2024-09-21T02:02:30.970147+0000","last_active":"2024-09-21T02:04:11.268289+0000","last_peered":"2024-09-21T02:04:11.268289+0000","last_clean":"2024-09-21T02:04:11.268289+0000","last_became_active":"2024-09-21T02:02:30.969672+0000","last_became_peered":"2024-09-21T02:02:30.969672+0000","last_unstale":"2024-09-21T02:04:11.268289+0000","last_undegraded":"2024-09-21T02:04:11.268289+0000","last_fullsized":"2024-09-21T02:04:11.268289+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:30.480357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.641780+0000","last_change":"2024-09-21T02:02:28.950355+0000","last_active":"2024-09-21T02:03:12.641780+0000","last_peered":"2024-09-21T02:03:12.641780+0000","last_clean":"2024-09-21T02:03:12.641780+0000","last_became_active":"2024-09-21T02:02:28.949985+0000","last_became_peered":"2024-09-21T02:02:28.949985+0000","last_unstale":"2024-09-21T02:03:12.641780+0000","last_undegraded":"2024-09-21T02:03:12.641780+0000","last_fullsized":"2024-09-21T02:03:12.641780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:05:05.190317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266485+0000","last_change":"2024-09-21T02:02:30.978453+0000","last_active":"2024-09-21T02:04:11.266485+0000","last_peered":"2024-09-21T02:04:11.266485+0000","last_clean":"2024-09-21T02:04:11.266485+0000","last_became_active":"2024-09-21T02:02:30.978010+0000","last_became_peered":"2024-09-21T02:02:30.978010+0000","last_unstale":"2024-09-21T02:04:11.266485+0000","last_undegraded":"2024-09-21T02:04:11.266485+0000","last_fullsized":"2024-09-21T02:04:11.266485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:51.648226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419713+0000","last_change":"2024-09-21T02:02:28.949977+0000","last_active":"2024-09-21T02:03:12.419713+0000","last_peered":"2024-09-21T02:03:12.419713+0000","last_clean":"2024-09-21T02:03:12.419713+0000","last_became_active":"2024-09-21T02:02:28.949307+0000","last_became_peered":"2024-09-21T02:02:28.949307+0000","last_unstale":"2024-09-21T02:03:12.419713+0000","last_undegraded":"2024-09-21T02:03:12.419713+0000","last_fullsized":"2024-09-21T02:03:12.419713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:50.009308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272095+0000","last_change":"2024-09-21T02:02:30.968571+0000","last_active":"2024-09-21T02:04:11.272095+0000","last_peered":"2024-09-21T02:04:11.272095+0000","last_clean":"2024-09-21T02:04:11.272095+0000","last_became_active":"2024-09-21T02:02:30.968201+0000","last_became_peered":"2024-09-21T02:02:30.968201+0000","last_unstale":"2024-09-21T02:04:11.272095+0000","last_undegraded":"2024-09-21T02:04:11.272095+0000","last_fullsized":"2024-09-21T02:04:11.272095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:05.404591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":137,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:22.460678+0000","last_change":"2024-09-21T02:02:29.404037+0000","last_active":"2024-09-21T02:04:22.460678+0000","last_peered":"2024-09-21T02:04:22.460678+0000","last_clean":"2024-09-21T02:04:22.460678+0000","last_became_active":"2024-09-21T02:02:29.403549+0000","last_became_peered":"2024-09-21T02:02:29.403549+0000","last_unstale":"2024-09-21T02:04:22.460678+0000","last_undegraded":"2024-09-21T02:04:22.460678+0000","last_fullsized":"2024-09-21T02:04:22.460678+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:04.578301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.266415+0000","last_change":"2024-09-21T02:02:30.984089+0000","last_active":"2024-09-21T02:04:11.266415+0000","last_peered":"2024-09-21T02:04:11.266415+0000","last_clean":"2024-09-21T02:04:11.266415+0000","last_became_active":"2024-09-21T02:02:30.983090+0000","last_became_peered":"2024-09-21T02:02:30.983090+0000","last_unstale":"2024-09-21T02:04:11.266415+0000","last_undegraded":"2024-09-21T02:04:11.266415+0000","last_fullsized":"2024-09-21T02:04:11.266415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:21.006079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422172+0000","last_change":"2024-09-21T02:01:20.994990+0000","last_active":"2024-09-21T02:03:12.422172+0000","last_peered":"2024-09-21T02:03:12.422172+0000","last_clean":"2024-09-21T02:03:12.422172+0000","last_became_active":"2024-09-21T02:01:20.989270+0000","last_became_peered":"2024-09-21T02:01:20.989270+0000","last_unstale":"2024-09-21T02:03:12.422172+0000","last_undegraded":"2024-09-21T02:03:12.422172+0000","last_fullsized":"2024-09-21T02:03:12.422172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:57:59.270371+0000","last_clean_scrub_stamp":"2024-09-21T01:57:59.270371+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:49.094216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419020+0000","last_change":"2024-09-21T02:02:28.950467+0000","last_active":"2024-09-21T02:03:12.419020+0000","last_peered":"2024-09-21T02:03:12.419020+0000","last_clean":"2024-09-21T02:03:12.419020+0000","last_became_active":"2024-09-21T02:02:28.949950+0000","last_became_peered":"2024-09-21T02:02:28.949950+0000","last_unstale":"2024-09-21T02:03:12.419020+0000","last_undegraded":"2024-09-21T02:03:12.419020+0000","last_fullsized":"2024-09-21T02:03:12.419020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:35.712343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271556+0000","last_change":"2024-09-21T02:02:30.977325+0000","last_active":"2024-09-21T02:04:11.271556+0000","last_peered":"2024-09-21T02:04:11.271556+0000","last_clean":"2024-09-21T02:04:11.271556+0000","last_became_active":"2024-09-21T02:02:30.976914+0000","last_became_peered":"2024-09-21T02:02:30.976914+0000","last_unstale":"2024-09-21T02:04:11.271556+0000","last_undegraded":"2024-09-21T02:04:11.271556+0000","last_fullsized":"2024-09-21T02:04:11.271556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:15.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.421843+0000","last_change":"2024-09-21T02:02:28.955926+0000","last_active":"2024-09-21T02:03:12.421843+0000","last_peered":"2024-09-21T02:03:12.421843+0000","last_clean":"2024-09-21T02:03:12.421843+0000","last_became_active":"2024-09-21T02:02:28.955562+0000","last_became_peered":"2024-09-21T02:02:28.955562+0000","last_unstale":"2024-09-21T02:03:12.421843+0000","last_undegraded":"2024-09-21T02:03:12.421843+0000","last_fullsized":"2024-09-21T02:03:12.421843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:54.820161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.258590+0000","last_change":"2024-09-21T02:02:30.977544+0000","last_active":"2024-09-21T02:04:11.258590+0000","last_peered":"2024-09-21T02:04:11.258590+0000","last_clean":"2024-09-21T02:04:11.258590+0000","last_became_active":"2024-09-21T02:02:30.977194+0000","last_became_peered":"2024-09-21T02:02:30.977194+0000","last_unstale":"2024-09-21T02:04:11.258590+0000","last_undegraded":"2024-09-21T02:04:11.258590+0000","last_fullsized":"2024-09-21T02:04:11.258590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:41.440836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:22.496334+0000","last_change":"2024-09-21T02:02:28.975273+0000","last_active":"2024-09-21T02:04:22.496334+0000","last_peered":"2024-09-21T02:04:22.496334+0000","last_clean":"2024-09-21T02:04:22.496334+0000","last_became_active":"2024-09-21T02:02:28.974692+0000","last_became_peered":"2024-09-21T02:02:28.974692+0000","last_unstale":"2024-09-21T02:04:22.496334+0000","last_undegraded":"2024-09-21T02:04:22.496334+0000","last_fullsized":"2024-09-21T02:04:22.496334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:17.288947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.270528+0000","last_change":"2024-09-21T02:02:30.983625+0000","last_active":"2024-09-21T02:04:11.270528+0000","last_peered":"2024-09-21T02:04:11.270528+0000","last_clean":"2024-09-21T02:04:11.270528+0000","last_became_active":"2024-09-21T02:02:30.983225+0000","last_became_peered":"2024-09-21T02:02:30.983225+0000","last_unstale":"2024-09-21T02:04:11.270528+0000","last_undegraded":"2024-09-21T02:04:11.270528+0000","last_fullsized":"2024-09-21T02:04:11.270528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:46.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'8","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.458899+0000","last_change":"2024-09-21T02:02:28.952897+0000","last_active":"2024-09-21T02:04:21.458899+0000","last_peered":"2024-09-21T02:04:21.458899+0000","last_clean":"2024-09-21T02:04:21.458899+0000","last_became_active":"2024-09-21T02:02:28.952486+0000","last_became_peered":"2024-09-21T02:02:28.952486+0000","last_unstale":"2024-09-21T02:04:21.458899+0000","last_undegraded":"2024-09-21T02:04:21.458899+0000","last_fullsized":"2024-09-21T02:04:21.458899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:54.277082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.259912+0000","last_change":"2024-09-21T02:02:30.977595+0000","last_active":"2024-09-21T02:04:11.259912+0000","last_peered":"2024-09-21T02:04:11.259912+0000","last_clean":"2024-09-21T02:04:11.259912+0000","last_became_active":"2024-09-21T02:02:30.977209+0000","last_became_peered":"2024-09-21T02:02:30.977209+0000","last_unstale":"2024-09-21T02:04:11.259912+0000","last_undegraded":"2024-09-21T02:04:11.259912+0000","last_fullsized":"2024-09-21T02:04:11.259912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.417918+0000","last_change":"2024-09-21T02:02:28.976136+0000","last_active":"2024-09-21T02:03:12.417918+0000","last_peered":"2024-09-21T02:03:12.417918+0000","last_clean":"2024-09-21T02:03:12.417918+0000","last_became_active":"2024-09-21T02:02:28.975655+0000","last_became_peered":"2024-09-21T02:02:28.975655+0000","last_unstale":"2024-09-21T02:03:12.417918+0000","last_undegraded":"2024-09-21T02:03:12.417918+0000","last_fullsized":"2024-09-21T02:03:12.417918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:02.523878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.255582+0000","last_change":"2024-09-21T02:02:30.980028+0000","last_active":"2024-09-21T02:04:11.255582+0000","last_peered":"2024-09-21T02:04:11.255582+0000","last_clean":"2024-09-21T02:04:11.255582+0000","last_became_active":"2024-09-21T02:02:30.979582+0000","last_became_peered":"2024-09-21T02:02:30.979582+0000","last_unstale":"2024-09-21T02:04:11.255582+0000","last_undegraded":"2024-09-21T02:04:11.255582+0000","last_fullsized":"2024-09-21T02:04:11.255582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:55.563814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.425092+0000","last_change":"2024-09-21T02:02:28.975075+0000","last_active":"2024-09-21T02:03:12.425092+0000","last_peered":"2024-09-21T02:03:12.425092+0000","last_clean":"2024-09-21T02:03:12.425092+0000","last_became_active":"2024-09-21T02:02:28.974732+0000","last_became_peered":"2024-09-21T02:02:28.974732+0000","last_unstale":"2024-09-21T02:03:12.425092+0000","last_undegraded":"2024-09-21T02:03:12.425092+0000","last_fullsized":"2024-09-21T02:03:12.425092+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:28.602974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.261100+0000","last_change":"2024-09-21T02:02:30.967923+0000","last_active":"2024-09-21T02:04:11.261100+0000","last_peered":"2024-09-21T02:04:11.261100+0000","last_clean":"2024-09-21T02:04:11.261100+0000","last_became_active":"2024-09-21T02:02:30.967571+0000","last_became_peered":"2024-09-21T02:02:30.967571+0000","last_unstale":"2024-09-21T02:04:11.261100+0000","last_undegraded":"2024-09-21T02:04:11.261100+0000","last_fullsized":"2024-09-21T02:04:11.261100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:53.412978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'19","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:20.991825+0000","last_change":"2024-09-21T02:02:28.953786+0000","last_active":"2024-09-21T02:04:20.991825+0000","last_peered":"2024-09-21T02:04:20.991825+0000","last_clean":"2024-09-21T02:04:20.991825+0000","last_became_active":"2024-09-21T02:02:28.953454+0000","last_became_peered":"2024-09-21T02:02:28.953454+0000","last_unstale":"2024-09-21T02:04:20.991825+0000","last_undegraded":"2024-09-21T02:04:20.991825+0000","last_fullsized":"2024-09-21T02:04:20.991825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:23:22.346060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267166+0000","last_change":"2024-09-21T02:02:30.969750+0000","last_active":"2024-09-21T02:04:11.267166+0000","last_peered":"2024-09-21T02:04:11.267166+0000","last_clean":"2024-09-21T02:04:11.267166+0000","last_became_active":"2024-09-21T02:02:30.969403+0000","last_became_peered":"2024-09-21T02:02:30.969403+0000","last_unstale":"2024-09-21T02:04:11.267166+0000","last_undegraded":"2024-09-21T02:04:11.267166+0000","last_fullsized":"2024-09-21T02:04:11.267166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:00.175914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.419259+0000","last_change":"2024-09-21T02:02:29.404207+0000","last_active":"2024-09-21T02:03:12.419259+0000","last_peered":"2024-09-21T02:03:12.419259+0000","last_clean":"2024-09-21T02:03:12.419259+0000","last_became_active":"2024-09-21T02:02:29.403792+0000","last_became_peered":"2024-09-21T02:02:29.403792+0000","last_unstale":"2024-09-21T02:03:12.419259+0000","last_undegraded":"2024-09-21T02:03:12.419259+0000","last_fullsized":"2024-09-21T02:03:12.419259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:34.498056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.262737+0000","last_change":"2024-09-21T02:02:30.971467+0000","last_active":"2024-09-21T02:04:11.262737+0000","last_peered":"2024-09-21T02:04:11.262737+0000","last_clean":"2024-09-21T02:04:11.262737+0000","last_became_active":"2024-09-21T02:02:30.971099+0000","last_became_peered":"2024-09-21T02:02:30.971099+0000","last_unstale":"2024-09-21T02:04:11.262737+0000","last_undegraded":"2024-09-21T02:04:11.262737+0000","last_fullsized":"2024-09-21T02:04:11.262737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:57.359255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.422234+0000","last_change":"2024-09-21T02:02:28.976767+0000","last_active":"2024-09-21T02:03:12.422234+0000","last_peered":"2024-09-21T02:03:12.422234+0000","last_clean":"2024-09-21T02:03:12.422234+0000","last_became_active":"2024-09-21T02:02:28.976399+0000","last_became_peered":"2024-09-21T02:02:28.976399+0000","last_unstale":"2024-09-21T02:03:12.422234+0000","last_undegraded":"2024-09-21T02:03:12.422234+0000","last_fullsized":"2024-09-21T02:03:12.422234+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:13:09.934205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.223134+0000","last_change":"2024-09-21T02:02:30.974346+0000","last_active":"2024-09-21T02:04:11.223134+0000","last_peered":"2024-09-21T02:04:11.223134+0000","last_clean":"2024-09-21T02:04:11.223134+0000","last_became_active":"2024-09-21T02:02:30.973906+0000","last_became_peered":"2024-09-21T02:02:30.973906+0000","last_unstale":"2024-09-21T02:04:11.223134+0000","last_undegraded":"2024-09-21T02:04:11.223134+0000","last_fullsized":"2024-09-21T02:04:11.223134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:46.454493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424628+0000","last_change":"2024-09-21T02:02:28.950469+0000","last_active":"2024-09-21T02:03:12.424628+0000","last_peered":"2024-09-21T02:03:12.424628+0000","last_clean":"2024-09-21T02:03:12.424628+0000","last_became_active":"2024-09-21T02:02:28.950059+0000","last_became_peered":"2024-09-21T02:02:28.950059+0000","last_unstale":"2024-09-21T02:03:12.424628+0000","last_undegraded":"2024-09-21T02:03:12.424628+0000","last_fullsized":"2024-09-21T02:03:12.424628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:30.281948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264940+0000","last_change":"2024-09-21T02:02:30.971086+0000","last_active":"2024-09-21T02:04:11.264940+0000","last_peered":"2024-09-21T02:04:11.264940+0000","last_clean":"2024-09-21T02:04:11.264940+0000","last_became_active":"2024-09-21T02:02:30.970691+0000","last_became_peered":"2024-09-21T02:02:30.970691+0000","last_unstale":"2024-09-21T02:04:11.264940+0000","last_undegraded":"2024-09-21T02:04:11.264940+0000","last_fullsized":"2024-09-21T02:04:11.264940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.806340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.426609+0000","last_change":"2024-09-21T02:02:28.955168+0000","last_active":"2024-09-21T02:03:12.426609+0000","last_peered":"2024-09-21T02:03:12.426609+0000","last_clean":"2024-09-21T02:03:12.426609+0000","last_became_active":"2024-09-21T02:02:28.954686+0000","last_became_peered":"2024-09-21T02:02:28.954686+0000","last_unstale":"2024-09-21T02:03:12.426609+0000","last_undegraded":"2024-09-21T02:03:12.426609+0000","last_fullsized":"2024-09-21T02:03:12.426609+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:15.268047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.255303+0000","last_change":"2024-09-21T02:02:30.973295+0000","last_active":"2024-09-21T02:04:11.255303+0000","last_peered":"2024-09-21T02:04:11.255303+0000","last_clean":"2024-09-21T02:04:11.255303+0000","last_became_active":"2024-09-21T02:02:30.972939+0000","last_became_peered":"2024-09-21T02:02:30.972939+0000","last_unstale":"2024-09-21T02:04:11.255303+0000","last_undegraded":"2024-09-21T02:04:11.255303+0000","last_fullsized":"2024-09-21T02:04:11.255303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:44.823614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.446014+0000","last_change":"2024-09-21T02:03:12.445516+0000","last_active":"2024-09-21T02:03:12.446014+0000","last_peered":"2024-09-21T02:03:12.446014+0000","last_clean":"2024-09-21T02:03:12.446014+0000","last_became_active":"2024-09-21T02:03:12.430149+0000","last_became_peered":"2024-09-21T02:03:12.430149+0000","last_unstale":"2024-09-21T02:03:12.446014+0000","last_undegraded":"2024-09-21T02:03:12.446014+0000","last_fullsized":"2024-09-21T02:03:12.446014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:23.573018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.264521+0000","last_change":"2024-09-21T02:02:30.972087+0000","last_active":"2024-09-21T02:04:11.264521+0000","last_peered":"2024-09-21T02:04:11.264521+0000","last_clean":"2024-09-21T02:04:11.264521+0000","last_became_active":"2024-09-21T02:02:30.971649+0000","last_became_peered":"2024-09-21T02:02:30.971649+0000","last_unstale":"2024-09-21T02:04:11.264521+0000","last_undegraded":"2024-09-21T02:04:11.264521+0000","last_fullsized":"2024-09-21T02:04:11.264521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:14.681675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.267355+0000","last_change":"2024-09-21T02:03:12.642935+0000","last_active":"2024-09-21T02:04:11.267355+0000","last_peered":"2024-09-21T02:04:11.267355+0000","last_clean":"2024-09-21T02:04:11.267355+0000","last_became_active":"2024-09-21T02:03:12.642520+0000","last_became_peered":"2024-09-21T02:03:12.642520+0000","last_unstale":"2024-09-21T02:04:11.267355+0000","last_undegraded":"2024-09-21T02:04:11.267355+0000","last_fullsized":"2024-09-21T02:04:11.267355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:32.147679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424736+0000","last_change":"2024-09-21T02:02:28.953449+0000","last_active":"2024-09-21T02:03:12.424736+0000","last_peered":"2024-09-21T02:03:12.424736+0000","last_clean":"2024-09-21T02:03:12.424736+0000","last_became_active":"2024-09-21T02:02:28.953019+0000","last_became_peered":"2024-09-21T02:02:28.953019+0000","last_unstale":"2024-09-21T02:03:12.424736+0000","last_undegraded":"2024-09-21T02:03:12.424736+0000","last_fullsized":"2024-09-21T02:03:12.424736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:31.652839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.272647+0000","last_change":"2024-09-21T02:02:30.981820+0000","last_active":"2024-09-21T02:04:11.272647+0000","last_peered":"2024-09-21T02:04:11.272647+0000","last_clean":"2024-09-21T02:04:11.272647+0000","last_became_active":"2024-09-21T02:02:30.981456+0000","last_became_peered":"2024-09-21T02:02:30.981456+0000","last_unstale":"2024-09-21T02:04:11.272647+0000","last_undegraded":"2024-09-21T02:04:11.272647+0000","last_fullsized":"2024-09-21T02:04:11.272647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:46.342845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"77'13","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.442095+0000","last_change":"2024-09-21T02:03:12.441361+0000","last_active":"2024-09-21T02:03:12.442095+0000","last_peered":"2024-09-21T02:03:12.442095+0000","last_clean":"2024-09-21T02:03:12.442095+0000","last_became_active":"2024-09-21T02:03:12.428877+0000","last_became_peered":"2024-09-21T02:03:12.428877+0000","last_unstale":"2024-09-21T02:03:12.442095+0000","last_undegraded":"2024-09-21T02:03:12.442095+0000","last_fullsized":"2024-09-21T02:03:12.442095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:43.464134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.271360+0000","last_change":"2024-09-21T02:02:30.976886+0000","last_active":"2024-09-21T02:04:11.271360+0000","last_peered":"2024-09-21T02:04:11.271360+0000","last_clean":"2024-09-21T02:04:11.271360+0000","last_became_active":"2024-09-21T02:02:30.976470+0000","last_became_peered":"2024-09-21T02:02:30.976470+0000","last_unstale":"2024-09-21T02:04:11.271360+0000","last_undegraded":"2024-09-21T02:04:11.271360+0000","last_fullsized":"2024-09-21T02:04:11.271360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:14.525470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:03:12.424866+0000","last_change":"2024-09-21T02:02:28.975243+0000","last_active":"2024-09-21T02:03:12.424866+0000","last_peered":"2024-09-21T02:03:12.424866+0000","last_clean":"2024-09-21T02:03:12.424866+0000","last_became_active":"2024-09-21T02:02:28.974775+0000","last_became_peered":"2024-09-21T02:02:28.974775+0000","last_unstale":"2024-09-21T02:03:12.424866+0000","last_undegraded":"2024-09-21T02:03:12.424866+0000","last_fullsized":"2024-09-21T02:03:12.424866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:20.452735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:11.268705+0000","last_change":"2024-09-21T02:02:30.982877+0000","last_active":"2024-09-21T02:04:11.268705+0000","last_peered":"2024-09-21T02:04:11.268705+0000","last_clean":"2024-09-21T02:04:11.268705+0000","last_became_active":"2024-09-21T02:02:30.982548+0000","last_became_peered":"2024-09-21T02:02:30.982548+0000","last_unstale":"2024-09-21T02:04:11.268705+0000","last_undegraded":"2024-09-21T02:04:11.268705+0000","last_fullsized":"2024-09-21T02:04:11.268705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:29.945295+0000","last_clean_scrub_stamp":"2024-09-21T02:02:29.945295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:14.103234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:04:21.458260+0000","last_change":"2024-09-21T02:02:28.956874+0000","last_active":"2024-09-21T02:04:21.458260+0000","last_peered":"2024-09-21T02:04:21.458260+0000","last_clean":"2024-09-21T02:04:21.458260+0000","last_became_active":"2024-09-21T02:02:28.956488+0000","last_became_peered":"2024-09-21T02:02:28.956488+0000","last_unstale":"2024-09-21T02:04:21.458260+0000","last_undegraded":"2024-09-21T02:04:21.458260+0000","last_fullsized":"2024-09-21T02:04:21.458260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:02:27.930324+0000","last_clean_scrub_stamp":"2024-09-21T02:02:27.930324+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:25.227623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":83514,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1078,"num_read_kb":630,"num_write":275,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":68,"num_keys_recovered":20,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":250542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":406987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1564672,"data_stored":1072213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":406317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":406695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1470464,"data_stored":1001117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":406874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36792,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708360,"statfs":{"total":95995035648,"available":95957360640,"internally_reserved":0,"allocated":1482752,"data_stored":999423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018952,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":406427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":405859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":482819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":933888,"data_stored":482052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":407905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":76857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":76533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:25.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:04:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:24 smithi099 ceph-mon[34404]: pgmap v343: 97 pgs: 97 active+clean; 651 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-21T02:04:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:24 smithi099 ceph-mon[34404]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-21T02:04:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:25.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 -- 172.21.15.92:0/710113787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a81094a0 msgr2=0x7f35a81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 --2- 172.21.15.92:0/710113787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a81094a0 0x7f35a81098b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c02a410 comp rx=0 tx=0).stop 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 -- 172.21.15.92:0/710113787 shutdown_connections 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 --2- 172.21.15.92:0/710113787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35a8109df0 0x7f35a8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 --2- 172.21.15.92:0/710113787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a81094a0 0x7f35a81098b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 --2- 172.21.15.92:0/710113787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a8103b60 0x7f35a8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 -- 172.21.15.92:0/710113787 >> 172.21.15.92:0/710113787 conn(0x7f35a807b850 msgr2=0x7f35a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 -- 172.21.15.92:0/710113787 shutdown_connections 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 -- 172.21.15.92:0/710113787 wait complete. 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.904+0000 7f35adf1b640 1 Processor -- start 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 -- start start 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 0x7f35a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a81094a0 0x7f35a8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35a8109df0 0x7f35a8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8113ad0 con 0x7f35a8103b60 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8113950 con 0x7f35a81094a0 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35adf1b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8113c50 con 0x7f35a8109df0 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 0x7f35a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 0x7f35a8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46558/0 (socket says 172.21.15.92:46558) 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 -- 172.21.15.92:0/141014251 learned_addr learned my addr 172.21.15.92:0/141014251 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a6ffd640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a81094a0 0x7f35a8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a7fff640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35a8109df0 0x7f35a8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 -- 172.21.15.92:0/141014251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35a8109df0 msgr2=0x7f35a8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35a8109df0 0x7f35a8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 -- 172.21.15.92:0/141014251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a81094a0 msgr2=0x7f35a8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a81094a0 0x7f35a8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.905+0000 7f35a77fe640 1 -- 172.21.15.92:0/141014251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81a8820 con 0x7f35a8103b60 2024-09-21T02:04:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.906+0000 7f35a77fe640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 0x7f35a8075710 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f359800eab0 tx=0x7f359800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.906+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359800ce40 con 0x7f35a8103b60 2024-09-21T02:04:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.906+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3598011920 con 0x7f35a8103b60 2024-09-21T02:04:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.906+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598010a30 con 0x7f35a8103b60 2024-09-21T02:04:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.906+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a8b10 con 0x7f35a8103b60 2024-09-21T02:04:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.906+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a9020 con 0x7f35a8103b60 2024-09-21T02:04:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.907+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a8103f70 con 0x7f35a8103b60 2024-09-21T02:04:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.908+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f3598010bd0 con 0x7f35a8103b60 2024-09-21T02:04:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.909+0000 7f35a4ff9640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f35900b65a0 0x7f35900b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.909+0000 7f35a6ffd640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f35900b65a0 0x7f35900b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.909+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f35980e7ac0 con 0x7f35a8103b60 2024-09-21T02:04:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.910+0000 7f35a6ffd640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f35900b65a0 0x7f35900b8ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f359c009230 tx=0x7f359c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:25.913+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3598016030 con 0x7f35a8103b60 2024-09-21T02:04:26.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.126+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35a8076a10 con 0x7f35a8103b60 2024-09-21T02:04:26.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.126+0000 7f35a4ff9640 1 -- 172.21.15.92:0/141014251 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f35a8076a10 con 0x7f35a8103b60 2024-09-21T02:04:26.127 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:26.127 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:04:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:04:26.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f35900b65a0 msgr2=0x7f35900b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f35900b65a0 0x7f35900b8ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f359c009230 tx=0x7f359c031400 comp rx=0 tx=0).stop 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 msgr2=0x7f35a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 0x7f35a8075710 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f359800eab0 tx=0x7f359800ef70 comp rx=0 tx=0).stop 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 shutdown_connections 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f35900b65a0 0x7f35900b8ab0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35a8109df0 0x7f35a8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a81094a0 0x7f35a8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 --2- 172.21.15.92:0/141014251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35a8103b60 0x7f35a8075710 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 >> 172.21.15.92:0/141014251 conn(0x7f35a807b850 msgr2=0x7f35a81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 shutdown_connections 2024-09-21T02:04:26.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:26.129+0000 7f35adf1b640 1 -- 172.21.15.92:0/141014251 wait complete. 2024-09-21T02:04:26.139 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:25 smithi092 ceph-mon[31190]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:26.140 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:25 smithi092 ceph-mon[31190]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:25 smithi136 ceph-mon[34754]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:26.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:25 smithi136 ceph-mon[34754]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:25 smithi099 ceph-mon[34404]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:26.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:25 smithi099 ceph-mon[34404]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:27.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:04:27.259 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:26 smithi136 ceph-mon[34754]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-21T02:04:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:26 smithi136 ceph-mon[34754]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:26 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/141014251' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:26 smithi092 ceph-mon[31190]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-21T02:04:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:26 smithi092 ceph-mon[31190]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:26 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/141014251' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:26 smithi099 ceph-mon[34404]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-21T02:04:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:26 smithi099 ceph-mon[34404]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:04:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:26 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/141014251' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:04:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/1783040216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 msgr2=0x7f6c74102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/1783040216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c74102c30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009900 tx=0x7f6c7002a410 comp rx=0 tx=0).stop 2024-09-21T02:04:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/1783040216 shutdown_connections 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/1783040216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74107c30 0x7f6c741080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/1783040216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c74103170 0x7f6c741076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/1783040216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c74102c30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/1783040216 >> 172.21.15.92:0/1783040216 conn(0x7f6c74079760 msgr2=0x7f6c74079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/1783040216 shutdown_connections 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.719+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/1783040216 wait complete. 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 Processor -- start 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 -- start start 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c741a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c74103170 0x7f6c741a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74107c30 0x7f6c741a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7411a0d0 con 0x7f6c74102820 2024-09-21T02:04:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74119f50 con 0x7f6c74103170 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.720+0000 7f6c7bd6a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7411a250 con 0x7f6c74107c30 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c741a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c741a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46574/0 (socket says 172.21.15.92:46574) 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 -- 172.21.15.92:0/2928630199 learned_addr learned my addr 172.21.15.92:0/2928630199 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c7a307640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74107c30 0x7f6c741a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79305640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c74103170 0x7f6c741a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 -- 172.21.15.92:0/2928630199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74107c30 msgr2=0x7f6c741a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74107c30 0x7f6c741a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 -- 172.21.15.92:0/2928630199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c74103170 msgr2=0x7f6c741a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c74103170 0x7f6c741a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 -- 172.21.15.92:0/2928630199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a6a90 con 0x7f6c74102820 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c79b06640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c741a0b20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6c70032fd0 tx=0x7f6c70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7003f030 con 0x7f6c74102820 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70004490 con 0x7f6c74102820 2024-09-21T02:04:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741a6d20 con 0x7f6c74102820 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70035070 con 0x7f6c74102820 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.721+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741ad4e0 con 0x7f6c74102820 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.722+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7406c1f0 con 0x7f6c74102820 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.724+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f6c70042030 con 0x7f6c74102820 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.725+0000 7f6c6affd640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6c600b65a0 0x7f6c600b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.725+0000 7f6c79305640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6c600b65a0 0x7f6c600b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.725+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6c70107d00 con 0x7f6c74102820 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.725+0000 7f6c79305640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6c600b65a0 0x7f6c600b8ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c741024f0 tx=0x7f6c6400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.728+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6c700d27f0 con 0x7f6c74102820 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.905+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c741ad6b0 con 0x7f6c74102820 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.907+0000 7f6c6affd640 1 -- 172.21.15.92:0/2928630199 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22797 (secure 0 0 0) 0x7f6c700d7730 con 0x7f6c74102820 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:03:12.414418+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6c600b65a0 msgr2=0x7f6c600b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6c600b65a0 0x7f6c600b8ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c741024f0 tx=0x7f6c6400a2d0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 msgr2=0x7f6c741a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c741a0b20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6c70032fd0 tx=0x7f6c70004270 comp rx=0 tx=0).stop 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 shutdown_connections 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6c600b65a0 0x7f6c600b8ab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74107c30 0x7f6c741a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c74103170 0x7f6c741a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 --2- 172.21.15.92:0/2928630199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74102820 0x7f6c741a0b20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 >> 172.21.15.92:0/2928630199 conn(0x7f6c74079760 msgr2=0x7f6c74105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 shutdown_connections 2024-09-21T02:04:27.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:27.909+0000 7f6c7bd6a640 1 -- 172.21.15.92:0/2928630199 wait complete. 2024-09-21T02:04:28.708 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T02:04:28.708 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T02:04:28.718 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:04:28.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T02:04:28.719 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T02:04:28.719 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:04:28.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:04:28.721 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T02:04:28.733 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:04:28.733 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T02:04:28.733 INFO:tasks.workunit:timeout=3h 2024-09-21T02:04:28.733 INFO:tasks.workunit:cleanup=True 2024-09-21T02:04:28.734 DEBUG:teuthology.orchestra.run.smithi092:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-21 02:02:49.851882793 +0000 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-21 02:02:49.851882793 +0000 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-21 02:04:20.058977114 +0000 2024-09-21T02:04:28.750 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-21 02:02:49.851882793 +0000 2024-09-21T02:04:28.750 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:28.750 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T02:04:28.824 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:04:28.879 INFO:tasks.workunit.client.0.smithi092.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T02:04:28.958 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:28 smithi092 ceph-mon[31190]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:04:28.959 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:28 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2928630199' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:28.979 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:28 smithi136 ceph-mon[34754]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:04:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:28 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2928630199' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:28 smithi099 ceph-mon[34404]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:04:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:28 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2928630199' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:04:29.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/2140142623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c48109e10 msgr2=0x7f5c48110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/2140142623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c48109e10 0x7f5c48110640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009ca0 tx=0x7f5c4002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/2140142623 shutdown_connections 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/2140142623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c48109e10 0x7f5c48110640 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/2140142623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c481094c0 0x7f5c481098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/2140142623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 0x7f5c48103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/2140142623 >> 172.21.15.92:0/2140142623 conn(0x7f5c480ff610 msgr2=0x7f5c48101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/2140142623 shutdown_connections 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.500+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/2140142623 wait complete. 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 Processor -- start 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 -- start start 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 0x7f5c4819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 0x7f5c4819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c48109e10 0x7f5c4819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48113ad0 con 0x7f5c481094c0 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48113950 con 0x7f5c48103b80 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4fcb5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48113c50 con 0x7f5c48109e10 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 0x7f5c4819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 0x7f5c4819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46596/0 (socket says 172.21.15.92:46596) 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.501+0000 7f5c4d250640 1 -- 172.21.15.92:0/3520123647 learned_addr learned my addr 172.21.15.92:0/3520123647 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4da51640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 0x7f5c4819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4e252640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c48109e10 0x7f5c4819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4d250640 1 -- 172.21.15.92:0/3520123647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c48109e10 msgr2=0x7f5c4819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4d250640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c48109e10 0x7f5c4819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4d250640 1 -- 172.21.15.92:0/3520123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 msgr2=0x7f5c4819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4d250640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 0x7f5c4819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4d250640 1 -- 172.21.15.92:0/3520123647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c481a0450 con 0x7f5c481094c0 2024-09-21T02:04:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4da51640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 0x7f5c4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4e252640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c48109e10 0x7f5c4819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4d250640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 0x7f5c4819aa80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c970 tx=0x7f5c3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c3affd640 1 -- 172.21.15.92:0/3520123647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c007bb0 con 0x7f5c481094c0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c481a0740 con 0x7f5c481094c0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c3affd640 1 -- 172.21.15.92:0/3520123647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c007d50 con 0x7f5c481094c0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481a6ea0 con 0x7f5c481094c0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.502+0000 7f5c3affd640 1 -- 172.21.15.92:0/3520123647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c004e10 con 0x7f5c481094c0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 Processor -- start 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 -- start start 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bb9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87e9490 0x5573e8bb9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c3affd640 1 -- 172.21.15.92:0/3520123647 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f5c3c004fb0 con 0x7f5c481094c0 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bb9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573e8b2fe60 con 0x5573e87f1670 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5573e8b1f290 con 0x5573e87e9490 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c56229740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5573e8b230a0 con 0x5573e8812350 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bb9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bb9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46608/0 (socket says 172.21.15.92:46608) 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c4d250640 1 -- 172.21.15.92:0/3100335418 learned_addr learned my addr 172.21.15.92:0/3100335418 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c4e252640 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bb9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.505+0000 7f5c4da51640 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87e9490 0x5573e8bb9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4d250640 1 -- 172.21.15.92:0/3100335418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 msgr2=0x5573e8bb9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4d250640 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bb9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4d250640 1 -- 172.21.15.92:0/3100335418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87e9490 msgr2=0x5573e8bb9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4d250640 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87e9490 0x5573e8bb9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4d250640 1 -- 172.21.15.92:0/3100335418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573e887d930 con 0x5573e87f1670 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4d250640 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bb9eb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0e4b10 tx=0x7f5c3c0efa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5130e0637b27a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c3affd640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c340b65a0 0x7f5c340b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c4e252640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c340b65a0 0x7f5c340b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c3affd640 1 -- 172.21.15.92:0/3520123647 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5c3c0e7240 con 0x7f5c481094c0 2024-09-21T02:04:29.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.506+0000 7f5c33fff640 1 -- 172.21.15.92:0/3100335418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0fb070 con 0x5573e87f1670 2024-09-21T02:04:29.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c33fff640 1 -- 172.21.15.92:0/3100335418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c0acd60 con 0x5573e87f1670 2024-09-21T02:04:29.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c33fff640 1 -- 172.21.15.92:0/3100335418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0a90f0 con 0x5573e87f1670 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c56229740 1 -- 172.21.15.92:0/3100335418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 msgr2=0x5573e8bb9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c4e252640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c340b65a0 0x7f5c340b8ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c40004af0 tx=0x7f5c40005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c56229740 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bb9eb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0e4b10 tx=0x7f5c3c0efa10 comp rx=0 tx=0).stop 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c56229740 1 -- 172.21.15.92:0/3100335418 shutdown_connections 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c56229740 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bb9eb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c56229740 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87e9490 0x5573e8bb9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.507+0000 7f5c56229740 1 --2- 172.21.15.92:0/3100335418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bb9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.508+0000 7f5c56229740 1 -- 172.21.15.92:0/3100335418 >> 172.21.15.92:0/3100335418 conn(0x5573e8806740 msgr2=0x5573e8bb7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.508+0000 7f5c56229740 1 -- 172.21.15.92:0/3100335418 shutdown_connections 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.508+0000 7f5c56229740 1 -- 172.21.15.92:0/3100335418 wait complete. 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.508+0000 7f5c56229740 1 -- start start 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.508+0000 7f5c56229740 1 Processor -- start 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bc5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bc5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e8812350 0x5573e8bd0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573e8980890 con 0x5573e87f1670 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5573e8ab9d40 con 0x5573e8812350 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5573e8bc7c10 con 0x5573e87e9490 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c56229740 10 client.0 _ll_register_callbacks cb 0x5573e8b1dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bc5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bc5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46614/0 (socket says 172.21.15.92:46614) 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 -- 172.21.15.92:0/255345424 learned_addr learned my addr 172.21.15.92:0/255345424 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4e252640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e8812350 0x5573e8bd0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4da51640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bc5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 -- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 msgr2=0x5573e8bc5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bc5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 -- 172.21.15.92:0/255345424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e8812350 msgr2=0x5573e8bd0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e8812350 0x5573e8bd0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 -- 172.21.15.92:0/255345424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573e8bd1520 con 0x5573e87f1670 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c4d250640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bc5540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0e4c50 tx=0x7f5c3c0e4b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc06c2cc4fa6c944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.509+0000 7f5c317fa640 1 -- 172.21.15.92:0/255345424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c01d040 con 0x5573e87f1670 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c317fa640 1 -- 172.21.15.92:0/255345424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5c3c0b69e0 con 0x5573e87f1670 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3c0acd60 con 0x5573e87f1670 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c317fa640 1 -- 172.21.15.92:0/255345424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c3c0ac9c0 con 0x5573e87f1670 2024-09-21T02:04:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 10 client.15483 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573e8bd3e30 con 0x5573e87f1670 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c317fa640 1 -- 172.21.15.92:0/255345424 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c0acd60 con 0x5573e87f1670 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c317fa640 1 -- 172.21.15.92:0/255345424 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c3c0d0bd0 con 0x5573e87f1670 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 10 client.15483 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 20 client.15483 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87d0cd0 0x5573e8bd7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 4 client.15483 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573e8b31db0 con 0x5573e87d0cd0 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.510+0000 7f5c4da51640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87d0cd0 0x5573e8bd7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.511+0000 7f5c317fa640 10 client.15483 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.511+0000 7f5c4da51640 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87d0cd0 0x5573e8bd7190 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4934346ce36826 server_cookie=3de1457929ac0465 in_seq=0 out_seq=0 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c317fa640 1 -- 172.21.15.92:0/255345424 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573e8b31db0 con 0x5573e87d0cd0 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c317fa640 10 client.15483 handle_command_reply: tid=0 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 client.15483 shutdown 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 msgr2=0x5573e8bc5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bc5540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0e4c50 tx=0x7f5c3c0e4b40 comp rx=0 tx=0).stop 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 shutdown_connections 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87d0cd0 0x5573e8bd7190 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e8812350 0x5573e8bd0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87f1670 0x5573e8bc5540 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 --2- 172.21.15.92:0/255345424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bc5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c317fa640 0 client.15483 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 >> 172.21.15.92:0/255345424 conn(0x5573e8806740 msgr2=0x5573e8bbc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 shutdown_connections 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 1 -- 172.21.15.92:0/255345424 wait complete. 2024-09-21T02:04:29.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.512+0000 7f5c56229740 20 client.15483 trim_cache size 0 max 16384 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.529+0000 7f5c56229740 1 Processor -- start 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.529+0000 7f5c56229740 1 -- start start 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.529+0000 7f5c56229740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 0x5573e8bd0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c56229740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c56229740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bda6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c56229740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481af500 con 0x5573e87e9490 2024-09-21T02:04:29.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c56229740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481b1040 con 0x5573e87f1670 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c56229740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481afbd0 con 0x5573e8812350 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 0x5573e8bd0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 0x5573e8bd0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46626/0 (socket says 172.21.15.92:46626) 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 -- 172.21.15.92:0/130757768 learned_addr learned my addr 172.21.15.92:0/130757768 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4d250640 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 -- 172.21.15.92:0/130757768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 msgr2=0x5573e8bda6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bda6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 -- 172.21.15.92:0/130757768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 msgr2=0x5573e8bda1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 -- 172.21.15.92:0/130757768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573e8bbb720 con 0x5573e87e9490 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c4da51640 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 0x5573e8bd0ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004700 tx=0x7f5c44014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb76a0fac4ae7b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c337fe640 1 -- 172.21.15.92:0/130757768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44014190 con 0x5573e87e9490 2024-09-21T02:04:29.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c337fe640 1 -- 172.21.15.92:0/130757768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c440054d0 con 0x5573e87e9490 2024-09-21T02:04:29.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.530+0000 7f5c337fe640 1 -- 172.21.15.92:0/130757768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44012450 con 0x5573e87e9490 2024-09-21T02:04:29.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 -- 172.21.15.92:0/130757768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 msgr2=0x5573e8bd0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 0x5573e8bd0ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004700 tx=0x7f5c44014900 comp rx=0 tx=0).stop 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 -- 172.21.15.92:0/130757768 shutdown_connections 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e8812350 0x5573e8bda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 --2- 172.21.15.92:0/130757768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e87e9490 0x5573e8bd0ea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 -- 172.21.15.92:0/130757768 >> 172.21.15.92:0/130757768 conn(0x5573e8806740 msgr2=0x5573e8bc3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 -- 172.21.15.92:0/130757768 shutdown_connections 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.531+0000 7f5c56229740 1 -- 172.21.15.92:0/130757768 wait complete. 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 -- start start 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 Processor -- start 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bda1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 0x5573e8bd5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481b0970 con 0x5573e8812350 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481b02a0 con 0x5573e87f1670 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481ad4e0 con 0x5573e87e9490 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.532+0000 7f5c56229740 10 client.0 _ll_register_callbacks cb 0x7f5c3c0f9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4e252640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 0x5573e8bd5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:34614/0 (socket says 172.21.15.92:34614) 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4da51640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bda1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4e252640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 0x5573e8bd5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46640/0 (socket says 172.21.15.92:46640) 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 -- 172.21.15.92:0/1853472070 learned_addr learned my addr 172.21.15.92:0/1853472070 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 -- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 msgr2=0x5573e8bda1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bda1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 -- 172.21.15.92:0/1853472070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 msgr2=0x5573e8bd5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:29.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 0x5573e8bd5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 -- 172.21.15.92:0/1853472070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573e8bdae20 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4e252640 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 0x5573e8bd5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c4d250640 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0e4b10 tx=0x7f5c3c0ef4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74a01bdba4c38a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c317fa640 1 -- 172.21.15.92:0/1853472070 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c01d040 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c440054d0 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.533+0000 7f5c317fa640 1 -- 172.21.15.92:0/1853472070 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5c3c0ea8c0 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c317fa640 1 -- 172.21.15.92:0/1853472070 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c3c0acba0 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 10 client.25040 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573e8c461f0 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c317fa640 1 -- 172.21.15.92:0/1853472070 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c440054d0 con 0x5573e87f1670 2024-09-21T02:04:29.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c317fa640 1 -- 172.21.15.92:0/1853472070 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c3c0b67e0 con 0x5573e87f1670 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 10 client.25040 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 10 client.25040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 20 client.25040 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87a6080 0x5573e8c485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 4 client.25040 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573e8b2d860 con 0x5573e87a6080 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.534+0000 7f5c4da51640 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87a6080 0x5573e8c485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.535+0000 7f5c4da51640 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87a6080 0x5573e8c485c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35657368599e4fe server_cookie=57f572910694b1bc in_seq=0 out_seq=0 2024-09-21T02:04:29.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.535+0000 7f5c317fa640 10 client.25040 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c317fa640 1 -- 172.21.15.92:0/1853472070 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573e8b2d860 con 0x5573e87a6080 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c317fa640 10 client.25040 handle_command_reply: tid=0 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 client.25040 shutdown 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 msgr2=0x5573e8bda710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0e4b10 tx=0x7f5c3c0ef4b0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 shutdown_connections 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5573e87a6080 0x5573e8c485c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573e8812350 0x5573e8bd5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573e87f1670 0x5573e8bda710 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c317fa640 0 client.25040 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 --2- 172.21.15.92:0/1853472070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573e87e9490 0x5573e8bda1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 >> 172.21.15.92:0/1853472070 conn(0x5573e8806740 msgr2=0x5573e8bbc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 shutdown_connections 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 1 -- 172.21.15.92:0/1853472070 wait complete. 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.537+0000 7f5c56229740 20 client.25040 trim_cache size 0 max 16384 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "80a7a045-f3bc-45a8-ac20-fc7e8853b101", 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:04:29.542 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:29.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c340b65a0 msgr2=0x7f5c340b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c340b65a0 0x7f5c340b8ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c40004af0 tx=0x7f5c40005d80 comp rx=0 tx=0).stop 2024-09-21T02:04:29.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 msgr2=0x7f5c4819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:29.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 0x7f5c4819aa80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c970 tx=0x7f5c3c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:04:29.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 shutdown_connections 2024-09-21T02:04:29.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c340b65a0 0x7f5c340b8ab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c48109e10 0x7f5c4819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c481094c0 0x7f5c4819aa80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 --2- 172.21.15.92:0/3520123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c48103b80 0x7f5c4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:29.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 >> 172.21.15.92:0/3520123647 conn(0x7f5c480ff610 msgr2=0x7f5c48106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:29.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 shutdown_connections 2024-09-21T02:04:29.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:29.538+0000 7f5c4fcb5640 1 -- 172.21.15.92:0/3520123647 wait complete. 2024-09-21T02:04:30.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:04:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 -- 172.21.15.92:0/1774086316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca8109df0 msgr2=0x7feca8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 --2- 172.21.15.92:0/1774086316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca8109df0 0x7feca8110620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feca0009900 tx=0x7feca002a410 comp rx=0 tx=0).stop 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 -- 172.21.15.92:0/1774086316 shutdown_connections 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 --2- 172.21.15.92:0/1774086316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca8109df0 0x7feca8110620 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 --2- 172.21.15.92:0/1774086316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca81094a0 0x7feca81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 --2- 172.21.15.92:0/1774086316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8103b60 0x7feca8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 -- 172.21.15.92:0/1774086316 >> 172.21.15.92:0/1774086316 conn(0x7feca807b850 msgr2=0x7feca807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 -- 172.21.15.92:0/1774086316 shutdown_connections 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.954+0000 7fecaed75640 1 -- 172.21.15.92:0/1774086316 wait complete. 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 Processor -- start 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 -- start start 2024-09-21T02:04:30.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca8103b60 0x7feca819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 0x7feca819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8109df0 0x7feca819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feca8113ab0 con 0x7feca81094a0 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feca8113930 con 0x7feca8103b60 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecaed75640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feca8113c30 con 0x7feca8109df0 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 0x7feca819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 0x7feca819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46658/0 (socket says 172.21.15.92:46658) 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 -- 172.21.15.92:0/1293926832 learned_addr learned my addr 172.21.15.92:0/1293926832 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecacb11640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca8103b60 0x7feca819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecad312640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8109df0 0x7feca819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 -- 172.21.15.92:0/1293926832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8109df0 msgr2=0x7feca819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8109df0 0x7feca819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 -- 172.21.15.92:0/1293926832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca8103b60 msgr2=0x7feca819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca8103b60 0x7feca819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7feca7fff640 1 -- 172.21.15.92:0/1293926832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca81a02f0 con 0x7feca81094a0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecad312640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8109df0 0x7feca819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.955+0000 7fecacb11640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca8103b60 0x7feca819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.956+0000 7feca7fff640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 0x7feca819a9e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c8c0 tx=0x7fec9000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.956+0000 7feca5ffb640 1 -- 172.21.15.92:0/1293926832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90007bb0 con 0x7feca81094a0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.956+0000 7feca5ffb640 1 -- 172.21.15.92:0/1293926832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90007d50 con 0x7feca81094a0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.956+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca81a05e0 con 0x7feca81094a0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.956+0000 7feca5ffb640 1 -- 172.21.15.92:0/1293926832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900187c0 con 0x7feca81094a0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.956+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca81a6e00 con 0x7feca81094a0 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.958+0000 7fecb52e9740 1 Processor -- start 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 -- start start 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 0x556554d00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7feca5ffb640 1 -- 172.21.15.92:0/1293926832 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fec9000fa20 con 0x7feca81094a0 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565549301b0 0x556554d00b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556554c76e50 con 0x556554959070 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556554c66280 con 0x5565549301b0 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecb52e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556554c6a090 con 0x556554938390 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 0x556554d00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 0x556554d00650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46666/0 (socket says 172.21.15.92:46666) 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 -- 172.21.15.92:0/3756646982 learned_addr learned my addr 172.21.15.92:0/3756646982 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7feca7fff640 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecacb11640 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565549301b0 0x556554d00b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 -- 172.21.15.92:0/3756646982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 msgr2=0x556554d010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 -- 172.21.15.92:0/3756646982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565549301b0 msgr2=0x556554d00b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565549301b0 0x556554d00b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecad312640 1 -- 172.21.15.92:0/3756646982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565549c4650 con 0x556554959070 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.959+0000 7fecacb11640 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565549301b0 0x556554d00b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7fecad312640 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 0x556554d00650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feca0031440 tx=0x7feca0032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b096a6e0a2f3e0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7feca5ffb640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec940b6650 0x7fec940b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7fec9effd640 1 -- 172.21.15.92:0/3756646982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca003f070 con 0x556554959070 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7fec9effd640 1 -- 172.21.15.92:0/3756646982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca00316a0 con 0x556554959070 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7fec9effd640 1 -- 172.21.15.92:0/3756646982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0035070 con 0x556554959070 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7fecacb11640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec940b6650 0x7fec940b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.960+0000 7feca5ffb640 1 -- 172.21.15.92:0/1293926832 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fec900e75b0 con 0x7feca81094a0 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 -- 172.21.15.92:0/3756646982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 msgr2=0x556554d00650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 0x556554d00650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feca0031440 tx=0x7feca0032e70 comp rx=0 tx=0).stop 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecacb11640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec940b6650 0x7fec940b8b60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x556554cff2d0 tx=0x7fec98008590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:30.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 -- 172.21.15.92:0/3756646982 shutdown_connections 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565549301b0 0x556554d00b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 --2- 172.21.15.92:0/3756646982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554959070 0x556554d00650 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 -- 172.21.15.92:0/3756646982 >> 172.21.15.92:0/3756646982 conn(0x55655494d460 msgr2=0x556554cfe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 -- 172.21.15.92:0/3756646982 shutdown_connections 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 -- 172.21.15.92:0/3756646982 wait complete. 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 -- start start 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.961+0000 7fecb52e9740 1 Processor -- start 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d0bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d0c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556554ac7700 con 0x556554938390 2024-09-21T02:04:30.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556554c01110 con 0x556554959070 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556554d0edd0 con 0x5565549301b0 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecb52e9740 10 client.0 _ll_register_callbacks cb 0x556554c1a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d0c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d0c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46670/0 (socket says 172.21.15.92:46670) 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 -- 172.21.15.92:0/4118598620 learned_addr learned my addr 172.21.15.92:0/4118598620 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecacb11640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d0bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecad312640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 -- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 msgr2=0x556554d0bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d0bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 -- 172.21.15.92:0/4118598620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 msgr2=0x556554d180f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d180f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 -- 172.21.15.92:0/4118598620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556554c2ec40 con 0x556554938390 2024-09-21T02:04:30.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fecad312640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7feca7fff640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d0c2a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec900a9b90 tx=0x7fec9001f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f697f0950815997 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.962+0000 7fec87fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fec87fff640 1 -- 172.21.15.92:0/4118598620 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900acf10 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fec87fff640 1 -- 172.21.15.92:0/4118598620 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fec900b6040 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca00316a0 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fec87fff640 1 -- 172.21.15.92:0/4118598620 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec900acd30 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 10 client.15501 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556554d1abc0 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fec87fff640 1 -- 172.21.15.92:0/4118598620 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feca00316a0 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fec87fff640 1 -- 172.21.15.92:0/4118598620 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fec900e70d0 con 0x556554938390 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 10 client.15501 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:30.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 20 client.15501 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565549179f0 0x556554d1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 4 client.15501 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.963+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556554c78da0 con 0x5565549179f0 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.964+0000 7fecacb11640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565549179f0 0x556554d1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.964+0000 7fecacb11640 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565549179f0 0x556554d1e090 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ca2d7c75b7f3e42 server_cookie=c19430344983e215 in_seq=0 out_seq=0 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.964+0000 7fec87fff640 10 client.15501 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fec87fff640 1 -- 172.21.15.92:0/4118598620 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556554c78da0 con 0x5565549179f0 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fec87fff640 10 client.15501 handle_command_reply: tid=0 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 client.15501 shutdown 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 msgr2=0x556554d0c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d0c2a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec900a9b90 tx=0x7fec9001f5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 shutdown_connections 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565549179f0 0x556554d1e090 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fec87fff640 0 client.15501 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d0c2a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4118598620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d0bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 >> 172.21.15.92:0/4118598620 conn(0x55655494d460 msgr2=0x556554d03830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:30.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 shutdown_connections 2024-09-21T02:04:30.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.965+0000 7fecb52e9740 1 -- 172.21.15.92:0/4118598620 wait complete. 2024-09-21T02:04:30.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.966+0000 7fecb52e9740 20 client.15501 trim_cache size 0 max 16384 2024-09-21T02:04:30.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 Processor -- start 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 -- start start 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 0x556554d01d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d17ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d08f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feca81af290 con 0x5565549301b0 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b0dd0 con 0x556554959070 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecb52e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feca81af960 con 0x556554938390 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecacb11640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 0x556554d01d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecacb11640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 0x556554d01d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46680/0 (socket says 172.21.15.92:46680) 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecacb11640 1 -- 172.21.15.92:0/4225994346 learned_addr learned my addr 172.21.15.92:0/4225994346 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7feca7fff640 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d17ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.986+0000 7fecad312640 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d08f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fecacb11640 1 -- 172.21.15.92:0/4225994346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 msgr2=0x556554d17ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fecacb11640 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d17ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fecacb11640 1 -- 172.21.15.92:0/4225994346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 msgr2=0x556554d08f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fecacb11640 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d08f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fecacb11640 1 -- 172.21.15.92:0/4225994346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556554d0e340 con 0x5565549301b0 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fecacb11640 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 0x556554d01d90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fec98005fd0 tx=0x7fec98012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd00e5a837cabd55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fec9e7fc640 1 -- 172.21.15.92:0/4225994346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98010a00 con 0x5565549301b0 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fec9e7fc640 1 -- 172.21.15.92:0/4225994346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec98010ba0 con 0x5565549301b0 2024-09-21T02:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.987+0000 7fec9e7fc640 1 -- 172.21.15.92:0/4225994346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9801cea0 con 0x5565549301b0 2024-09-21T02:04:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 -- 172.21.15.92:0/4225994346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 msgr2=0x556554d01d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 0x556554d01d90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fec98005fd0 tx=0x7fec98012000 comp rx=0 tx=0).stop 2024-09-21T02:04:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 -- 172.21.15.92:0/4225994346 shutdown_connections 2024-09-21T02:04:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d08f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556554938390 0x556554d17ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 --2- 172.21.15.92:0/4225994346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565549301b0 0x556554d01d90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 -- 172.21.15.92:0/4225994346 >> 172.21.15.92:0/4225994346 conn(0x55655494d460 msgr2=0x556554d01680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 -- 172.21.15.92:0/4225994346 shutdown_connections 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 -- 172.21.15.92:0/4225994346 wait complete. 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.988+0000 7fecb52e9740 1 -- start start 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 Processor -- start 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d10190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b0700 con 0x556554938390 2024-09-21T02:04:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b0030 con 0x556554959070 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feca81ad270 con 0x5565549301b0 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecb52e9740 10 client.0 _ll_register_callbacks cb 0x7fec9000a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46690/0 (socket says 172.21.15.92:46690) 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 -- 172.21.15.92:0/2393330746 learned_addr learned my addr 172.21.15.92:0/2393330746 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecad312640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7fecacb11640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d10190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 -- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 msgr2=0x556554d10190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d10190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 -- 172.21.15.92:0/2393330746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 msgr2=0x556554d10c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d10c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.989+0000 7feca7fff640 1 -- 172.21.15.92:0/2393330746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556554d87a50 con 0x556554938390 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7feca7fff640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d106d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fec900e7470 tx=0x7fec900e7900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd222fc34a55fa45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fec87fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fec87fff640 1 -- 172.21.15.92:0/2393330746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900acf10 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fec87fff640 1 -- 172.21.15.92:0/2393330746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fec900ac820 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec98010ba0 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fec87fff640 1 -- 172.21.15.92:0/2393330746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec900ad0d0 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 10 client.15507 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556554d88ce0 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fec87fff640 1 -- 172.21.15.92:0/2393330746 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec98010ba0 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fec87fff640 1 -- 172.21.15.92:0/2393330746 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fec900b3230 con 0x556554938390 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 20 client.15507 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.990+0000 7fecb52e9740 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565548ecda0 0x556554d8b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fecb52e9740 4 client.15507 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:04:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556554c74850 con 0x5565548ecda0 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fecacb11640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565548ecda0 0x556554d8b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fec87fff640 10 client.15507 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fecacb11640 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565548ecda0 0x556554d8b3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a4c40099ef4260 server_cookie=120717c05734b34a in_seq=0 out_seq=0 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fec87fff640 1 -- 172.21.15.92:0/2393330746 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x556554c74850 con 0x5565548ecda0 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fec87fff640 10 client.15507 handle_command_reply: tid=0 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.991+0000 7fecb52e9740 1 client.15507 shutdown 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 msgr2=0x556554d106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d106d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fec900e7470 tx=0x7fec900e7900 comp rx=0 tx=0).stop 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 shutdown_connections 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565548ecda0 0x556554d8b3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fec87fff640 0 client.15507 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556554959070 0x556554d10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556554938390 0x556554d106d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 --2- 172.21.15.92:0/2393330746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565549301b0 0x556554d10190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 >> 172.21.15.92:0/2393330746 conn(0x55655494d460 msgr2=0x556554d01580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 shutdown_connections 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 1 -- 172.21.15.92:0/2393330746 wait complete. 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecb52e9740 20 client.15507 trim_cache size 0 max 16384 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:04:30.998 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "80a7a045-f3bc-45a8-ac20-fc7e8853b101", 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:04:30.999 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:04:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec940b6650 msgr2=0x7fec940b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecaed75640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec940b6650 0x7fec940b8b60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x556554cff2d0 tx=0x7fec98008590 comp rx=0 tx=0).stop 2024-09-21T02:04:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 msgr2=0x7feca819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.992+0000 7fecaed75640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 0x7feca819a9e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c8c0 tx=0x7fec9000cd80 comp rx=0 tx=0).stop 2024-09-21T02:04:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 shutdown_connections 2024-09-21T02:04:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec940b6650 0x7fec940b8b60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feca8109df0 0x7feca819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feca81094a0 0x7feca819a9e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 --2- 172.21.15.92:0/1293926832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feca8103b60 0x7feca819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 >> 172.21.15.92:0/1293926832 conn(0x7feca807b850 msgr2=0x7feca8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 shutdown_connections 2024-09-21T02:04:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:04:30.993+0000 7fecaed75640 1 -- 172.21.15.92:0/1293926832 wait complete. 2024-09-21T02:04:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:30 smithi136 ceph-mon[34754]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T02:04:31.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:30 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:04:31.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:30 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:04:31.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:30 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:04:31.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:30 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:04:31.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:30 smithi092 ceph-mon[31190]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T02:04:31.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:30 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:04:31.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:30 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:04:31.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:30 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:04:31.300 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:30 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:04:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:30 smithi099 ceph-mon[34404]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T02:04:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:30 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:04:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:30 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:04:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:30 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:04:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:30 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:04:31.838 INFO:tasks.cephfs.filesystem:scrub status for tag:80a7a045-f3bc-45a8-ac20-fc7e8853b101 - {'path': '/', 'tag': '80a7a045-f3bc-45a8-ac20-fc7e8853b101', 'options': 'recursive,force'} 2024-09-21T02:04:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:32 smithi136 ceph-mon[34754]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T02:04:33.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:32 smithi092 ceph-mon[31190]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T02:04:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:32 smithi099 ceph-mon[34404]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-21T02:04:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:34 smithi136 ceph-mon[34754]: pgmap v348: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-21T02:04:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:34 smithi092 ceph-mon[31190]: pgmap v348: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-21T02:04:35.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:34 smithi099 ceph-mon[34404]: pgmap v348: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-21T02:04:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:36 smithi136 ceph-mon[34754]: pgmap v349: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-21T02:04:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:36 smithi092 ceph-mon[31190]: pgmap v349: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-21T02:04:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:36 smithi099 ceph-mon[34404]: pgmap v349: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-21T02:04:37.978 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:37 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:04:38.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:37 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:04:38.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:37 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:04:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: pgmap v350: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:04:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:38 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: pgmap v350: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:04:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:38 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: pgmap v350: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:38 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:04:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:04:40.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:04:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:04:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:04:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:04:40.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:04:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:40 smithi136 ceph-mon[34754]: pgmap v351: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:40 smithi092 ceph-mon[31190]: pgmap v351: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:40 smithi099 ceph-mon[34404]: pgmap v351: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T02:04:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 -- 172.21.15.92:0/2938822878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 msgr2=0x7f818810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 --2- 172.21.15.92:0/2938822878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f818810a980 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8180009ca0 tx=0x7f818002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 -- 172.21.15.92:0/2938822878 shutdown_connections 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 --2- 172.21.15.92:0/2938822878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f818810a980 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 --2- 172.21.15.92:0/2938822878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8188103640 0x7f8188103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 --2- 172.21.15.92:0/2938822878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81881025c0 0x7f8188102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 -- 172.21.15.92:0/2938822878 >> 172.21.15.92:0/2938822878 conn(0x7f81880fdf00 msgr2=0x7f81881003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 -- 172.21.15.92:0/2938822878 shutdown_connections 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 -- 172.21.15.92:0/2938822878 wait complete. 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 Processor -- start 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.869+0000 7f818ee84640 1 -- start start 2024-09-21T02:04:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81881025c0 0x7f818806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8188103640 0x7f818819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f81881a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f818810e1f0 con 0x7f8188103fe0 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f818810e070 con 0x7f8188103640 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f818810e370 con 0x7f81881025c0 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f81881a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f81881a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44148/0 (socket says 172.21.15.92:44148) 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 -- 172.21.15.92:0/2826276539 learned_addr learned my addr 172.21.15.92:0/2826276539 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818cc20640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81881025c0 0x7f818806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f817ffff640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8188103640 0x7f818819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 -- 172.21.15.92:0/2826276539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81881025c0 msgr2=0x7f818806c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81881025c0 0x7f818806c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 -- 172.21.15.92:0/2826276539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8188103640 msgr2=0x7f818819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8188103640 0x7f818819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 -- 172.21.15.92:0/2826276539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81881a19b0 con 0x7f8188103fe0 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818d421640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f81881a11b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f81800097b0 tx=0x7f8180003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f817dffb640 1 -- 172.21.15.92:0/2826276539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180004240 con 0x7f8188103fe0 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f817dffb640 1 -- 172.21.15.92:0/2826276539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81800317c0 con 0x7f8188103fe0 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f817dffb640 1 -- 172.21.15.92:0/2826276539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818002fc30 con 0x7f8188103fe0 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.870+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81881a1c40 con 0x7f8188103fe0 2024-09-21T02:04:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.871+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881a2190 con 0x7f8188103fe0 2024-09-21T02:04:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.873+0000 7f8195df0740 1 Processor -- start 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.873+0000 7f817dffb640 1 -- 172.21.15.92:0/2826276539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f81800043e0 con 0x7f8188103fe0 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.873+0000 7f8195df0740 1 -- start start 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f8195df0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f8195df0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f8195df0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372e3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f8195df0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b7372546f0 con 0x55b736f112f0 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f8195df0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b737243b20 con 0x55b736f194d0 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f8195df0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b737247930 con 0x55b736f3a1b0 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372e24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44152/0 (socket says 172.21.15.92:44152) 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 -- 172.21.15.92:0/1716466336 learned_addr learned my addr 172.21.15.92:0/1716466336 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f817ffff640 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f817ffff640 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 -- 172.21.15.92:0/1716466336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 msgr2=0x55b7372e1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372e1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 -- 172.21.15.92:0/1716466336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 msgr2=0x55b7372e3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372e3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 -- 172.21.15.92:0/1716466336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b737209530 con 0x55b736f112f0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f817dffb640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81700b6620 0x7f81700b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f818cc20640 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372e24c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f818400bd10 tx=0x7f81840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e31801f8edfc9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.874+0000 7f816effd640 1 -- 172.21.15.92:0/1716466336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184004030 con 0x55b736f112f0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f817ffff640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81700b6620 0x7f81700b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f817dffb640 1 -- 172.21.15.92:0/2826276539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8180108990 con 0x7f8188103fe0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f816effd640 1 -- 172.21.15.92:0/1716466336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81840035f0 con 0x55b736f112f0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f816effd640 1 -- 172.21.15.92:0/1716466336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81840116a0 con 0x55b736f112f0 2024-09-21T02:04:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f8195df0740 1 -- 172.21.15.92:0/1716466336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 msgr2=0x55b7372e24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f817ffff640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81700b6620 0x7f81700b8b30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f817800d010 tx=0x7f8178016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.875+0000 7f8195df0740 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372e24c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f818400bd10 tx=0x7f81840107a0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 -- 172.21.15.92:0/1716466336 shutdown_connections 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372e3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372e24c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 --2- 172.21.15.92:0/1716466336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372e1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 -- 172.21.15.92:0/1716466336 >> 172.21.15.92:0/1716466336 conn(0x55b736f2e300 msgr2=0x55b7372dd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 -- 172.21.15.92:0/1716466336 shutdown_connections 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 -- 172.21.15.92:0/1716466336 wait complete. 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 -- start start 2024-09-21T02:04:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 Processor -- start 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.876+0000 7f8195df0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f194d0 0x55b7372ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8195df0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f3a1b0 0x55b7372ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8195df0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b73717bab0 con 0x55b736f112f0 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8195df0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b7371e9730 con 0x55b736f3a1b0 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8195df0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7372e3b30 con 0x55b736f194d0 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8195df0740 10 client.0 _ll_register_callbacks cb 0x55b73723d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372eb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44160/0 (socket says 172.21.15.92:44160) 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 -- 172.21.15.92:0/1204675853 learned_addr learned my addr 172.21.15.92:0/1204675853 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f817ffff640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f194d0 0x55b7372ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818d421640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f3a1b0 0x55b7372ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 -- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f194d0 msgr2=0x55b7372ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f194d0 0x55b7372ebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 -- 172.21.15.92:0/1204675853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f3a1b0 msgr2=0x55b7372ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f3a1b0 0x55b7372ec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 -- 172.21.15.92:0/1204675853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b736fa5790 con 0x55b736f112f0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f818cc20640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372eb610 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f81840039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32738521b2ba6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8167fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8167fff640 1 -- 172.21.15.92:0/1204675853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184029060 con 0x55b736f112f0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8167fff640 1 -- 172.21.15.92:0/1204675853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81840041f0 con 0x55b736f112f0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.877+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81840035f0 con 0x55b736f112f0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8167fff640 1 -- 172.21.15.92:0/1204675853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f818400b900 con 0x55b736f112f0 2024-09-21T02:04:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b73701e620 con 0x55b736f112f0 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8167fff640 1 -- 172.21.15.92:0/1204675853 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81840035f0 con 0x55b736f112f0 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8167fff640 1 -- 172.21.15.92:0/1204675853 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8184004a30 con 0x55b736f112f0 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 20 client.15519 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ef8d80 0x55b7372f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 4 client.15519 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b737256640 con 0x55b736ef8d80 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.878+0000 7f817ffff640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ef8d80 0x55b7372f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.879+0000 7f8167fff640 10 client.15519 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.879+0000 7f817ffff640 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ef8d80 0x55b7372f89c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3346df8096dc148a server_cookie=348efee45dc554c3 in_seq=0 out_seq=0 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.879+0000 7f8167fff640 1 -- 172.21.15.92:0/1204675853 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b737256640 con 0x55b736ef8d80 2024-09-21T02:04:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.879+0000 7f8167fff640 10 client.15519 handle_command_reply: tid=0 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 client.15519 shutdown 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 msgr2=0x55b7372eb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372eb610 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f81840039d0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 shutdown_connections 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ef8d80 0x55b7372f89c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f3a1b0 0x55b7372ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f194d0 0x55b7372ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 --2- 172.21.15.92:0/1204675853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372eb610 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8167fff640 0 client.15519 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 >> 172.21.15.92:0/1204675853 conn(0x55b736f2e300 msgr2=0x55b7372de4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 shutdown_connections 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 1 -- 172.21.15.92:0/1204675853 wait complete. 2024-09-21T02:04:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.880+0000 7f8195df0740 20 client.15519 trim_cache size 0 max 16384 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.894+0000 7f8195df0740 1 Processor -- start 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.894+0000 7f8195df0740 1 -- start start 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f8195df0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f112f0 0x55b7372dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f8195df0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 0x55b7372dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f8195df0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f8195df0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81881aafa0 con 0x55b736f194d0 2024-09-21T02:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f8195df0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81881acae0 con 0x55b736f112f0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f8195df0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f81881ab670 con 0x55b736f3a1b0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 0x55b7372dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 0x55b7372dbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44172/0 (socket says 172.21.15.92:44172) 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 -- 172.21.15.92:0/1626513213 learned_addr learned my addr 172.21.15.92:0/1626513213 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f818d421640 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 -- 172.21.15.92:0/1626513213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 msgr2=0x55b7372dc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372dc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 -- 172.21.15.92:0/1626513213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f112f0 msgr2=0x55b7372dd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f112f0 0x55b7372dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 -- 172.21.15.92:0/1626513213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7372ecb40 con 0x55b736f194d0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f817ffff640 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 0x55b7372dbea0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55b7372f90a0 tx=0x7f817800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6346e00ef7261f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f816e7fc640 1 -- 172.21.15.92:0/1626513213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817801b070 con 0x55b736f194d0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f816e7fc640 1 -- 172.21.15.92:0/1626513213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81780054d0 con 0x55b736f194d0 2024-09-21T02:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.895+0000 7f816e7fc640 1 -- 172.21.15.92:0/1626513213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178004bc0 con 0x55b736f194d0 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 -- 172.21.15.92:0/1626513213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 msgr2=0x55b7372dbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 0x55b7372dbea0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55b7372f90a0 tx=0x7f817800f000 comp rx=0 tx=0).stop 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 -- 172.21.15.92:0/1626513213 shutdown_connections 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372dc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f194d0 0x55b7372dbea0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 --2- 172.21.15.92:0/1626513213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f112f0 0x55b7372dd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 -- 172.21.15.92:0/1626513213 >> 172.21.15.92:0/1626513213 conn(0x55b736f2e300 msgr2=0x55b7372dd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 -- 172.21.15.92:0/1626513213 shutdown_connections 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 -- 172.21.15.92:0/1626513213 wait complete. 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.896+0000 7f8195df0740 1 -- start start 2024-09-21T02:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 Processor -- start 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372f2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81881ac410 con 0x55b736f112f0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81881abd40 con 0x55b736f194d0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f81881a8f80 con 0x55b736f3a1b0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f8195df0740 10 client.0 _ll_register_callbacks cb 0x7f817800c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44184/0 (socket says 172.21.15.92:44184) 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 -- 172.21.15.92:0/2389918677 learned_addr learned my addr 172.21.15.92:0/2389918677 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f817ffff640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818d421640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 -- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 msgr2=0x55b7372f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372f2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 -- 172.21.15.92:0/2389918677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 msgr2=0x55b7372f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.897+0000 7f818cc20640 1 -- 172.21.15.92:0/2389918677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7372f27b0 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f818cc20640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372f2de0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8184012ef0 tx=0x7f8184012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e3fd07ac63a64c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8167fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8167fff640 1 -- 172.21.15.92:0/2389918677 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184029070 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81780054d0 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8167fff640 1 -- 172.21.15.92:0/2389918677 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f818400b690 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8167fff640 1 -- 172.21.15.92:0/2389918677 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f818403e070 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 10 client.15525 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b737363300 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8167fff640 1 -- 172.21.15.92:0/2389918677 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81780054d0 con 0x55b736f112f0 2024-09-21T02:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8167fff640 1 -- 172.21.15.92:0/2389918677 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f81840041f0 con 0x55b736f112f0 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 10 client.15525 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 20 client.15525 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ece1c0 0x55b737365c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 4 client.15525 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.898+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7372520f0 con 0x55b736ece1c0 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.899+0000 7f817ffff640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ece1c0 0x55b737365c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.899+0000 7f8167fff640 10 client.15525 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.899+0000 7f817ffff640 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ece1c0 0x55b737365c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26801dc0b0785e8 server_cookie=4e3d6d1d0c98b83c in_seq=0 out_seq=0 2024-09-21T02:04:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.910+0000 7f8167fff640 1 -- 172.21.15.92:0/2389918677 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7372520f0 con 0x55b736ece1c0 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.910+0000 7f8167fff640 10 client.15525 handle_command_reply: tid=0 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.910+0000 7f8195df0740 1 client.15525 shutdown 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 msgr2=0x55b7372f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372f2de0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8184012ef0 tx=0x7f8184012f20 comp rx=0 tx=0).stop 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 shutdown_connections 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b736ece1c0 0x55b737365c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b736f3a1b0 0x55b7372f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b736f194d0 0x55b7372f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 --2- 172.21.15.92:0/2389918677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b736f112f0 0x55b7372f2de0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8167fff640 0 client.15525 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 >> 172.21.15.92:0/2389918677 conn(0x55b736f2e300 msgr2=0x55b7372e96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 shutdown_connections 2024-09-21T02:04:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 1 -- 172.21.15.92:0/2389918677 wait complete. 2024-09-21T02:04:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f8195df0740 20 client.15525 trim_cache size 0 max 16384 2024-09-21T02:04:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:04:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:04:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:04:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81700b6620 msgr2=0x7f81700b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f818ee84640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81700b6620 0x7f81700b8b30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f817800d010 tx=0x7f8178016040 comp rx=0 tx=0).stop 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 msgr2=0x7f81881a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.911+0000 7f818ee84640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f81881a11b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f81800097b0 tx=0x7f8180003c20 comp rx=0 tx=0).stop 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 shutdown_connections 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f81700b6620 0x7f81700b8b30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8188103fe0 0x7f81881a11b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8188103640 0x7f818819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 --2- 172.21.15.92:0/2826276539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81881025c0 0x7f818806c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 >> 172.21.15.92:0/2826276539 conn(0x7f81880fdf00 msgr2=0x7f8188106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 shutdown_connections 2024-09-21T02:04:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:41.912+0000 7f818ee84640 1 -- 172.21.15.92:0/2826276539 wait complete. 2024-09-21T02:04:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:42 smithi136 ceph-mon[34754]: pgmap v352: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T02:04:43.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:42 smithi092 ceph-mon[31190]: pgmap v352: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T02:04:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:42 smithi099 ceph-mon[34404]: pgmap v352: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T02:04:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:43 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:04:44.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:43 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:04:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:43 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:04:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:44 smithi136 ceph-mon[34754]: pgmap v353: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T02:04:45.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:44 smithi092 ceph-mon[31190]: pgmap v353: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T02:04:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:44 smithi099 ceph-mon[34404]: pgmap v353: 97 pgs: 97 active+clean; 664 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T02:04:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:46 smithi136 ceph-mon[34754]: pgmap v354: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 16 op/s 2024-09-21T02:04:47.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:46 smithi092 ceph-mon[31190]: pgmap v354: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 16 op/s 2024-09-21T02:04:47.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:46 smithi099 ceph-mon[34404]: pgmap v354: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 16 op/s 2024-09-21T02:04:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:48 smithi136 ceph-mon[34754]: pgmap v355: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:04:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:48 smithi092 ceph-mon[31190]: pgmap v355: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:04:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:48 smithi099 ceph-mon[34404]: pgmap v355: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:04:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:50 smithi136 ceph-mon[34754]: pgmap v356: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 16 op/s 2024-09-21T02:04:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:50 smithi092 ceph-mon[31190]: pgmap v356: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 16 op/s 2024-09-21T02:04:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:50 smithi099 ceph-mon[34404]: pgmap v356: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 16 op/s 2024-09-21T02:04:53.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:52 smithi136 ceph-mon[34754]: pgmap v357: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:04:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:52 smithi092 ceph-mon[31190]: pgmap v357: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:04:53.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:52 smithi099 ceph-mon[34404]: pgmap v357: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:04:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 -- 172.21.15.92:0/1577398755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103fc0 msgr2=0x7f6cc410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/1577398755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103fc0 0x7f6cc410a960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009c90 tx=0x7f6cbc02a860 comp rx=0 tx=0).stop 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 -- 172.21.15.92:0/1577398755 shutdown_connections 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/1577398755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103fc0 0x7f6cc410a960 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/1577398755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc4103620 0x7f6cc4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/1577398755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cc41025a0 0x7f6cc4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 -- 172.21.15.92:0/1577398755 >> 172.21.15.92:0/1577398755 conn(0x7f6cc40fdee0 msgr2=0x7f6cc4100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 -- 172.21.15.92:0/1577398755 shutdown_connections 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.011+0000 7f6cca5f1640 1 -- 172.21.15.92:0/1577398755 wait complete. 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 Processor -- start 2024-09-21T02:04:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 -- start start 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc41025a0 0x7f6cc406b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 0x7f6cc419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cc4103fc0 0x7f6cc41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e220 con 0x7f6cc4103620 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e0a0 con 0x7f6cc41025a0 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.012+0000 7f6cca5f1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e3a0 con 0x7f6cc4103fc0 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 0x7f6cc419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 0x7f6cc419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44878/0 (socket says 172.21.15.92:44878) 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 -- 172.21.15.92:0/4274799503 learned_addr learned my addr 172.21.15.92:0/4274799503 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cc4103fc0 0x7f6cc41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc41025a0 0x7f6cc406b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 -- 172.21.15.92:0/4274799503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cc4103fc0 msgr2=0x7f6cc41a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cc4103fc0 0x7f6cc41a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 -- 172.21.15.92:0/4274799503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc41025a0 msgr2=0x7f6cc406b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc41025a0 0x7f6cc406b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 -- 172.21.15.92:0/4274799503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc41a19d0 con 0x7f6cc4103620 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc41025a0 0x7f6cc406b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 0x7f6cc419ec60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c890 tx=0x7f6cb800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc17fa640 1 -- 172.21.15.92:0/4274799503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8004220 con 0x7f6cc4103620 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc17fa640 1 -- 172.21.15.92:0/4274799503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb80043c0 con 0x7f6cc4103620 2024-09-21T02:04:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc41a1cc0 con 0x7f6cc4103620 2024-09-21T02:04:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cc17fa640 1 -- 172.21.15.92:0/4274799503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8004a00 con 0x7f6cc4103620 2024-09-21T02:04:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.013+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc41a21c0 con 0x7f6cc4103620 2024-09-21T02:04:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 Processor -- start 2024-09-21T02:04:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 -- start start 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c6a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298a14d0 0x561d29c6b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561d29bdc6f0 con 0x561d298992f0 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561d29bcbb20 con 0x561d298c21b0 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.016+0000 7f6cd163e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561d29bcf930 con 0x561d298a14d0 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c6a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc17fa640 1 -- 172.21.15.92:0/4274799503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f6cb80079b0 con 0x7f6cc4103620 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c6a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44884/0 (socket says 172.21.15.92:44884) 2024-09-21T02:04:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 -- 172.21.15.92:0/628319381 learned_addr learned my addr 172.21.15.92:0/628319381 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298a14d0 0x561d29c6b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 -- 172.21.15.92:0/628319381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298a14d0 msgr2=0x561d29c6b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298a14d0 0x561d29c6b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 -- 172.21.15.92:0/628319381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 msgr2=0x561d29c69e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c69e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 -- 172.21.15.92:0/628319381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d29b91530 con 0x561d298992f0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c6a4c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400bd40 tx=0x7f6cb400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb0528e19393703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6caa7fc640 1 -- 172.21.15.92:0/628319381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4004220 con 0x561d298992f0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.017+0000 7f6caa7fc640 1 -- 172.21.15.92:0/628319381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb400be30 con 0x561d298992f0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.018+0000 7f6caa7fc640 1 -- 172.21.15.92:0/628319381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40116a0 con 0x561d298992f0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.018+0000 7f6cc17fa640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cac0b65a0 0x7f6cac0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.018+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cac0b65a0 0x7f6cac0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.018+0000 7f6cc17fa640 1 -- 172.21.15.92:0/4274799503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6cb80e71b0 con 0x7f6cc4103620 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cac0b65a0 0x7f6cac0b8ab0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc02ae50 tx=0x7f6cbc031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 -- 172.21.15.92:0/628319381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 msgr2=0x561d29c6a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c6a4c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400bd40 tx=0x7f6cb400f730 comp rx=0 tx=0).stop 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 -- 172.21.15.92:0/628319381 shutdown_connections 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298a14d0 0x561d29c6b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c6a4c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 --2- 172.21.15.92:0/628319381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c69e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 -- 172.21.15.92:0/628319381 >> 172.21.15.92:0/628319381 conn(0x561d298b6300 msgr2=0x561d29c658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 -- 172.21.15.92:0/628319381 shutdown_connections 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 -- 172.21.15.92:0/628319381 wait complete. 2024-09-21T02:04:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 -- start start 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.019+0000 7f6cd163e740 1 Processor -- start 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c73600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561d29b03ab0 con 0x561d298a14d0 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561d29b71730 con 0x561d298c21b0 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561d29c6bb30 con 0x561d298992f0 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 10 client.0 _ll_register_callbacks cb 0x561d29bc56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c73cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44894/0 (socket says 172.21.15.92:44894) 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 -- 172.21.15.92:0/1936858723 learned_addr learned my addr 172.21.15.92:0/1936858723 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c73600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 -- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 msgr2=0x561d29c73600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c73600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 -- 172.21.15.92:0/1936858723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 msgr2=0x561d29c747c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c747c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 -- 172.21.15.92:0/1936858723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2992d790 con 0x561d298a14d0 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c73600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c73cc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80a9790 tx=0x7f6cb800a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa890172075b9413 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6ca37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6ca37fe640 1 -- 172.21.15.92:0/1936858723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb801d040 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6ca37fe640 1 -- 172.21.15.92:0/1936858723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6cb80ac630 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.020+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb400be30 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6ca37fe640 1 -- 172.21.15.92:0/1936858723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cb80acb10 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d299a6620 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6ca37fe640 1 -- 172.21.15.92:0/1936858723 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb400be30 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6ca37fe640 1 -- 172.21.15.92:0/1936858723 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cb80a8b10 con 0x561d298a14d0 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 20 client.15537 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d29880d80 0x561d29c80c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 4 client.15537 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d29bde640 con 0x561d29880d80 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.021+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d29880d80 0x561d29c80c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.022+0000 7f6ca37fe640 10 client.15537 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.022+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d29880d80 0x561d29c80c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79360a0cb0c55356 server_cookie=33a9aa9641570b4c in_seq=0 out_seq=0 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6ca37fe640 1 -- 172.21.15.92:0/1936858723 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d29bde640 con 0x561d29880d80 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6ca37fe640 10 client.15537 handle_command_reply: tid=0 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 client.15537 shutdown 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 msgr2=0x561d29c73cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c73cc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80a9790 tx=0x7f6cb800a5e0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 shutdown_connections 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d29880d80 0x561d29c80c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c73cc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1936858723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c73600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 >> 172.21.15.92:0/1936858723 conn(0x561d298b6300 msgr2=0x561d29c664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6ca37fe640 0 client.15537 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 shutdown_connections 2024-09-21T02:04:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 1 -- 172.21.15.92:0/1936858723 wait complete. 2024-09-21T02:04:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.023+0000 7f6cd163e740 20 client.15537 trim_cache size 0 max 16384 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 Processor -- start 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 -- start start 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c65930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298a14d0 0x561d29c6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298c21b0 0x561d29c71fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41ab0f0 con 0x561d298992f0 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41acc30 con 0x561d298a14d0 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cd163e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41ab7c0 con 0x561d298c21b0 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c65930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c65930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44906/0 (socket says 172.21.15.92:44906) 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 -- 172.21.15.92:0/1847267811 learned_addr learned my addr 172.21.15.92:0/1847267811 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298a14d0 0x561d29c6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 -- 172.21.15.92:0/1847267811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298c21b0 msgr2=0x561d29c71fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298c21b0 0x561d29c71fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298c21b0 0x561d29c71fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 -- 172.21.15.92:0/1847267811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298a14d0 msgr2=0x561d29c6ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298a14d0 0x561d29c6ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc3fff640 1 -- 172.21.15.92:0/1847267811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d29c71780 con 0x561d298992f0 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.038+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298a14d0 0x561d29c6ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.039+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c65930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4006010 tx=0x7f6cb400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711f39c52d983e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.039+0000 7f6ca9ffb640 1 -- 172.21.15.92:0/1847267811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4033070 con 0x561d298992f0 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.039+0000 7f6ca9ffb640 1 -- 172.21.15.92:0/1847267811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb40054f0 con 0x561d298992f0 2024-09-21T02:04:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.039+0000 7f6ca9ffb640 1 -- 172.21.15.92:0/1847267811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4003830 con 0x561d298992f0 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 -- 172.21.15.92:0/1847267811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 msgr2=0x561d29c65930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c65930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4006010 tx=0x7f6cb400b660 comp rx=0 tx=0).stop 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 -- 172.21.15.92:0/1847267811 shutdown_connections 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298c21b0 0x561d29c71fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298a14d0 0x561d29c6ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 --2- 172.21.15.92:0/1847267811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298992f0 0x561d29c65930 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 -- 172.21.15.92:0/1847267811 >> 172.21.15.92:0/1847267811 conn(0x561d298b6300 msgr2=0x561d29c65220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 -- 172.21.15.92:0/1847267811 shutdown_connections 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 -- 172.21.15.92:0/1847267811 wait complete. 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 -- start start 2024-09-21T02:04:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 Processor -- start 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.040+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c78090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c78750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c79250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41ac560 con 0x561d298a14d0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41abe90 con 0x561d298c21b0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41a90d0 con 0x561d298992f0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 10 client.0 _ll_register_callbacks cb 0x7f6cb80058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c78750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c78750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44916/0 (socket says 172.21.15.92:44916) 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 -- 172.21.15.92:0/2051094972 learned_addr learned my addr 172.21.15.92:0/2051094972 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c78090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c79250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 -- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 msgr2=0x561d29c78090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c78090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 -- 172.21.15.92:0/2051094972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 msgr2=0x561d29c79250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c79250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 -- 172.21.15.92:0/2051094972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d29c799b0 con 0x561d298a14d0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc8b8e640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c79250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cc37fe640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c78750 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80e4bc0 tx=0x7f6cb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f681b9c9a3b1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6ca37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6ca37fe640 1 -- 172.21.15.92:0/2051094972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb801d040 con 0x561d298a14d0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6ca37fe640 1 -- 172.21.15.92:0/2051094972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6cb80ac0a0 con 0x561d298a14d0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.041+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb40054f0 con 0x561d298a14d0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6ca37fe640 1 -- 172.21.15.92:0/2051094972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cb80ac930 con 0x561d298a14d0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d29c7acd0 con 0x561d298a14d0 2024-09-21T02:04:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6ca37fe640 1 -- 172.21.15.92:0/2051094972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb40054f0 con 0x561d298a14d0 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6ca37fe640 1 -- 172.21.15.92:0/2051094972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cb80b2e30 con 0x561d298a14d0 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 20 client.15543 populate_metadata read hostname 'smithi092' 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d298561c0 0x561d29cf1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 4 client.15543 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d29bda0f0 con 0x561d298561c0 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.042+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d298561c0 0x561d29cf1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.043+0000 7f6ca37fe640 10 client.15543 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.043+0000 7f6cc3fff640 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d298561c0 0x561d29cf1a60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916042d34807ec4d server_cookie=f2c546aba033fb3b in_seq=0 out_seq=0 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6ca37fe640 1 -- 172.21.15.92:0/2051094972 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d29bda0f0 con 0x561d298561c0 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6ca37fe640 10 client.15543 handle_command_reply: tid=0 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 client.15543 shutdown 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 msgr2=0x561d29c78750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c78750 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80e4bc0 tx=0x7f6cb8004730 comp rx=0 tx=0).stop 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 shutdown_connections 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561d298561c0 0x561d29cf1a60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d298c21b0 0x561d29c79250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561d298a14d0 0x561d29c78750 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6ca37fe640 0 client.15543 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 --2- 172.21.15.92:0/2051094972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561d298992f0 0x561d29c78090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.048+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 >> 172.21.15.92:0/2051094972 conn(0x561d298b6300 msgr2=0x561d29c71fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 shutdown_connections 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cd163e740 1 -- 172.21.15.92:0/2051094972 wait complete. 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cd163e740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:04:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:04:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cac0b65a0 msgr2=0x7f6cac0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cac0b65a0 0x7f6cac0b8ab0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc02ae50 tx=0x7f6cbc031000 comp rx=0 tx=0).stop 2024-09-21T02:04:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 msgr2=0x7f6cc419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 0x7f6cc419ec60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c890 tx=0x7f6cb800cd50 comp rx=0 tx=0).stop 2024-09-21T02:04:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 shutdown_connections 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6cac0b65a0 0x7f6cac0b8ab0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cc4103fc0 0x7f6cc41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc4103620 0x7f6cc419ec60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 --2- 172.21.15.92:0/4274799503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc41025a0 0x7f6cc406b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 >> 172.21.15.92:0/4274799503 conn(0x7f6cc40fdee0 msgr2=0x7f6cc4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.049+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 shutdown_connections 2024-09-21T02:04:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:04:55.050+0000 7f6cca5f1640 1 -- 172.21.15.92:0/4274799503 wait complete. 2024-09-21T02:04:55.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:54 smithi136 ceph-mon[34754]: pgmap v358: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-21T02:04:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:54 smithi092 ceph-mon[31190]: pgmap v358: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-21T02:04:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:54 smithi099 ceph-mon[34404]: pgmap v358: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-21T02:04:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:56 smithi136 ceph-mon[34754]: pgmap v359: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-21T02:04:57.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:56 smithi092 ceph-mon[31190]: pgmap v359: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-21T02:04:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:56 smithi099 ceph-mon[34404]: pgmap v359: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-21T02:04:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:04:58 smithi136 ceph-mon[34754]: pgmap v360: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:59.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:04:58 smithi092 ceph-mon[31190]: pgmap v360: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:04:59.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:04:58 smithi099 ceph-mon[34404]: pgmap v360: 97 pgs: 97 active+clean; 665 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T02:05:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:00 smithi136 ceph-mon[34754]: pgmap v361: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T02:05:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:00 smithi092 ceph-mon[31190]: pgmap v361: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T02:05:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:00 smithi099 ceph-mon[34404]: pgmap v361: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T02:05:01.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:05:02.134 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/1498661725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a58106c10 msgr2=0x7f4a58110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/1498661725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a58106c10 0x7f4a58110650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ca0 tx=0x7f4a5002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/1498661725 shutdown_connections 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/1498661725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a58106c10 0x7f4a58110650 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/1498661725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a581062c0 0x7f4a581066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/1498661725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 0x7f4a581024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/1498661725 >> 172.21.15.92:0/1498661725 conn(0x7f4a58077cc0 msgr2=0x7f4a58078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/1498661725 shutdown_connections 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.651+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/1498661725 wait complete. 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 Processor -- start 2024-09-21T02:05:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 -- start start 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 0x7f4a5819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a581062c0 0x7f4a5819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a58106c10 0x7f4a581a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58113ae0 con 0x7f4a581062c0 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58113960 con 0x7f4a58106c10 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.652+0000 7f4a5d9c4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58113c60 con 0x7f4a581020d0 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a581062c0 0x7f4a5819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a581062c0 0x7f4a5819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44926/0 (socket says 172.21.15.92:44926) 2024-09-21T02:05:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a567fc640 1 -- 172.21.15.92:0/51402308 learned_addr learned my addr 172.21.15.92:0/51402308 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 0x7f4a5819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a577fe640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a58106c10 0x7f4a581a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 -- 172.21.15.92:0/51402308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a58106c10 msgr2=0x7f4a581a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a58106c10 0x7f4a581a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 -- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a581062c0 msgr2=0x7f4a5819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a581062c0 0x7f4a5819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 -- 172.21.15.92:0/51402308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581a25c0 con 0x7f4a581020d0 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.653+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 0x7f4a5819c650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9f0 tx=0x7f4a4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:02.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.654+0000 7f4a5c9c2640 1 -- 172.21.15.92:0/51402308 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800ccc0 con 0x7f4a581020d0 2024-09-21T02:05:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.654+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581a28b0 con 0x7f4a581020d0 2024-09-21T02:05:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.654+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a9010 con 0x7f4a581020d0 2024-09-21T02:05:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.654+0000 7f4a5c9c2640 1 -- 172.21.15.92:0/51402308 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a48005020 con 0x7f4a581020d0 2024-09-21T02:05:02.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.654+0000 7f4a5c9c2640 1 -- 172.21.15.92:0/51402308 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48004a00 con 0x7f4a581020d0 2024-09-21T02:05:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.657+0000 7f4a63f38740 1 Processor -- start 2024-09-21T02:05:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.657+0000 7f4a5c9c2640 1 -- 172.21.15.92:0/51402308 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f4a48004ba0 con 0x7f4a581020d0 2024-09-21T02:05:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 -- start start 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb199070 0x55cfbb540270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb5407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 0x55cfbb540cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cfbb4b6e50 con 0x55cfbb199070 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cfbb4a6280 con 0x55cfbb178390 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a63f38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cfbb4aa090 con 0x55cfbb1701b0 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a5c9c2640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4a400b6620 0x7f4a400b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb199070 0x55cfbb540270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 0x55cfbb540cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb199070 0x55cfbb540270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44942/0 (socket says 172.21.15.92:44942) 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a577fe640 1 -- 172.21.15.92:0/4196585615 learned_addr learned my addr 172.21.15.92:0/4196585615 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 0x55cfbb540cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:47636/0 (socket says 172.21.15.92:47636) 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a5c9c2640 1 -- 172.21.15.92:0/51402308 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4a480e7260 con 0x7f4a581020d0 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.658+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb5407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a567fc640 1 -- 172.21.15.92:0/4196585615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 msgr2=0x55cfbb5407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a567fc640 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb5407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a577fe640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4a400b6620 0x7f4a400b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a567fc640 1 -- 172.21.15.92:0/4196585615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb199070 msgr2=0x55cfbb540270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a567fc640 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb199070 0x55cfbb540270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a567fc640 1 -- 172.21.15.92:0/4196585615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfbb204650 con 0x55cfbb178390 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a567fc640 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 0x55cfbb540cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009b80 tx=0x7f4a4c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61022d60a7db2b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a3dffb640 1 -- 172.21.15.92:0/4196585615 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c012450 con 0x55cfbb178390 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a577fe640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4a400b6620 0x7f4a400b8b30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a5002f500 tx=0x7f4a5002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a3dffb640 1 -- 172.21.15.92:0/4196585615 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c012aa0 con 0x55cfbb178390 2024-09-21T02:05:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.659+0000 7f4a3dffb640 1 -- 172.21.15.92:0/4196585615 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00faa0 con 0x55cfbb178390 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 -- 172.21.15.92:0/4196585615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 msgr2=0x55cfbb540cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 0x55cfbb540cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009b80 tx=0x7f4a4c0117b0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 -- 172.21.15.92:0/4196585615 shutdown_connections 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb178390 0x55cfbb540cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb5407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 --2- 172.21.15.92:0/4196585615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb199070 0x55cfbb540270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 -- 172.21.15.92:0/4196585615 >> 172.21.15.92:0/4196585615 conn(0x55cfbb18d460 msgr2=0x55cfbb53e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 -- 172.21.15.92:0/4196585615 shutdown_connections 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.660+0000 7f4a63f38740 1 -- 172.21.15.92:0/4196585615 wait complete. 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 -- start start 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 Processor -- start 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb1701b0 0x55cfbb54b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb54bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb199070 0x55cfbb54c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cfbb307700 con 0x55cfbb178390 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cfbb441110 con 0x55cfbb1701b0 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.661+0000 7f4a63f38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cfbb54ea50 con 0x55cfbb199070 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a63f38740 10 client.0 _ll_register_callbacks cb 0x55cfbb45a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb54bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb54bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44958/0 (socket says 172.21.15.92:44958) 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 -- 172.21.15.92:0/3546422177 learned_addr learned my addr 172.21.15.92:0/3546422177 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a577fe640 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb199070 0x55cfbb54c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 -- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb199070 msgr2=0x55cfbb54c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb199070 0x55cfbb54c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 -- 172.21.15.92:0/3546422177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb1701b0 msgr2=0x55cfbb54b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb1701b0 0x55cfbb54b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 -- 172.21.15.92:0/3546422177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfbb46ec40 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a567fc640 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb54bdd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c01c040 tx=0x7f4a4c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97587012011444d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.662+0000 7f4a36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a36ffd640 1 -- 172.21.15.92:0/3546422177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c028070 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a36ffd640 1 -- 172.21.15.92:0/3546422177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4a4c012860 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c012aa0 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a36ffd640 1 -- 172.21.15.92:0/3546422177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a4c033050 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a63f38740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfbb55a6c0 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.663+0000 7f4a36ffd640 1 -- 172.21.15.92:0/3546422177 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c012aa0 con 0x55cfbb178390 2024-09-21T02:05:02.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a36ffd640 1 -- 172.21.15.92:0/3546422177 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a4c0046c0 con 0x55cfbb178390 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a63f38740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a63f38740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a63f38740 20 client.15555 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a63f38740 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb1579f0 0x55cfbb55da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a63f38740 4 client.15555 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfbb4b8da0 con 0x55cfbb1579f0 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.664+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb1579f0 0x55cfbb55da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.665+0000 7f4a36ffd640 10 client.15555 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:02.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.665+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb1579f0 0x55cfbb55da80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e88ef65a6182f86 server_cookie=34af2aed5d9aaa7e in_seq=0 out_seq=0 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a36ffd640 1 -- 172.21.15.92:0/3546422177 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfbb4b8da0 con 0x55cfbb1579f0 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a36ffd640 10 client.15555 handle_command_reply: tid=0 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 client.15555 shutdown 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 msgr2=0x55cfbb54bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb54bdd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c01c040 tx=0x7f4a4c009e80 comp rx=0 tx=0).stop 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 shutdown_connections 2024-09-21T02:05:02.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb1579f0 0x55cfbb55da80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb199070 0x55cfbb54c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb54bdd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 --2- 172.21.15.92:0/3546422177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb1701b0 0x55cfbb54b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 >> 172.21.15.92:0/3546422177 conn(0x55cfbb18d460 msgr2=0x55cfbb543400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a36ffd640 0 client.15555 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 shutdown_connections 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 1 -- 172.21.15.92:0/3546422177 wait complete. 2024-09-21T02:05:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.666+0000 7f4a63f38740 20 client.15555 trim_cache size 0 max 16384 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.699+0000 7f4a63f38740 1 Processor -- start 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.699+0000 7f4a63f38740 1 -- start start 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb54d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb5419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb53d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a63f38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4400e4c0 con 0x55cfbb178390 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a63f38740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44010000 con 0x55cfbb199070 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a63f38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4400eb90 con 0x55cfbb1701b0 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb5419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb5419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44972/0 (socket says 172.21.15.92:44972) 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 -- 172.21.15.92:0/2390625281 learned_addr learned my addr 172.21.15.92:0/2390625281 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb54d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a577fe640 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb53d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 -- 172.21.15.92:0/2390625281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 msgr2=0x55cfbb54d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb54d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 -- 172.21.15.92:0/2390625281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 msgr2=0x55cfbb53d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb53d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.700+0000 7f4a567fc640 1 -- 172.21.15.92:0/2390625281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfbb54e3c0 con 0x55cfbb178390 2024-09-21T02:05:02.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.701+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb5419b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c001eb0 tx=0x7f4a4c0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540486961a2f0623 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:02.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.701+0000 7f4a3d7fa640 1 -- 172.21.15.92:0/2390625281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c028040 con 0x55cfbb178390 2024-09-21T02:05:02.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.701+0000 7f4a3d7fa640 1 -- 172.21.15.92:0/2390625281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c00f440 con 0x55cfbb178390 2024-09-21T02:05:02.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.701+0000 7f4a3d7fa640 1 -- 172.21.15.92:0/2390625281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0042a0 con 0x55cfbb178390 2024-09-21T02:05:02.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 -- 172.21.15.92:0/2390625281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 msgr2=0x55cfbb5419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb5419b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c001eb0 tx=0x7f4a4c0058f0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 -- 172.21.15.92:0/2390625281 shutdown_connections 2024-09-21T02:05:02.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb53d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb178390 0x55cfbb5419b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2390625281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb1701b0 0x55cfbb54d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 -- 172.21.15.92:0/2390625281 >> 172.21.15.92:0/2390625281 conn(0x55cfbb18d460 msgr2=0x55cfbb5412a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 -- 172.21.15.92:0/2390625281 shutdown_connections 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.702+0000 7f4a63f38740 1 -- 172.21.15.92:0/2390625281 wait complete. 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 -- start start 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 Processor -- start 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 0x55cfbb550030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb178390 0x55cfbb550570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb5c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4400f930 con 0x55cfbb1701b0 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4400f260 con 0x55cfbb199070 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4400c4a0 con 0x55cfbb178390 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.703+0000 7f4a63f38740 10 client.0 _ll_register_callbacks cb 0x7f4a4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 0x55cfbb550030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 0x55cfbb550030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44984/0 (socket says 172.21.15.92:44984) 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 -- 172.21.15.92:0/2547744980 learned_addr learned my addr 172.21.15.92:0/2547744980 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb178390 0x55cfbb550570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a577fe640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb5c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 -- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb178390 msgr2=0x55cfbb550570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb178390 0x55cfbb550570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 -- 172.21.15.92:0/2547744980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 msgr2=0x55cfbb5c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb5c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 -- 172.21.15.92:0/2547744980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfbb550e60 con 0x55cfbb1701b0 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb178390 0x55cfbb550570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a577fe640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb5c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a56ffd640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 0x55cfbb550030 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a480a9840 tx=0x7f4a4800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b375f40ebbfefe9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a36ffd640 1 -- 172.21.15.92:0/2547744980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480ac6d0 con 0x55cfbb1701b0 2024-09-21T02:05:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a36ffd640 1 -- 172.21.15.92:0/2547744980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4a480a8bc0 con 0x55cfbb1701b0 2024-09-21T02:05:02.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.704+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c00f440 con 0x55cfbb1701b0 2024-09-21T02:05:02.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a36ffd640 1 -- 172.21.15.92:0/2547744980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a480acd80 con 0x55cfbb1701b0 2024-09-21T02:05:02.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 10 client.15561 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:02.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfbb5c88b0 con 0x55cfbb1701b0 2024-09-21T02:05:02.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a36ffd640 1 -- 172.21.15.92:0/2547744980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a480ad970 con 0x55cfbb1701b0 2024-09-21T02:05:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a36ffd640 1 -- 172.21.15.92:0/2547744980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a48071aa0 con 0x55cfbb1701b0 2024-09-21T02:05:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 10 client.15561 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 20 client.15561 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb12cda0 0x55cfbb5cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 4 client.15561 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4a4c00f440 con 0x55cfbb12cda0 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.705+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb12cda0 0x55cfbb5cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.706+0000 7f4a36ffd640 10 client.15561 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.706+0000 7f4a567fc640 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb12cda0 0x55cfbb5cb070 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8682bb01c7b81d0 server_cookie=a1a01beca694cad7 in_seq=0 out_seq=0 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.706+0000 7f4a36ffd640 1 -- 172.21.15.92:0/2547744980 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7f4a4c00f440 con 0x55cfbb12cda0 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.706+0000 7f4a36ffd640 10 client.15561 handle_command_reply: tid=0 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.706+0000 7f4a63f38740 1 client.15561 shutdown 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.706+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 msgr2=0x55cfbb550030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 0x55cfbb550030 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a480a9840 tx=0x7f4a4800c7d0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 shutdown_connections 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cfbb12cda0 0x55cfbb5cb070 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cfbb199070 0x55cfbb5c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfbb178390 0x55cfbb550570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 --2- 172.21.15.92:0/2547744980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cfbb1701b0 0x55cfbb550030 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 >> 172.21.15.92:0/2547744980 conn(0x55cfbb18d460 msgr2=0x55cfbb54d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a36ffd640 0 client.15561 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 shutdown_connections 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 1 -- 172.21.15.92:0/2547744980 wait complete. 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.707+0000 7f4a63f38740 20 client.15561 trim_cache size 0 max 16384 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:05:02.711 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:05:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4a400b6620 msgr2=0x7f4a400b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4a400b6620 0x7f4a400b8b30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a5002f500 tx=0x7f4a5002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 msgr2=0x7f4a5819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 0x7f4a5819c650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9f0 tx=0x7f4a4800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 shutdown_connections 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4a400b6620 0x7f4a400b8b30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a58106c10 0x7f4a581a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a581062c0 0x7f4a5819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 --2- 172.21.15.92:0/51402308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a581020d0 0x7f4a5819c650 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 >> 172.21.15.92:0/51402308 conn(0x7f4a58077cc0 msgr2=0x7f4a581083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 shutdown_connections 2024-09-21T02:05:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:02.708+0000 7f4a5d9c4640 1 -- 172.21.15.92:0/51402308 wait complete. 2024-09-21T02:05:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:02 smithi136 ceph-mon[34754]: pgmap v362: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:03.298 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:02 smithi092 ceph-mon[31190]: pgmap v362: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:03.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:02 smithi099 ceph-mon[34404]: pgmap v362: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:03.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:05:03.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:05:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:04.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 -- 172.21.15.92:0/1581892503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe784102a60 msgr2=0x7fe784102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 --2- 172.21.15.92:0/1581892503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe784102a60 0x7fe784102e70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe774009900 tx=0x7fe77402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 -- 172.21.15.92:0/1581892503 shutdown_connections 2024-09-21T02:05:04.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 --2- 172.21.15.92:0/1581892503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe784108cf0 0x7fe78410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 --2- 172.21.15.92:0/1581892503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7841083a0 0x7fe7841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 --2- 172.21.15.92:0/1581892503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe784102a60 0x7fe784102e70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 -- 172.21.15.92:0/1581892503 >> 172.21.15.92:0/1581892503 conn(0x7fe784079730 msgr2=0x7fe784079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.495+0000 7fe789124640 1 -- 172.21.15.92:0/1581892503 shutdown_connections 2024-09-21T02:05:04.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.496+0000 7fe789124640 1 -- 172.21.15.92:0/1581892503 wait complete. 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 Processor -- start 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 -- start start 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7841083a0 0x7fe784198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe784108cf0 0x7fe7841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 0x7fe78419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe784111860 con 0x7fe78419dbd0 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841116e0 con 0x7fe784108cf0 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe789124640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7841119e0 con 0x7fe7841083a0 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe78359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 0x7fe78419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe78359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 0x7fe78419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39004/0 (socket says 172.21.15.92:39004) 2024-09-21T02:05:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.497+0000 7fe78359e640 1 -- 172.21.15.92:0/1303038683 learned_addr learned my addr 172.21.15.92:0/1303038683 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78259c640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe784108cf0 0x7fe7841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe782d9d640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7841083a0 0x7fe784198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78359e640 1 -- 172.21.15.92:0/1303038683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7841083a0 msgr2=0x7fe784198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78359e640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7841083a0 0x7fe784198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78359e640 1 -- 172.21.15.92:0/1303038683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe784108cf0 msgr2=0x7fe7841989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78359e640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe784108cf0 0x7fe7841989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78359e640 1 -- 172.21.15.92:0/1303038683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78419e7b0 con 0x7fe78419dbd0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78259c640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe784108cf0 0x7fe7841989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe78359e640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 0x7fe78419dfe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00db80 tx=0x7fe77c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c017070 con 0x7fe78419dbd0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7841a4d50 con 0x7fe78419dbd0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841a5150 con 0x7fe78419dbd0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c004d20 con 0x7fe78419dbd0 2024-09-21T02:05:04.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.498+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0106c0 con 0x7fe78419dbd0 2024-09-21T02:05:04.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.500+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe784103ce0 con 0x7fe78419dbd0 2024-09-21T02:05:04.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.503+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fe77c005000 con 0x7fe78419dbd0 2024-09-21T02:05:04.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.504+0000 7fe76bfff640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe76c0b65a0 0x7fe76c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.504+0000 7fe782d9d640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe76c0b65a0 0x7fe76c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.504+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe77c0e7600 con 0x7fe78419dbd0 2024-09-21T02:05:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.505+0000 7fe782d9d640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe76c0b65a0 0x7fe76c0b8ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe774009230 tx=0x7fe774031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.505+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe77c0add10 con 0x7fe78419dbd0 2024-09-21T02:05:04.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.754+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7841994c0 con 0x7fe78419dbd0 2024-09-21T02:05:04.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.754+0000 7fe76bfff640 1 -- 172.21.15.92:0/1303038683 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7fe77c0b20f0 con 0x7fe78419dbd0 2024-09-21T02:05:04.755 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:05:04.755 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:05:04.756 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe76c0b65a0 msgr2=0x7fe76c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe76c0b65a0 0x7fe76c0b8ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe774009230 tx=0x7fe774031400 comp rx=0 tx=0).stop 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 msgr2=0x7fe78419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 0x7fe78419dfe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00db80 tx=0x7fe77c00b630 comp rx=0 tx=0).stop 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 shutdown_connections 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe76c0b65a0 0x7fe76c0b8ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe78419dbd0 0x7fe78419dfe0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe784108cf0 0x7fe7841989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 --2- 172.21.15.92:0/1303038683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7841083a0 0x7fe784198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 >> 172.21.15.92:0/1303038683 conn(0x7fe784079730 msgr2=0x7fe784101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 shutdown_connections 2024-09-21T02:05:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:04.758+0000 7fe789124640 1 -- 172.21.15.92:0/1303038683 wait complete. 2024-09-21T02:05:05.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:04 smithi092 ceph-mon[31190]: pgmap v363: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:05.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:04 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1303038683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:05:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:04 smithi136 ceph-mon[34754]: pgmap v363: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:04 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1303038683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:05:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:04 smithi099 ceph-mon[34404]: pgmap v363: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:04 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1303038683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:05:05.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:05:05.789 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:06.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 -- 172.21.15.92:0/1793024233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28104840 msgr2=0x7f9c28104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/1793024233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28104840 0x7f9c28104c50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009900 tx=0x7f9c1802a410 comp rx=0 tx=0).stop 2024-09-21T02:05:06.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 -- 172.21.15.92:0/1793024233 shutdown_connections 2024-09-21T02:05:06.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/1793024233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 0x7f9c28110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/1793024233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28105270 0x7f9c2810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/1793024233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28104840 0x7f9c28104c50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 -- 172.21.15.92:0/1793024233 >> 172.21.15.92:0/1793024233 conn(0x7f9c28079730 msgr2=0x7f9c28079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 -- 172.21.15.92:0/1793024233 shutdown_connections 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.310+0000 7f9c2de6a640 1 -- 172.21.15.92:0/1793024233 wait complete. 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.311+0000 7f9c2de6a640 1 Processor -- start 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.311+0000 7f9c2de6a640 1 -- start start 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c2de6a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28104840 0x7f9c2819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c2de6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 0x7f9c2819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c2de6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 0x7f9c2819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c2de6a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28113a90 con 0x7f9c28105270 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c2de6a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28113910 con 0x7f9c2810ddf0 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c2de6a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28113c10 con 0x7f9c28104840 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 0x7f9c2819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 0x7f9c2819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39018/0 (socket says 172.21.15.92:39018) 2024-09-21T02:05:06.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28104840 0x7f9c2819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 -- 172.21.15.92:0/2126502903 learned_addr learned my addr 172.21.15.92:0/2126502903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 0x7f9c2819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 -- 172.21.15.92:0/2126502903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28104840 msgr2=0x7f9c2819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28104840 0x7f9c2819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 -- 172.21.15.92:0/2126502903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 msgr2=0x7f9c2819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 0x7f9c2819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c26ffd640 1 -- 172.21.15.92:0/2126502903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c281a0430 con 0x7f9c28105270 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 0x7f9c2819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.312+0000 7f9c277fe640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28104840 0x7f9c2819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.313+0000 7f9c26ffd640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 0x7f9c2819ab20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d750 tx=0x7f9c1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.313+0000 7f9c24ff9640 1 -- 172.21.15.92:0/2126502903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c017070 con 0x7f9c28105270 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.313+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c281a0720 con 0x7f9c28105270 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.313+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c28102630 con 0x7f9c28105270 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.313+0000 7f9c24ff9640 1 -- 172.21.15.92:0/2126502903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c1c0054d0 con 0x7f9c28105270 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.313+0000 7f9c24ff9640 1 -- 172.21.15.92:0/2126502903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c004c30 con 0x7f9c28105270 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 Processor -- start 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 -- start start 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cc0a016ea0 con 0x55cc09cf9070 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc0a0062d0 con 0x55cc09cd8390 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c343de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc0a00a0e0 con 0x55cc09cd01b0 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.316+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56792/0 (socket says 172.21.15.92:56792) 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 -- 172.21.15.92:0/835237340 learned_addr learned my addr 172.21.15.92:0/835237340 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c27fff640 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 -- 172.21.15.92:0/835237340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 msgr2=0x55cc0a0a1120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 -- 172.21.15.92:0/835237340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 msgr2=0x55cc0a0a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c277fe640 1 -- 172.21.15.92:0/835237340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc09d64650 con 0x55cc09cd01b0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c27fff640 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.317+0000 7f9c24ff9640 1 -- 172.21.15.92:0/2126502903 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f9c1c004dd0 con 0x7f9c28105270 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.318+0000 7f9c277fe640 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0a0be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c2806bc00 tx=0x7f9c180315e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df8636904ac3a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.318+0000 7f9c0dffb640 1 -- 172.21.15.92:0/835237340 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18040070 con 0x55cc09cd01b0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.318+0000 7f9c0dffb640 1 -- 172.21.15.92:0/835237340 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c18005040 con 0x55cc09cd01b0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.318+0000 7f9c0dffb640 1 -- 172.21.15.92:0/835237340 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18035040 con 0x55cc09cd01b0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.318+0000 7f9c24ff9640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c100b6650 0x7f9c100b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c100b6650 0x7f9c100b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c24ff9640 1 -- 172.21.15.92:0/2126502903 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c1c0e6e80 con 0x7f9c28105270 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 -- 172.21.15.92:0/835237340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 msgr2=0x55cc0a0a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0a0be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c2806bc00 tx=0x7f9c180315e0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c100b6650 0x7f9c100b8b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c20007fd0 tx=0x7f9c200076a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 -- 172.21.15.92:0/835237340 shutdown_connections 2024-09-21T02:05:06.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0a0be0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 --2- 172.21.15.92:0/835237340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 -- 172.21.15.92:0/835237340 >> 172.21.15.92:0/835237340 conn(0x55cc09ced460 msgr2=0x55cc0a09e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 -- 172.21.15.92:0/835237340 shutdown_connections 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.319+0000 7f9c343de740 1 -- 172.21.15.92:0/835237340 wait complete. 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.320+0000 7f9c343de740 1 -- start start 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.320+0000 7f9c343de740 1 Processor -- start 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c343de740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 0x55cc0a0abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c343de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c343de740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cf9070 0x55cc0a0b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 0x55cc0a0abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 0x55cc0a0abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39046/0 (socket says 172.21.15.92:39046) 2024-09-21T02:05:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a0ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56798/0 (socket says 172.21.15.92:56798) 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c277fe640 1 -- 172.21.15.92:0/1287576685 learned_addr learned my addr 172.21.15.92:0/1287576685 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cc09e67700 con 0x55cc09cd01b0 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.321+0000 7f9c27fff640 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cf9070 0x55cc0a0b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc09fa1160 con 0x55cc09cf9070 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc0a0aeec0 con 0x55cc09cd8390 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c343de740 10 client.0 _ll_register_callbacks cb 0x55cc09fba9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c277fe640 1 -- 172.21.15.92:0/1287576685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 msgr2=0x55cc0a0ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c277fe640 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a0ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c277fe640 1 -- 172.21.15.92:0/1287576685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cf9070 msgr2=0x55cc0a0b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c277fe640 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cf9070 0x55cc0a0b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c277fe640 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc09fcec90 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c277fe640 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 0x55cc0a0abd90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9c1802a930 tx=0x7f9c1802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c3961c5e5bad72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.322+0000 7f9c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c06ffd640 1 -- 172.21.15.92:0/1287576685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18040070 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c06ffd640 1 -- 172.21.15.92:0/1287576685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c18004850 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c18005040 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c06ffd640 1 -- 172.21.15.92:0/1287576685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c18035040 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c343de740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc0a0babb0 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.323+0000 7f9c06ffd640 1 -- 172.21.15.92:0/1287576685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c18005040 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c06ffd640 1 -- 172.21.15.92:0/1287576685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c1803cac0 con 0x55cc09cd01b0 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c343de740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c343de740 10 client.15579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:05:06.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c343de740 20 client.15579 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:06.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c343de740 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09cb79f0 0x55cc0a0bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c343de740 4 client.15579 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:06.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc0a018df0 con 0x55cc09cb79f0 2024-09-21T02:05:06.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c26ffd640 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09cb79f0 0x55cc0a0bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.324+0000 7f9c06ffd640 10 client.15579 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:06.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.325+0000 7f9c26ffd640 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09cb79f0 0x55cc0a0bdf10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=260339d5238760dc server_cookie=75f2f202ea133f19 in_seq=0 out_seq=0 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c06ffd640 1 -- 172.21.15.92:0/1287576685 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc0a018df0 con 0x55cc09cb79f0 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c06ffd640 10 client.15579 handle_command_reply: tid=0 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 client.15579 shutdown 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 msgr2=0x55cc0a0abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 0x55cc0a0abd90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9c1802a930 tx=0x7f9c1802a410 comp rx=0 tx=0).stop 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 shutdown_connections 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09cb79f0 0x55cc0a0bdf10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c06ffd640 0 client.15579 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cf9070 0x55cc0a0b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a0ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 --2- 172.21.15.92:0/1287576685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cd01b0 0x55cc0a0abd90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 >> 172.21.15.92:0/1287576685 conn(0x55cc09ced460 msgr2=0x55cc0a0a16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 shutdown_connections 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.326+0000 7f9c343de740 1 -- 172.21.15.92:0/1287576685 wait complete. 2024-09-21T02:05:06.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.327+0000 7f9c343de740 20 client.15579 trim_cache size 0 max 16384 2024-09-21T02:05:06.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 Processor -- start 2024-09-21T02:05:06.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 -- start start 2024-09-21T02:05:06.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b17b0 con 0x55cc09cf9070 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b32f0 con 0x55cc09cd8390 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c343de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b1e80 con 0x55cc09cd01b0 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39062/0 (socket says 172.21.15.92:39062) 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.340+0000 7f9c27fff640 1 -- 172.21.15.92:0/2977713453 learned_addr learned my addr 172.21.15.92:0/2977713453 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c26ffd640 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c277fe640 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c27fff640 1 -- 172.21.15.92:0/2977713453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 msgr2=0x55cc0a0ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c27fff640 1 -- 172.21.15.92:0/2977713453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 msgr2=0x55cc0a0a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.341+0000 7f9c27fff640 1 -- 172.21.15.92:0/2977713453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc0a0ae4d0 con 0x55cc09cf9070 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.342+0000 7f9c27fff640 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a94c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000f6d0 tx=0x7f9c20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7b2fb4030744c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.342+0000 7f9c0d7fa640 1 -- 172.21.15.92:0/2977713453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004b20 con 0x55cc09cf9070 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.342+0000 7f9c0d7fa640 1 -- 172.21.15.92:0/2977713453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c20004cc0 con 0x55cc09cf9070 2024-09-21T02:05:06.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.342+0000 7f9c0d7fa640 1 -- 172.21.15.92:0/2977713453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200286d0 con 0x55cc09cf9070 2024-09-21T02:05:06.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.343+0000 7f9c343de740 1 -- 172.21.15.92:0/2977713453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 msgr2=0x55cc0a0a94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.343+0000 7f9c343de740 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a94c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000f6d0 tx=0x7f9c20014000 comp rx=0 tx=0).stop 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 -- 172.21.15.92:0/2977713453 shutdown_connections 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a0a94c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd8390 0x55cc0a0a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 --2- 172.21.15.92:0/2977713453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd01b0 0x55cc0a0ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 -- 172.21.15.92:0/2977713453 >> 172.21.15.92:0/2977713453 conn(0x55cc09ced460 msgr2=0x55cc0a0ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 -- 172.21.15.92:0/2977713453 shutdown_connections 2024-09-21T02:05:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 -- 172.21.15.92:0/2977713453 wait complete. 2024-09-21T02:05:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.344+0000 7f9c343de740 1 -- start start 2024-09-21T02:05:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.345+0000 7f9c343de740 1 Processor -- start 2024-09-21T02:05:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.345+0000 7f9c343de740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd01b0 0x55cc0a11c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c343de740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a11cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c343de740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a11d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c343de740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b2c20 con 0x55cc09cf9070 2024-09-21T02:05:06.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a11cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c343de740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b2550 con 0x55cc09cd01b0 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c343de740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281af790 con 0x55cc09cd8390 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a11cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56808/0 (socket says 172.21.15.92:56808) 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c26ffd640 1 -- 172.21.15.92:0/4228104912 learned_addr learned my addr 172.21.15.92:0/4228104912 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a11d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c343de740 10 client.0 _ll_register_callbacks cb 0x7f9c20007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c277fe640 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd01b0 0x55cc0a11c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 -- 172.21.15.92:0/4228104912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 msgr2=0x55cc0a11cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a11cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 -- 172.21.15.92:0/4228104912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd01b0 msgr2=0x55cc0a11c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd01b0 0x55cc0a11c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 -- 172.21.15.92:0/4228104912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc0a1276d0 con 0x55cc09cf9070 2024-09-21T02:05:06.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.346+0000 7f9c27fff640 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a11d0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9c20007ec0 tx=0x7f9c200058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e972bce131e6ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:06.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:06.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c06ffd640 1 -- 172.21.15.92:0/4228104912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20003970 con 0x55cc09cf9070 2024-09-21T02:05:06.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c06ffd640 1 -- 172.21.15.92:0/4228104912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c20003b10 con 0x55cc09cf9070 2024-09-21T02:05:06.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c20004cc0 con 0x55cc09cf9070 2024-09-21T02:05:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c06ffd640 1 -- 172.21.15.92:0/4228104912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c20038070 con 0x55cc09cf9070 2024-09-21T02:05:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c343de740 10 client.15585 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.347+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc0a128910 con 0x55cc09cf9070 2024-09-21T02:05:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c06ffd640 1 -- 172.21.15.92:0/4228104912 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c20003d10 con 0x55cc09cf9070 2024-09-21T02:05:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c06ffd640 1 -- 172.21.15.92:0/4228104912 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c20030020 con 0x55cc09cf9070 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c343de740 10 client.15585 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c343de740 10 client.15585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c343de740 20 client.15585 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c343de740 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09c8cda0 0x55cc0a12af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c343de740 4 client.15585 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9c20004cc0 con 0x55cc09c8cda0 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.348+0000 7f9c277fe640 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09c8cda0 0x55cc0a12af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.349+0000 7f9c06ffd640 10 client.15585 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.349+0000 7f9c277fe640 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09c8cda0 0x55cc0a12af90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d2fe8b4d3ef3918 server_cookie=2098cec99dfd877b in_seq=0 out_seq=0 2024-09-21T02:05:06.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.349+0000 7f9c06ffd640 1 -- 172.21.15.92:0/4228104912 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9c20004cc0 con 0x55cc09c8cda0 2024-09-21T02:05:06.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.349+0000 7f9c06ffd640 10 client.15585 handle_command_reply: tid=0 2024-09-21T02:05:06.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.349+0000 7f9c343de740 1 client.15585 shutdown 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 msgr2=0x55cc0a11d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a11d0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9c20007ec0 tx=0x7f9c200058e0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 shutdown_connections 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55cc09c8cda0 0x55cc0a12af90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cc09cf9070 0x55cc0a11d0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc09cd8390 0x55cc0a11cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c06ffd640 0 client.15585 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 --2- 172.21.15.92:0/4228104912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc09cd01b0 0x55cc0a11c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 >> 172.21.15.92:0/4228104912 conn(0x55cc09ced460 msgr2=0x55cc0a0adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 shutdown_connections 2024-09-21T02:05:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 1 -- 172.21.15.92:0/4228104912 wait complete. 2024-09-21T02:05:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.352+0000 7f9c343de740 20 client.15585 trim_cache size 0 max 16384 2024-09-21T02:05:06.355 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:06.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c100b6650 msgr2=0x7f9c100b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c100b6650 0x7f9c100b8b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c20007fd0 tx=0x7f9c200076a0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 msgr2=0x7f9c2819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 0x7f9c2819ab20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d750 tx=0x7f9c1c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 shutdown_connections 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c100b6650 0x7f9c100b8b60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c2810ddf0 0x7f9c2819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c28105270 0x7f9c2819ab20 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d750 tx=0x7f9c1c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 --2- 172.21.15.92:0/2126502903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c28104840 0x7f9c2819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:06.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 >> 172.21.15.92:0/2126502903 conn(0x7f9c28079730 msgr2=0x7f9c281008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:06.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 shutdown_connections 2024-09-21T02:05:06.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:06.353+0000 7f9c2de6a640 1 -- 172.21.15.92:0/2126502903 wait complete. 2024-09-21T02:05:07.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:05:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:06 smithi136 ceph-mon[34754]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:05:07.290 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:07.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:06 smithi092 ceph-mon[31190]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:05:07.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:06 smithi099 ceph-mon[34404]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 -- 172.21.15.92:0/3498623954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f010de30 msgr2=0x7ff1f0110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3498623954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f010de30 0x7ff1f0110340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009ca0 tx=0x7ff1e802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 -- 172.21.15.92:0/3498623954 shutdown_connections 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3498623954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f010de30 0x7ff1f0110340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3498623954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f01052b0 0x7ff1f010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3498623954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1f0104880 0x7ff1f0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 -- 172.21.15.92:0/3498623954 >> 172.21.15.92:0/3498623954 conn(0x7ff1f0079750 msgr2=0x7ff1f0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.903+0000 7ff1f8232640 1 -- 172.21.15.92:0/3498623954 shutdown_connections 2024-09-21T02:05:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.904+0000 7ff1f8232640 1 -- 172.21.15.92:0/3498623954 wait complete. 2024-09-21T02:05:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.905+0000 7ff1f8232640 1 Processor -- start 2024-09-21T02:05:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.905+0000 7ff1f8232640 1 -- start start 2024-09-21T02:05:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f8232640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f0104880 0x7ff1f019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f8232640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 0x7ff1f019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f8232640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1f010de30 0x7ff1f019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f8232640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0113b00 con 0x7ff1f0104880 2024-09-21T02:05:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f8232640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0113980 con 0x7ff1f010de30 2024-09-21T02:05:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f8232640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0113c80 con 0x7ff1f01052b0 2024-09-21T02:05:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f57cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 0x7ff1f019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f57cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 0x7ff1f019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56824/0 (socket says 172.21.15.92:56824) 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3628758957 learned_addr learned my addr 172.21.15.92:0/3628758957 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1f010de30 0x7ff1f019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.906+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f0104880 0x7ff1f019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3628758957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1f010de30 msgr2=0x7ff1f019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1f010de30 0x7ff1f019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f0104880 msgr2=0x7ff1f019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f0104880 0x7ff1f019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3628758957 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f01a04c0 con 0x7ff1f01052b0 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f0104880 0x7ff1f019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.907+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 0x7ff1f019aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e970 tx=0x7ff1e000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.908+0000 7ff1e6ffd640 1 -- 172.21.15.92:0/3628758957 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000cd80 con 0x7ff1f01052b0 2024-09-21T02:05:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.908+0000 7ff1e6ffd640 1 -- 172.21.15.92:0/3628758957 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00035f0 con 0x7ff1f01052b0 2024-09-21T02:05:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.908+0000 7ff1e6ffd640 1 -- 172.21.15.92:0/3628758957 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0010690 con 0x7ff1f01052b0 2024-09-21T02:05:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.908+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f01a07b0 con 0x7ff1f01052b0 2024-09-21T02:05:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.908+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01a6f10 con 0x7ff1f01052b0 2024-09-21T02:05:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 Processor -- start 2024-09-21T02:05:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 -- start start 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119e3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558911959ea0 con 0x55891161b390 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5589119492d0 con 0x5589116131b0 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55891194d0e0 con 0x55891163c070 2024-09-21T02:05:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.912+0000 7ff1e6ffd640 1 -- 172.21.15.92:0/3628758957 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7ff1e0003aa0 con 0x7ff1f01052b0 2024-09-21T02:05:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.913+0000 7ff1f67cf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.913+0000 7ff1f67cf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56840/0 (socket says 172.21.15.92:56840) 2024-09-21T02:05:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.913+0000 7ff1f67cf640 1 -- 172.21.15.92:0/2537320650 learned_addr learned my addr 172.21.15.92:0/2537320650 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.913+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.913+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1e6ffd640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1dc0b6620 0x7ff1dc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1f67cf640 1 -- 172.21.15.92:0/2537320650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 msgr2=0x5589119e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1f67cf640 1 -- 172.21.15.92:0/2537320650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 msgr2=0x5589119e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1e6ffd640 1 -- 172.21.15.92:0/3628758957 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff1e00e76c0 con 0x7ff1f01052b0 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119e3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.914+0000 7ff1f67cf640 1 -- 172.21.15.92:0/2537320650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589116a7650 con 0x55891163c070 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.915+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:07.916 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.915+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1dc0b6620 0x7ff1dc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.915+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119e32c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009ca0 tx=0x7ff1e8009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce981d958f1f721e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.915+0000 7ff1dbfff640 1 -- 172.21.15.92:0/2537320650 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e804a070 con 0x55891163c070 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.915+0000 7ff1dbfff640 1 -- 172.21.15.92:0/2537320650 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8036030 con 0x55891163c070 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.915+0000 7ff1dbfff640 1 -- 172.21.15.92:0/2537320650 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8004510 con 0x55891163c070 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1dc0b6620 0x7ff1dc0b8b30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8047000 tx=0x7ff1e8004770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/2537320650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 msgr2=0x5589119e32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119e32c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009ca0 tx=0x7ff1e8009430 comp rx=0 tx=0).stop 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/2537320650 shutdown_connections 2024-09-21T02:05:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/2537320650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119e32c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.916+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/2537320650 >> 172.21.15.92:0/2537320650 conn(0x558911630460 msgr2=0x5589119e11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.917+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/2537320650 shutdown_connections 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.917+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/2537320650 wait complete. 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.917+0000 7ff1fe7a6740 1 -- start start 2024-09-21T02:05:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.917+0000 7ff1fe7a6740 1 Processor -- start 2024-09-21T02:05:07.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5589117aa700 con 0x55891161b390 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5589118e4160 con 0x5589116131b0 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5589119f5c80 con 0x55891163c070 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1f57cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1fe7a6740 10 client.0 _ll_register_callbacks cb 0x5589118fd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1f57cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39106/0 (socket says 172.21.15.92:39106) 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.918+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3601086611 learned_addr learned my addr 172.21.15.92:0/3601086611 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 msgr2=0x5589119f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3601086611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 msgr2=0x5589119f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f57cd640 1 -- 172.21.15.92:0/3601086611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558911911c90 con 0x55891161b390 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f3260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00ad600 tx=0x7ff1e00e5470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bad1018af43e0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3601086611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00add70 con 0x55891161b390 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e8036030 con 0x55891161b390 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.919+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3601086611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff1e00b6040 con 0x55891161b390 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3601086611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1e0096e60 con 0x55891161b390 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 10 client.15594 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558911a01930 con 0x55891161b390 2024-09-21T02:05:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3601086611 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8036030 con 0x55891161b390 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3601086611 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff1e00e7110 con 0x55891161b390 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 10 client.15594 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 10 client.15594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 20 client.15594 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115fa9f0 0x558911a05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 4 client.15594 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.920+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891195bdf0 con 0x5589115fa9f0 2024-09-21T02:05:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.921+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115fa9f0 0x558911a05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.921+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115fa9f0 0x558911a05330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26a80ff7acc0eefe server_cookie=b6cf29545a0af8a2 in_seq=0 out_seq=0 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.921+0000 7ff1d97fa640 10 client.15594 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3601086611 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55891195bdf0 con 0x5589115fa9f0 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1d97fa640 10 client.15594 handle_command_reply: tid=0 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1fe7a6740 1 client.15594 shutdown 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 msgr2=0x5589119f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f3260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00ad600 tx=0x7ff1e00e5470 comp rx=0 tx=0).stop 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 shutdown_connections 2024-09-21T02:05:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.922+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f3260 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115fa9f0 0x558911a05330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3601086611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 >> 172.21.15.92:0/3601086611 conn(0x558911630460 msgr2=0x5589119e4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1d97fa640 0 client.15594 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 shutdown_connections 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3601086611 wait complete. 2024-09-21T02:05:07.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.923+0000 7ff1fe7a6740 20 client.15594 trim_cache size 0 max 16384 2024-09-21T02:05:07.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 Processor -- start 2024-09-21T02:05:07.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 -- start start 2024-09-21T02:05:07.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01ad210 con 0x55891161b390 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01aed50 con 0x5589116131b0 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01ad8e0 con 0x55891163c070 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1f57cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1f57cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39112/0 (socket says 172.21.15.92:39112) 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1f57cd640 1 -- 172.21.15.92:0/1958575331 learned_addr learned my addr 172.21.15.92:0/1958575331 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.949+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f57cd640 1 -- 172.21.15.92:0/1958575331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 msgr2=0x5589119f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f57cd640 1 -- 172.21.15.92:0/1958575331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 msgr2=0x5589119f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f57cd640 1 -- 172.21.15.92:0/1958575331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589119f3b30 con 0x55891161b390 2024-09-21T02:05:07.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.950+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f0120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0003e20 tx=0x7ff1e00ac620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73137d31bcb484bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.951+0000 7ff1db7fe640 1 -- 172.21.15.92:0/1958575331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00acde0 con 0x55891161b390 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.951+0000 7ff1db7fe640 1 -- 172.21.15.92:0/1958575331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00add70 con 0x55891161b390 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.951+0000 7ff1db7fe640 1 -- 172.21.15.92:0/1958575331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00e7110 con 0x55891161b390 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/1958575331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 msgr2=0x5589119f0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f0120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0003e20 tx=0x7ff1e00ac620 comp rx=0 tx=0).stop 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/1958575331 shutdown_connections 2024-09-21T02:05:07.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55891163c070 0x5589119f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f0120 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/1958575331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589116131b0 0x5589119f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/1958575331 >> 172.21.15.92:0/1958575331 conn(0x558911630460 msgr2=0x5589119f2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/1958575331 shutdown_connections 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/1958575331 wait complete. 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 -- start start 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.952+0000 7ff1fe7a6740 1 Processor -- start 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 0x5589119f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55891163c070 0x558911a67dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01ae680 con 0x55891161b390 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01adfb0 con 0x55891163c070 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01ab210 con 0x5589116131b0 2024-09-21T02:05:07.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1fe7a6740 10 client.0 _ll_register_callbacks cb 0x7ff1e00a9510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1f5fce640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 0x5589119f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1f5fce640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 0x5589119f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56866/0 (socket says 172.21.15.92:56866) 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1f5fce640 1 -- 172.21.15.92:0/3199464849 learned_addr learned my addr 172.21.15.92:0/3199464849 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.953+0000 7ff1f67cf640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55891163c070 0x558911a67dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1f5fce640 1 -- 172.21.15.92:0/3199464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55891163c070 msgr2=0x558911a67dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55891163c070 0x558911a67dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1f5fce640 1 -- 172.21.15.92:0/3199464849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 msgr2=0x5589119f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1f5fce640 1 -- 172.21.15.92:0/3199464849 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558911a68440 con 0x5589116131b0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1f5fce640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 0x5589119f69b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x558911a05ad0 tx=0x7ff1ec013980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70ea9936b064819 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.954+0000 7ff1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3199464849 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec02a060 con 0x5589116131b0 2024-09-21T02:05:07.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e00add70 con 0x5589116131b0 2024-09-21T02:05:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3199464849 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff1ec0042f0 con 0x5589116131b0 2024-09-21T02:05:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3199464849 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec021040 con 0x5589116131b0 2024-09-21T02:05:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1fe7a6740 10 client.25162 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558911a6f890 con 0x5589116131b0 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.955+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3199464849 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00add70 con 0x5589116131b0 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3199464849 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff1ec01f5e0 con 0x5589116131b0 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1fe7a6740 10 client.25162 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1fe7a6740 10 client.25162 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1fe7a6740 20 client.25162 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115cfda0 0x558911a72060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1fe7a6740 4 client.25162 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589119578a0 con 0x5589115cfda0 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.956+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115cfda0 0x558911a72060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1d97fa640 10 client.25162 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1f57cd640 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115cfda0 0x558911a72060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=645f3021b928caa server_cookie=7916d1c8adf4438e in_seq=0 out_seq=0 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1d97fa640 1 -- 172.21.15.92:0/3199464849 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589119578a0 con 0x5589115cfda0 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1d97fa640 10 client.25162 handle_command_reply: tid=0 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1fe7a6740 1 client.25162 shutdown 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 msgr2=0x5589119f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 0x5589119f69b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x558911a05ad0 tx=0x7ff1ec013980 comp rx=0 tx=0).stop 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 shutdown_connections 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.957+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589115cfda0 0x558911a72060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1d97fa640 0 client.25162 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55891163c070 0x558911a67dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55891161b390 0x5589119f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 1 --2- 172.21.15.92:0/3199464849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589116131b0 0x5589119f69b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 >> 172.21.15.92:0/3199464849 conn(0x558911630460 msgr2=0x5589119f0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 shutdown_connections 2024-09-21T02:05:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 1 -- 172.21.15.92:0/3199464849 wait complete. 2024-09-21T02:05:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1fe7a6740 20 client.25162 trim_cache size 0 max 16384 2024-09-21T02:05:07.962 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1dc0b6620 msgr2=0x7ff1dc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1dc0b6620 0x7ff1dc0b8b30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8047000 tx=0x7ff1e8004770 comp rx=0 tx=0).stop 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 msgr2=0x7ff1f019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.958+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 0x7ff1f019aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e970 tx=0x7ff1e000ee30 comp rx=0 tx=0).stop 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 shutdown_connections 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff1dc0b6620 0x7ff1dc0b8b30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1f010de30 0x7ff1f019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1f01052b0 0x7ff1f019aaf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 --2- 172.21.15.92:0/3628758957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1f0104880 0x7ff1f019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 >> 172.21.15.92:0/3628758957 conn(0x7ff1f0079750 msgr2=0x7ff1f00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 shutdown_connections 2024-09-21T02:05:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:07.959+0000 7ff1f8232640 1 -- 172.21.15.92:0/3628758957 wait complete. 2024-09-21T02:05:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 -- 172.21.15.92:0/4203243773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa354103fd0 msgr2=0x7fa35410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 --2- 172.21.15.92:0/4203243773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa354103fd0 0x7fa35410a970 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009ca0 tx=0x7fa34c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 -- 172.21.15.92:0/4203243773 shutdown_connections 2024-09-21T02:05:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 --2- 172.21.15.92:0/4203243773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa354103fd0 0x7fa35410a970 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 --2- 172.21.15.92:0/4203243773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354103630 0x7fa354103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 --2- 172.21.15.92:0/4203243773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3541025b0 0x7fa354102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 -- 172.21.15.92:0/4203243773 >> 172.21.15.92:0/4203243773 conn(0x7fa3540fdf30 msgr2=0x7fa3541003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 -- 172.21.15.92:0/4203243773 shutdown_connections 2024-09-21T02:05:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.155+0000 7fa3594bd640 1 -- 172.21.15.92:0/4203243773 wait complete. 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 Processor -- start 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 -- start start 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 0x7fa354198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354103630 0x7fa3541994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa354103fd0 0x7fa35419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa35410e1f0 con 0x7fa3541025b0 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa35410e070 con 0x7fa354103fd0 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa3594bd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa35410e370 con 0x7fa354103630 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 0x7fa354198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 0x7fa354198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39128/0 (socket says 172.21.15.92:39128) 2024-09-21T02:05:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.156+0000 7fa352ffd640 1 -- 172.21.15.92:0/729093933 learned_addr learned my addr 172.21.15.92:0/729093933 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa3527fc640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354103630 0x7fa3541994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa3537fe640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa354103fd0 0x7fa35419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa352ffd640 1 -- 172.21.15.92:0/729093933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354103630 msgr2=0x7fa3541994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa352ffd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354103630 0x7fa3541994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa352ffd640 1 -- 172.21.15.92:0/729093933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa354103fd0 msgr2=0x7fa35419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa352ffd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa354103fd0 0x7fa35419a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa352ffd640 1 -- 172.21.15.92:0/729093933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35419a880 con 0x7fa3541025b0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa352ffd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 0x7fa354198e40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa34400ea60 tx=0x7fa34400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa33bfff640 1 -- 172.21.15.92:0/729093933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34400cde0 con 0x7fa3541025b0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35419ab70 con 0x7fa3541025b0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa33bfff640 1 -- 172.21.15.92:0/729093933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3440035f0 con 0x7fa3541025b0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541a1260 con 0x7fa3541025b0 2024-09-21T02:05:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.157+0000 7fa33bfff640 1 -- 172.21.15.92:0/729093933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344010690 con 0x7fa3541025b0 2024-09-21T02:05:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.160+0000 7fa33bfff640 1 -- 172.21.15.92:0/729093933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fa3440037a0 con 0x7fa3541025b0 2024-09-21T02:05:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 Processor -- start 2024-09-21T02:05:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa33bfff640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa33c0b65a0 0x7fa33c0b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 -- start start 2024-09-21T02:05:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa33bfff640 1 -- 172.21.15.92:0/729093933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa3440e78f0 con 0x7fa3541025b0 2024-09-21T02:05:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981bb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897df24d0 0x5638981bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56389812d6f0 con 0x563897e131b0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56389811cb20 con 0x563897dea2f0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa36042d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563898120930 con 0x563897df24d0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981bae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39140/0 (socket says 172.21.15.92:39140) 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa352ffd640 1 -- 172.21.15.92:0/2861691615 learned_addr learned my addr 172.21.15.92:0/2861691615 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.161+0000 7fa3537fe640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa33c0b65a0 0x7fa33c0b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa3527fc640 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa3537fe640 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897df24d0 0x5638981bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa352ffd640 1 -- 172.21.15.92:0/2861691615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897df24d0 msgr2=0x5638981bbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa352ffd640 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897df24d0 0x5638981bbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa352ffd640 1 -- 172.21.15.92:0/2861691615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 msgr2=0x5638981bb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa352ffd640 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981bb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa352ffd640 1 -- 172.21.15.92:0/2861691615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638980e2530 con 0x563897e131b0 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa352ffd640 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981bae10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa3440a9ed0 tx=0x7fa3440ad790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36205c7a34ae25bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa3537fe640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa33c0b65a0 0x7fa33c0b8ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa34c004af0 tx=0x7fa34c031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa3397fa640 1 -- 172.21.15.92:0/2861691615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344092070 con 0x563897e131b0 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa3397fa640 1 -- 172.21.15.92:0/2861691615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3440ad410 con 0x563897e131b0 2024-09-21T02:05:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.162+0000 7fa3397fa640 1 -- 172.21.15.92:0/2861691615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344020390 con 0x563897e131b0 2024-09-21T02:05:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.163+0000 7fa36042d740 1 -- 172.21.15.92:0/2861691615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 msgr2=0x5638981bae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.163+0000 7fa36042d740 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981bae10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa3440a9ed0 tx=0x7fa3440ad790 comp rx=0 tx=0).stop 2024-09-21T02:05:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.163+0000 7fa36042d740 1 -- 172.21.15.92:0/2861691615 shutdown_connections 2024-09-21T02:05:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.164+0000 7fa36042d740 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897df24d0 0x5638981bbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.164+0000 7fa36042d740 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981bb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.164+0000 7fa36042d740 1 --2- 172.21.15.92:0/2861691615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981bae10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.164+0000 7fa36042d740 1 -- 172.21.15.92:0/2861691615 >> 172.21.15.92:0/2861691615 conn(0x563897e07300 msgr2=0x5638981b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.164+0000 7fa36042d740 1 -- 172.21.15.92:0/2861691615 shutdown_connections 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.164+0000 7fa36042d740 1 -- 172.21.15.92:0/2861691615 wait complete. 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.165+0000 7fa36042d740 1 -- start start 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.165+0000 7fa36042d740 1 Processor -- start 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.165+0000 7fa36042d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.165+0000 7fa36042d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 0x5638981c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.165+0000 7fa36042d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981c5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa36042d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563898054ab0 con 0x563897df24d0 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa3527fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 0x5638981c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa3527fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 0x5638981c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39156/0 (socket says 172.21.15.92:39156) 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa3527fc640 1 -- 172.21.15.92:0/3667120638 learned_addr learned my addr 172.21.15.92:0/3667120638 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5638980c2730 con 0x563897dea2f0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa3537fe640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5638981bcaf0 con 0x563897e131b0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa36042d740 10 client.0 _ll_register_callbacks cb 0x5638981166b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 -- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 msgr2=0x5638981c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 -- 172.21.15.92:0/3667120638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 msgr2=0x5638981c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 0x5638981c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 -- 172.21.15.92:0/3667120638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563897e7e790 con 0x563897dea2f0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa3527fc640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 0x5638981c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3440ad410 con 0x563897dea2f0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa352ffd640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981c4680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x5638981bc6c0 tx=0x7fa344004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73943247be931f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.166+0000 7fa336ff5640 10 client.25133 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:05:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa336ff5640 1 -- 172.21.15.92:0/3667120638 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344092070 con 0x563897dea2f0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa336ff5640 1 -- 172.21.15.92:0/3667120638 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa344093260 con 0x563897dea2f0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa336ff5640 1 -- 172.21.15.92:0/3667120638 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa344020590 con 0x563897dea2f0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa36042d740 10 client.25133 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=29}) -- 0x563897ef7620 con 0x563897dea2f0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa336ff5640 1 -- 172.21.15.92:0/3667120638 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa3440acb60 con 0x563897dea2f0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa36042d740 10 client.25133 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.167+0000 7fa36042d740 10 client.25133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.168+0000 7fa36042d740 20 client.25133 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.168+0000 7fa36042d740 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897dd1d80 0x5638981d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.168+0000 7fa36042d740 4 client.25133 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.168+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56389812f640 con 0x563897dd1d80 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.168+0000 7fa3527fc640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897dd1d80 0x5638981d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.169+0000 7fa3527fc640 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897dd1d80 0x5638981d17c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a863d3e3c24ba5e server_cookie=b688c6f5415bebe4 in_seq=0 out_seq=0 2024-09-21T02:05:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.169+0000 7fa336ff5640 10 client.25133 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.169+0000 7fa336ff5640 1 -- 172.21.15.92:0/3667120638 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56389812f640 con 0x563897dd1d80 2024-09-21T02:05:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.169+0000 7fa336ff5640 10 client.25133 handle_command_reply: tid=0 2024-09-21T02:05:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.169+0000 7fa36042d740 1 client.25133 shutdown 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 msgr2=0x5638981c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981c4680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x5638981bc6c0 tx=0x7fa344004560 comp rx=0 tx=0).stop 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 shutdown_connections 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897dd1d80 0x5638981d17c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897df24d0 0x5638981c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 --2- 172.21.15.92:0/3667120638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897dea2f0 0x5638981c4680 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 >> 172.21.15.92:0/3667120638 conn(0x563897e07300 msgr2=0x5638981b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa336ff5640 0 client.25133 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 shutdown_connections 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 1 -- 172.21.15.92:0/3667120638 wait complete. 2024-09-21T02:05:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.170+0000 7fa36042d740 20 client.25133 trim_cache size 0 max 16384 2024-09-21T02:05:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.198+0000 7fa36042d740 1 Processor -- start 2024-09-21T02:05:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.198+0000 7fa36042d740 1 -- start start 2024-09-21T02:05:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.198+0000 7fa36042d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897dea2f0 0x5638981c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.198+0000 7fa36042d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.198+0000 7fa36042d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa36042d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541a8ec0 con 0x563897e131b0 2024-09-21T02:05:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa36042d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541aaa00 con 0x563897df24d0 2024-09-21T02:05:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa36042d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541a9590 con 0x563897dea2f0 2024-09-21T02:05:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39164/0 (socket says 172.21.15.92:39164) 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 -- 172.21.15.92:0/4112020070 learned_addr learned my addr 172.21.15.92:0/4112020070 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa352ffd640 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897dea2f0 0x5638981c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3527fc640 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 -- 172.21.15.92:0/4112020070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897dea2f0 msgr2=0x5638981c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897dea2f0 0x5638981c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 -- 172.21.15.92:0/4112020070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 msgr2=0x5638981b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 -- 172.21.15.92:0/4112020070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638981b0ba0 con 0x563897e131b0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa3537fe640 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981b5bf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0093b0 tx=0x7fa34c040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a906a269410bf87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa338ff9640 1 -- 172.21.15.92:0/4112020070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c051070 con 0x563897e131b0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.199+0000 7fa338ff9640 1 -- 172.21.15.92:0/4112020070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c036040 con 0x563897e131b0 2024-09-21T02:05:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.200+0000 7fa338ff9640 1 -- 172.21.15.92:0/4112020070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c02f880 con 0x563897e131b0 2024-09-21T02:05:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 -- 172.21.15.92:0/4112020070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 msgr2=0x5638981b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981b5bf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0093b0 tx=0x7fa34c040070 comp rx=0 tx=0).stop 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 -- 172.21.15.92:0/4112020070 shutdown_connections 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897e131b0 0x5638981b5bf0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 --2- 172.21.15.92:0/4112020070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897dea2f0 0x5638981c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 -- 172.21.15.92:0/4112020070 >> 172.21.15.92:0/4112020070 conn(0x563897e07300 msgr2=0x5638981b61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 -- 172.21.15.92:0/4112020070 shutdown_connections 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 -- 172.21.15.92:0/4112020070 wait complete. 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 -- start start 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.201+0000 7fa36042d740 1 Processor -- start 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 0x5638981c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 0x5638981c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 0x5638981c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39168/0 (socket says 172.21.15.92:39168) 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541aa330 con 0x563897dea2f0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 -- 172.21.15.92:0/4049694893 learned_addr learned my addr 172.21.15.92:0/4049694893 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541a9c60 con 0x563897df24d0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541a6ea0 con 0x563897e131b0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa36042d740 10 client.0 _ll_register_callbacks cb 0x7fa3440a97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa3527fc640 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 -- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 msgr2=0x5638981ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 -- 172.21.15.92:0/4049694893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 msgr2=0x5638981c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 -- 172.21.15.92:0/4049694893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638981cb000 con 0x563897dea2f0 2024-09-21T02:05:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa352ffd640 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 0x5638981c96c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa3440ad890 tx=0x7fa3440d10d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f143ab9d5e8967c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.202+0000 7fa336ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa336ff5640 1 -- 172.21.15.92:0/4049694893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344092040 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa336ff5640 1 -- 172.21.15.92:0/4049694893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa344093470 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c036040 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa336ff5640 1 -- 172.21.15.92:0/4049694893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3440b21d0 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa36042d740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638981cb000 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa336ff5640 1 -- 172.21.15.92:0/4049694893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c036040 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa336ff5640 1 -- 172.21.15.92:0/4049694893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa3440c3ab0 con 0x563897dea2f0 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.203+0000 7fa36042d740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.204+0000 7fa36042d740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.204+0000 7fa36042d740 20 client.15618 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.204+0000 7fa36042d740 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897da71c0 0x56389823e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.204+0000 7fa36042d740 4 client.15618 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:05:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.204+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56389812b0f0 con 0x563897da71c0 2024-09-21T02:05:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.204+0000 7fa3527fc640 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897da71c0 0x56389823e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.205+0000 7fa336ff5640 10 client.15618 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.205+0000 7fa3527fc640 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897da71c0 0x56389823e0f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cbff7abeb73138 server_cookie=75e74ebff22ac4af in_seq=0 out_seq=0 2024-09-21T02:05:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.212+0000 7fa336ff5640 1 -- 172.21.15.92:0/4049694893 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56389812b0f0 con 0x563897da71c0 2024-09-21T02:05:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.212+0000 7fa336ff5640 10 client.15618 handle_command_reply: tid=0 2024-09-21T02:05:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.212+0000 7fa36042d740 1 client.15618 shutdown 2024-09-21T02:05:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 msgr2=0x5638981c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 0x5638981c96c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa3440ad890 tx=0x7fa3440d10d0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 shutdown_connections 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563897da71c0 0x56389823e0f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa336ff5640 0 client.15618 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563897e131b0 0x5638981ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563897df24d0 0x5638981c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 --2- 172.21.15.92:0/4049694893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563897dea2f0 0x5638981c96c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 >> 172.21.15.92:0/4049694893 conn(0x563897e07300 msgr2=0x5638981c41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 shutdown_connections 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 1 -- 172.21.15.92:0/4049694893 wait complete. 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.213+0000 7fa36042d740 20 client.15618 trim_cache size 0 max 16384 2024-09-21T02:05:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:05:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:05:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:05:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa33c0b65a0 msgr2=0x7fa33c0b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa33c0b65a0 0x7fa33c0b8ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa34c004af0 tx=0x7fa34c031af0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 msgr2=0x7fa354198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 0x7fa354198e40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa34400ea60 tx=0x7fa34400ef20 comp rx=0 tx=0).stop 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 shutdown_connections 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa33c0b65a0 0x7fa33c0b8ab0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa354103fd0 0x7fa35419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa354103630 0x7fa3541994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 --2- 172.21.15.92:0/729093933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3541025b0 0x7fa354198e40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 >> 172.21.15.92:0/729093933 conn(0x7fa3540fdf30 msgr2=0x7fa354106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 shutdown_connections 2024-09-21T02:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:08.214+0000 7fa3594bd640 1 -- 172.21.15.92:0/729093933 wait complete. 2024-09-21T02:05:08.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:05:08.969 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:08 smithi092 ceph-mon[31190]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T02:05:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:08 smithi136 ceph-mon[34754]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T02:05:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:08 smithi099 ceph-mon[34404]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 -- 172.21.15.92:0/2614455678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 msgr2=0x7f6f2c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 --2- 172.21.15.92:0/2614455678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c1098c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009900 tx=0x7f6f2002a410 comp rx=0 tx=0).stop 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 -- 172.21.15.92:0/2614455678 shutdown_connections 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 --2- 172.21.15.92:0/2614455678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 0x7f6f2c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 --2- 172.21.15.92:0/2614455678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c1098c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 --2- 172.21.15.92:0/2614455678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f2c101f70 0x7f6f2c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 -- 172.21.15.92:0/2614455678 >> 172.21.15.92:0/2614455678 conn(0x7f6f2c077c60 msgr2=0x7f6f2c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 -- 172.21.15.92:0/2614455678 shutdown_connections 2024-09-21T02:05:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.521+0000 7f6f327c6640 1 -- 172.21.15.92:0/2614455678 wait complete. 2024-09-21T02:05:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.522+0000 7f6f327c6640 1 Processor -- start 2024-09-21T02:05:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.522+0000 7f6f327c6640 1 -- start start 2024-09-21T02:05:09.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f327c6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f2c101f70 0x7f6f2c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f327c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f327c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 0x7f6f2c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f327c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c113ae0 con 0x7f6f2c1094b0 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f327c6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c113960 con 0x7f6f2c109e00 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f327c6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c113c60 con 0x7f6f2c101f70 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f2b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f30d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 0x7f6f2c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f2b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39192/0 (socket says 172.21.15.92:39192) 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f2b7fe640 1 -- 172.21.15.92:0/4108375200 learned_addr learned my addr 172.21.15.92:0/4108375200 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.523+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f2c101f70 0x7f6f2c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f30d63640 1 -- 172.21.15.92:0/4108375200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f2c101f70 msgr2=0x7f6f2c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f30d63640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f2c101f70 0x7f6f2c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f30d63640 1 -- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 msgr2=0x7f6f2c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f30d63640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f30d63640 1 -- 172.21.15.92:0/4108375200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2c1a04c0 con 0x7f6f2c109e00 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f30d63640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 0x7f6f2c19fd90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f24007440 tx=0x7f6f24007840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f297fa640 1 -- 172.21.15.92:0/4108375200 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24012c70 con 0x7f6f2c109e00 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2c1a0750 con 0x7f6f2c109e00 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.524+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f2c102df0 con 0x7f6f2c109e00 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.525+0000 7f6f297fa640 1 -- 172.21.15.92:0/4108375200 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f24012e10 con 0x7f6f2c109e00 2024-09-21T02:05:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.525+0000 7f6f297fa640 1 -- 172.21.15.92:0/4108375200 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2402b060 con 0x7f6f2c109e00 2024-09-21T02:05:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.528+0000 7f6f38d3a740 1 Processor -- start 2024-09-21T02:05:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.528+0000 7f6f297fa640 1 -- 172.21.15.92:0/4108375200 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f6f2401d9a0 con 0x7f6f2c109e00 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.528+0000 7f6f38d3a740 1 -- start start 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77c7070 0x55ffe7b6e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b6ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77a6390 0x55ffe7b6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7ae4ea0 con 0x55ffe779e1b0 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7ad42d0 con 0x55ffe77a6390 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7ad80e0 con 0x55ffe77c7070 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b6ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b6ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39196/0 (socket says 172.21.15.92:39196) 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1693542663 learned_addr learned my addr 172.21.15.92:0/1693542663 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77c7070 0x55ffe7b6e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77a6390 0x55ffe7b6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.529+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1693542663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77c7070 msgr2=0x55ffe7b6e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77c7070 0x55ffe7b6e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1693542663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77a6390 msgr2=0x55ffe7b6f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77a6390 0x55ffe7b6f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1693542663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe7832650 con 0x55ffe779e1b0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77a6390 0x55ffe7b6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b6ebe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009a70 tx=0x7f6f1c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c37bc7ead91d84f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f297fa640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f140b66a0 0x7f6f140b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f297fa640 1 -- 172.21.15.92:0/4108375200 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6f240eee90 con 0x7f6f2c109e00 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.530+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f140b66a0 0x7f6f140b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.531+0000 7f6f127fc640 1 -- 172.21.15.92:0/1693542663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c017070 con 0x55ffe779e1b0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.531+0000 7f6f127fc640 1 -- 172.21.15.92:0/1693542663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1c013470 con 0x55ffe779e1b0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.531+0000 7f6f127fc640 1 -- 172.21.15.92:0/1693542663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c012630 con 0x55ffe779e1b0 2024-09-21T02:05:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.531+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1693542663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 msgr2=0x55ffe7b6ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.531+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b6ebe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009a70 tx=0x7f6f1c009f30 comp rx=0 tx=0).stop 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1693542663 shutdown_connections 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77a6390 0x55ffe7b6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b6ebe0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1693542663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77c7070 0x55ffe7b6e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1693542663 >> 172.21.15.92:0/1693542663 conn(0x55ffe77bb460 msgr2=0x55ffe7b6c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f140b66a0 0x7f6f140b8bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c109cf0 tx=0x7f6f20009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1693542663 shutdown_connections 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1693542663 wait complete. 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.532+0000 7f6f38d3a740 1 -- start start 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 Processor -- start 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe779e1b0 0x55ffe7b79d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe77c7070 0x55ffe7b861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7935700 con 0x55ffe77c7070 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7a6f160 con 0x55ffe779e1b0 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7b7ce80 con 0x55ffe77a6390 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f38d3a740 10 client.0 _ll_register_callbacks cb 0x55ffe7a889e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f2b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f2b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:56938/0 (socket says 172.21.15.92:56938) 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe779e1b0 0x55ffe7b79d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f2b7fe640 1 -- 172.21.15.92:0/3659914272 learned_addr learned my addr 172.21.15.92:0/3659914272 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.533+0000 7f6f30d63640 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe77c7070 0x55ffe7b861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f2b7fe640 1 -- 172.21.15.92:0/3659914272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe779e1b0 msgr2=0x55ffe7b79d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe779e1b0 0x55ffe7b79d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f2b7fe640 1 -- 172.21.15.92:0/3659914272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe77c7070 msgr2=0x55ffe7b861a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe77c7070 0x55ffe7b861a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f2b7fe640 1 -- 172.21.15.92:0/3659914272 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe7a9cc90 con 0x55ffe77a6390 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7a2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f2002a930 tx=0x7f6f20009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ddccf153fee271 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/3659914272 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2004e070 con 0x55ffe77a6390 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/3659914272 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6f20005020 con 0x55ffe77a6390 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.534+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f1c013470 con 0x55ffe77a6390 2024-09-21T02:05:09.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/3659914272 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f20035040 con 0x55ffe77a6390 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 10 client.25186 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffe7b88b80 con 0x55ffe77a6390 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/3659914272 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1c013470 con 0x55ffe77a6390 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/3659914272 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f200035f0 con 0x55ffe77a6390 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 10 client.25186 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 10 client.25186 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 20 client.25186 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe77859f0 0x55ffe7b8c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 4 client.25186 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe7ae6df0 con 0x55ffe77859f0 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.535+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe77859f0 0x55ffe7b8c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.536+0000 7f6f0b7fe640 10 client.25186 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:09.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.536+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe77859f0 0x55ffe7b8c020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec8095f3a9eab79d server_cookie=17c17382dc07c6e3 in_seq=0 out_seq=0 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/3659914272 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffe7ae6df0 con 0x55ffe77859f0 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f0b7fe640 10 client.25186 handle_command_reply: tid=0 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 client.25186 shutdown 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 msgr2=0x55ffe7b7a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7a2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f2002a930 tx=0x7f6f20009260 comp rx=0 tx=0).stop 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 shutdown_connections 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe77859f0 0x55ffe7b8c020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe77c7070 0x55ffe7b861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7a2c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f0b7fe640 0 client.25186 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/3659914272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe779e1b0 0x55ffe7b79d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 >> 172.21.15.92:0/3659914272 conn(0x55ffe77bb460 msgr2=0x55ffe7b71830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 shutdown_connections 2024-09-21T02:05:09.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 1 -- 172.21.15.92:0/3659914272 wait complete. 2024-09-21T02:05:09.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.537+0000 7f6f38d3a740 20 client.25186 trim_cache size 0 max 16384 2024-09-21T02:05:09.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.550+0000 7f6f38d3a740 1 Processor -- start 2024-09-21T02:05:09.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.550+0000 7f6f38d3a740 1 -- start start 2024-09-21T02:05:09.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b6fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c0732f0 con 0x55ffe779e1b0 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c074e30 con 0x55ffe77c7070 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c0739c0 con 0x55ffe77a6390 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39214/0 (socket says 172.21.15.92:39214) 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1408572970 learned_addr learned my addr 172.21.15.92:0/1408572970 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b6fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f30d63640 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1408572970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 msgr2=0x55ffe7b6fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b6fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1408572970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 msgr2=0x55ffe7b860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.551+0000 7f6f2bfff640 1 -- 172.21.15.92:0/1408572970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe7b6cf00 con 0x55ffe779e1b0 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.552+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b6fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:09.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.552+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7bc60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c010010 tx=0x7f6f1c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c9ff94fe2fe9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.552+0000 7f6f11ffb640 1 -- 172.21.15.92:0/1408572970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c02a070 con 0x55ffe779e1b0 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.552+0000 7f6f11ffb640 1 -- 172.21.15.92:0/1408572970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1c013920 con 0x55ffe779e1b0 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.552+0000 7f6f11ffb640 1 -- 172.21.15.92:0/1408572970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c017040 con 0x55ffe779e1b0 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1408572970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 msgr2=0x55ffe7b7bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7bc60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c010010 tx=0x7f6f1c01f040 comp rx=0 tx=0).stop 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1408572970 shutdown_connections 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b6fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/1408572970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7bc60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.553+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1408572970 >> 172.21.15.92:0/1408572970 conn(0x55ffe77bb460 msgr2=0x55ffe7b6f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.554+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1408572970 shutdown_connections 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.554+0000 7f6f38d3a740 1 -- 172.21.15.92:0/1408572970 wait complete. 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.554+0000 7f6f38d3a740 1 -- start start 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.554+0000 7f6f38d3a740 1 Processor -- start 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b7eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c074760 con 0x55ffe779e1b0 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c074090 con 0x55ffe77c7070 2024-09-21T02:05:09.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c0712d0 con 0x55ffe77a6390 2024-09-21T02:05:09.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f38d3a740 10 client.0 _ll_register_callbacks cb 0x7f6f20005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:09.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b7eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b7eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:39090/0 (socket says 172.21.15.92:39090) 2024-09-21T02:05:09.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 -- 172.21.15.92:0/2740214158 learned_addr learned my addr 172.21.15.92:0/2740214158 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f2b7fe640 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 -- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 msgr2=0x55ffe7b7e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 -- 172.21.15.92:0/2740214158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 msgr2=0x55ffe7b7e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.555+0000 7f6f30d63640 1 -- 172.21.15.92:0/2740214158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe7bf5a10 con 0x55ffe77c7070 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f30d63640 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b7eb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f240b1410 tx=0x7f6f240ecd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150304619e1d7fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/2740214158 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240b7070 con 0x55ffe77c7070 2024-09-21T02:05:09.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/2740214158 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6f240dc170 con 0x55ffe77c7070 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f1c013920 con 0x55ffe77c7070 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/2740214158 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f240b9920 con 0x55ffe77c7070 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f38d3a740 10 client.25157 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.556+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffe7bf6c90 con 0x55ffe77c7070 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/2740214158 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f240d6e50 con 0x55ffe77c7070 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/2740214158 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f240b3b10 con 0x55ffe77c7070 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f38d3a740 10 client.25157 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f38d3a740 10 client.25157 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f38d3a740 20 client.25157 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe775ada0 0x55ffe7bf9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f38d3a740 4 client.25157 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.557+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6f1c013920 con 0x55ffe775ada0 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.558+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe775ada0 0x55ffe7bf9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.558+0000 7f6f0b7fe640 10 client.25157 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.558+0000 7f6f2bfff640 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe775ada0 0x55ffe7bf9480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb4c01058881effe server_cookie=f0fc01681d946963 in_seq=0 out_seq=0 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.558+0000 7f6f0b7fe640 1 -- 172.21.15.92:0/2740214158 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6f1c013920 con 0x55ffe775ada0 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.558+0000 7f6f0b7fe640 10 client.25157 handle_command_reply: tid=0 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.558+0000 7f6f38d3a740 1 client.25157 shutdown 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 msgr2=0x55ffe7b7eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b7eb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f240b1410 tx=0x7f6f240ecd00 comp rx=0 tx=0).stop 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 shutdown_connections 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ffe775ada0 0x55ffe7bf9480 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ffe77c7070 0x55ffe7b7eb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffe77a6390 0x55ffe7b7e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 --2- 172.21.15.92:0/2740214158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffe779e1b0 0x55ffe7b7e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 >> 172.21.15.92:0/2740214158 conn(0x55ffe77bb460 msgr2=0x55ffe7b7b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f0b7fe640 0 client.25157 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 shutdown_connections 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 1 -- 172.21.15.92:0/2740214158 wait complete. 2024-09-21T02:05:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.559+0000 7f6f38d3a740 20 client.25157 trim_cache size 0 max 16384 2024-09-21T02:05:09.565 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f140b66a0 msgr2=0x7f6f140b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f140b66a0 0x7f6f140b8bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c109cf0 tx=0x7f6f20009560 comp rx=0 tx=0).stop 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 msgr2=0x7f6f2c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 0x7f6f2c19fd90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f24007440 tx=0x7f6f24007840 comp rx=0 tx=0).stop 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 shutdown_connections 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6f140b66a0 0x7f6f140b8bb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f2c109e00 0x7f6f2c19fd90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f2c1094b0 0x7f6f2c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 --2- 172.21.15.92:0/4108375200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f2c101f70 0x7f6f2c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 >> 172.21.15.92:0/4108375200 conn(0x7f6f2c077c60 msgr2=0x7f6f2c1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 shutdown_connections 2024-09-21T02:05:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:09.560+0000 7f6f327c6640 1 -- 172.21.15.92:0/4108375200 wait complete. 2024-09-21T02:05:10.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:05:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:11.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 -- 172.21.15.92:0/1307838531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10108cf0 msgr2=0x7f9c1010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 --2- 172.21.15.92:0/1307838531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10108cf0 0x7f9c1010f520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009ca0 tx=0x7f9c0802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 -- 172.21.15.92:0/1307838531 shutdown_connections 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 --2- 172.21.15.92:0/1307838531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10108cf0 0x7f9c1010f520 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 --2- 172.21.15.92:0/1307838531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c101083a0 0x7f9c101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 --2- 172.21.15.92:0/1307838531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c10102a60 0x7f9c10102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.091+0000 7f9c17daf640 1 -- 172.21.15.92:0/1307838531 >> 172.21.15.92:0/1307838531 conn(0x7f9c10079730 msgr2=0x7f9c10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.092+0000 7f9c17daf640 1 -- 172.21.15.92:0/1307838531 shutdown_connections 2024-09-21T02:05:11.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.092+0000 7f9c17daf640 1 -- 172.21.15.92:0/1307838531 wait complete. 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 Processor -- start 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 -- start start 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10102a60 0x7f9c1010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c101083a0 0x7f9c101104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c10108cf0 0x7f9c10110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10113ab0 con 0x7f9c10102a60 2024-09-21T02:05:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10113930 con 0x7f9c101083a0 2024-09-21T02:05:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c17daf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10113c30 con 0x7f9c10108cf0 2024-09-21T02:05:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c15b4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10102a60 0x7f9c1010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c15b4b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10102a60 0x7f9c1010ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39236/0 (socket says 172.21.15.92:39236) 2024-09-21T02:05:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3854488452 learned_addr learned my addr 172.21.15.92:0/3854488452 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.093+0000 7f9c1534a640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c101083a0 0x7f9c101104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c10108cf0 0x7f9c10110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 -- 172.21.15.92:0/3854488452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c101083a0 msgr2=0x7f9c101104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c101083a0 0x7f9c101104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 -- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10102a60 msgr2=0x7f9c1010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10102a60 0x7f9c1010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 -- 172.21.15.92:0/3854488452 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c101a87a0 con 0x7f9c10108cf0 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1534a640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c101083a0 0x7f9c101104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.094+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c10108cf0 0x7f9c10110a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005e30 tx=0x7f9c080317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.095+0000 7f9c06ffd640 1 -- 172.21.15.92:0/3854488452 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08003b70 con 0x7f9c10108cf0 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.095+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101a8a30 con 0x7f9c10108cf0 2024-09-21T02:05:11.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.095+0000 7f9c06ffd640 1 -- 172.21.15.92:0/3854488452 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c08003d10 con 0x7f9c10108cf0 2024-09-21T02:05:11.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.095+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101a8fc0 con 0x7f9c10108cf0 2024-09-21T02:05:11.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.096+0000 7f9c06ffd640 1 -- 172.21.15.92:0/3854488452 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0802fa00 con 0x7f9c10108cf0 2024-09-21T02:05:11.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.098+0000 7f9c1e323740 1 Processor -- start 2024-09-21T02:05:11.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.099+0000 7f9c06ffd640 1 -- 172.21.15.92:0/3854488452 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f9c0802fc00 con 0x7f9c10108cf0 2024-09-21T02:05:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.099+0000 7f9c1e323740 1 -- start start 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.099+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 0x5618da0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9d04390 0x5618da0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1e323740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5618da042ea0 con 0x5618d9d25070 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1e323740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618da0322d0 con 0x5618d9d04390 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1e323740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5618da0360e0 con 0x5618d9cfc1b0 2024-09-21T02:05:11.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 0x5618da0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 0x5618da0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39242/0 (socket says 172.21.15.92:39242) 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 -- 172.21.15.92:0/1892803046 learned_addr learned my addr 172.21.15.92:0/1892803046 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 -- 172.21.15.92:0/1892803046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 msgr2=0x5618da0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 -- 172.21.15.92:0/1892803046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9d04390 msgr2=0x5618da0cd100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9d04390 0x5618da0cd100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.100+0000 7f9c1534a640 1 -- 172.21.15.92:0/1892803046 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d9d90650 con 0x5618d9d25070 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.101+0000 7f9c06ffd640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9bfc0b6620 0x7f9bfc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.101+0000 7f9c1534a640 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 0x5618da0cc680 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000bf90 tx=0x7f9c00010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c7863a03604c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.101+0000 7f9c06ffd640 1 -- 172.21.15.92:0/3854488452 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c081084f0 con 0x7f9c10108cf0 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.101+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9bfc0b6620 0x7f9bfc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.101+0000 7f9bfbfff640 1 -- 172.21.15.92:0/1892803046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00011700 con 0x5618d9d25070 2024-09-21T02:05:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.101+0000 7f9bfbfff640 1 -- 172.21.15.92:0/1892803046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c00011d50 con 0x5618d9d25070 2024-09-21T02:05:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.102+0000 7f9bfbfff640 1 -- 172.21.15.92:0/1892803046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0001ad40 con 0x5618d9d25070 2024-09-21T02:05:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 -- 172.21.15.92:0/1892803046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 msgr2=0x5618da0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9bfc0b6620 0x7f9bfc0b8b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5618da0cb300 tx=0x7f9c0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 0x5618da0cc680 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000bf90 tx=0x7f9c00010860 comp rx=0 tx=0).stop 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 -- 172.21.15.92:0/1892803046 shutdown_connections 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9d04390 0x5618da0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 --2- 172.21.15.92:0/1892803046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d25070 0x5618da0cc680 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 -- 172.21.15.92:0/1892803046 >> 172.21.15.92:0/1892803046 conn(0x5618d9d19460 msgr2=0x5618da0ca570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 -- 172.21.15.92:0/1892803046 shutdown_connections 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.103+0000 7f9c1e323740 1 -- 172.21.15.92:0/1892803046 wait complete. 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.104+0000 7f9c1e323740 1 -- start start 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.104+0000 7f9c1e323740 1 Processor -- start 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9cfc1b0 0x5618da0d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9ce39f0 0x5618da0e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5618d9e93700 con 0x5618d9d04390 2024-09-21T02:05:11.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618d9fcd160 con 0x5618d9cfc1b0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5618da0daf00 con 0x5618d9ce39f0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1e323740 10 client.0 _ll_register_callbacks cb 0x5618d9fe69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1634c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9ce39f0 0x5618da0e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c15b4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9cfc1b0 0x5618da0d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c15b4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9cfc1b0 0x5618da0d7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:39102/0 (socket says 172.21.15.92:39102) 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c15b4b640 1 -- 172.21.15.92:0/4108908583 learned_addr learned my addr 172.21.15.92:0/4108908583 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1534a640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1634c640 1 -- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9cfc1b0 msgr2=0x5618da0d7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1634c640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9cfc1b0 0x5618da0d7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.105+0000 7f9c1634c640 1 -- 172.21.15.92:0/4108908583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 msgr2=0x5618da0d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9c1634c640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9c1634c640 1 -- 172.21.15.92:0/4108908583 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d9ffac90 con 0x5618d9ce39f0 2024-09-21T02:05:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9c1534a640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9c1634c640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9ce39f0 0x5618da0e4240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c08105e80 tx=0x7f9c080cad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f0833300abdd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9bf97fa640 1 -- 172.21.15.92:0/4108908583 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080cddc0 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9bf97fa640 1 -- 172.21.15.92:0/4108908583 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c080d7a90 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.106+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c00011d50 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9bf97fa640 1 -- 172.21.15.92:0/4108908583 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c080ce160 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 10 client.25201 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618da0ef430 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9bf97fa640 1 -- 172.21.15.92:0/4108908583 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c00011d50 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9bf97fa640 1 -- 172.21.15.92:0/4108908583 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c080c9fa0 con 0x5618d9ce39f0 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 10 client.25201 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 10 client.25201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:05:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 20 client.25201 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618d9cb8da0 0x5618da0f2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 4 client.25201 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.107+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618da044df0 con 0x5618d9cb8da0 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.108+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618d9cb8da0 0x5618da0f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.108+0000 7f9bf97fa640 10 client.25201 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.108+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618d9cb8da0 0x5618da0f2990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f615c5e246ad9b9 server_cookie=71310ba8d1269634 in_seq=0 out_seq=0 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9bf97fa640 1 -- 172.21.15.92:0/4108908583 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618da044df0 con 0x5618d9cb8da0 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9bf97fa640 10 client.25201 handle_command_reply: tid=0 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 client.25201 shutdown 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9ce39f0 msgr2=0x5618da0e4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9ce39f0 0x5618da0e4240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c08105e80 tx=0x7f9c080cad70 comp rx=0 tx=0).stop 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 shutdown_connections 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9ce39f0 0x5618da0e4240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618d9cb8da0 0x5618da0f2990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9bf97fa640 0 client.25201 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 --2- 172.21.15.92:0/4108908583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9cfc1b0 0x5618da0d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.109+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 >> 172.21.15.92:0/4108908583 conn(0x5618d9d19460 msgr2=0x5618da0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.110+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 shutdown_connections 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.110+0000 7f9c1e323740 1 -- 172.21.15.92:0/4108908583 wait complete. 2024-09-21T02:05:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.110+0000 7f9c1e323740 20 client.25201 trim_cache size 0 max 16384 2024-09-21T02:05:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.135+0000 7f9c1e323740 1 Processor -- start 2024-09-21T02:05:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.135+0000 7f9c1e323740 1 -- start start 2024-09-21T02:05:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1e323740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ad180 con 0x5618d9d04390 2024-09-21T02:05:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1e323740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101aecc0 con 0x5618d9ce39f0 2024-09-21T02:05:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1e323740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ad850 con 0x5618d9cfc1b0 2024-09-21T02:05:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1634c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1634c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39248/0 (socket says 172.21.15.92:39248) 2024-09-21T02:05:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1634c640 1 -- 172.21.15.92:0/3334080592 learned_addr learned my addr 172.21.15.92:0/3334080592 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1534a640 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3334080592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 msgr2=0x5618da0e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3334080592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 msgr2=0x5618da0d4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3334080592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618da0cf4e0 con 0x5618d9ce39f0 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.136+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.137+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0d9be0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5618da0f30c0 tx=0x7f9c0c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e17696dadb92ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.137+0000 7f9bfb7fe640 1 -- 172.21.15.92:0/3334080592 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c035070 con 0x5618d9ce39f0 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.137+0000 7f9bfb7fe640 1 -- 172.21.15.92:0/3334080592 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0c0308d0 con 0x5618d9ce39f0 2024-09-21T02:05:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.137+0000 7f9bfb7fe640 1 -- 172.21.15.92:0/3334080592 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c02a8e0 con 0x5618d9ce39f0 2024-09-21T02:05:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 -- 172.21.15.92:0/3334080592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 msgr2=0x5618da0d9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0d9be0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5618da0f30c0 tx=0x7f9c0c01b000 comp rx=0 tx=0).stop 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 -- 172.21.15.92:0/3334080592 shutdown_connections 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0d4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3334080592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0d9be0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.138+0000 7f9c1e323740 1 -- 172.21.15.92:0/3334080592 >> 172.21.15.92:0/3334080592 conn(0x5618d9d19460 msgr2=0x5618da0d6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.139+0000 7f9c1e323740 1 -- 172.21.15.92:0/3334080592 shutdown_connections 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.139+0000 7f9c1e323740 1 -- 172.21.15.92:0/3334080592 wait complete. 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.139+0000 7f9c1e323740 1 -- start start 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.139+0000 7f9c1e323740 1 Processor -- start 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0e0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ae5f0 con 0x5618d9d04390 2024-09-21T02:05:11.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101adf20 con 0x5618d9ce39f0 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ab180 con 0x5618d9cfc1b0 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1e323740 10 client.0 _ll_register_callbacks cb 0x7f9c08009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c15b4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c15b4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0e0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:39118/0 (socket says 172.21.15.92:39118) 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3550069358 learned_addr learned my addr 172.21.15.92:0/3550069358 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c1534a640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3550069358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 msgr2=0x5618da0dcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0dcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.140+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3550069358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 msgr2=0x5618da0dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:11.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9c15b4b640 1 -- 172.21.15.92:0/3550069358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618da0dd920 con 0x5618d9ce39f0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9c1634c640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9c15b4b640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0e0880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009f30 tx=0x7f9c0c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4de876aa7ce8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9bf97fa640 1 -- 172.21.15.92:0/3550069358 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c035040 con 0x5618d9ce39f0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0c0308d0 con 0x5618d9ce39f0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.141+0000 7f9bf97fa640 1 -- 172.21.15.92:0/3550069358 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c0c030a80 con 0x5618d9ce39f0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.142+0000 7f9bf97fa640 1 -- 172.21.15.92:0/3550069358 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c0c030c80 con 0x5618d9ce39f0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.142+0000 7f9c1e323740 10 client.25172 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.142+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618da15e190 con 0x5618d9ce39f0 2024-09-21T02:05:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.142+0000 7f9bf97fa640 1 -- 172.21.15.92:0/3550069358 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0c0308d0 con 0x5618d9ce39f0 2024-09-21T02:05:11.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9bf97fa640 1 -- 172.21.15.92:0/3550069358 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c0c038070 con 0x5618d9ce39f0 2024-09-21T02:05:11.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9c1e323740 10 client.25172 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:11.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9c1e323740 10 client.25172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:05:11.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9c1e323740 20 client.25172 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:11.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618da15e850 0x5618da160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:11.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9c1e323740 4 client.25172 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.143+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618da0408a0 con 0x5618da15e850 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.144+0000 7f9c1534a640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618da15e850 0x5618da160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.144+0000 7f9c1534a640 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618da15e850 0x5618da160c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfd2a5e37dcad456 server_cookie=d0610ad16499bb4b in_seq=0 out_seq=0 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.144+0000 7f9bf97fa640 10 client.25172 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.144+0000 7f9bf97fa640 1 -- 172.21.15.92:0/3550069358 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618da0408a0 con 0x5618da15e850 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.144+0000 7f9bf97fa640 10 client.25172 handle_command_reply: tid=0 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.144+0000 7f9c1e323740 1 client.25172 shutdown 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 msgr2=0x5618da0e0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0e0880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009f30 tx=0x7f9c0c009f60 comp rx=0 tx=0).stop 2024-09-21T02:05:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 shutdown_connections 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5618da15e850 0x5618da160c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5618d9d04390 0x5618da0dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5618d9cfc1b0 0x5618da0dcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9bf97fa640 0 client.25172 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 --2- 172.21.15.92:0/3550069358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d9ce39f0 0x5618da0e0880 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 >> 172.21.15.92:0/3550069358 conn(0x5618d9d19460 msgr2=0x5618da0d9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 shutdown_connections 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 1 -- 172.21.15.92:0/3550069358 wait complete. 2024-09-21T02:05:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.145+0000 7f9c1e323740 20 client.25172 trim_cache size 0 max 16384 2024-09-21T02:05:11.151 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9bfc0b6620 msgr2=0x7f9bfc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9bfc0b6620 0x7f9bfc0b8b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5618da0cb300 tx=0x7f9c0c005f90 comp rx=0 tx=0).stop 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c10108cf0 msgr2=0x7f9c10110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c10108cf0 0x7f9c10110a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005e30 tx=0x7f9c080317c0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 shutdown_connections 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9bfc0b6620 0x7f9bfc0b8b30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c10108cf0 0x7f9c10110a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c101083a0 0x7f9c101104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 --2- 172.21.15.92:0/3854488452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c10102a60 0x7f9c1010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 >> 172.21.15.92:0/3854488452 conn(0x7f9c10079730 msgr2=0x7f9c10104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.146+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 shutdown_connections 2024-09-21T02:05:11.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:11.147+0000 7f9c17daf640 1 -- 172.21.15.92:0/3854488452 wait complete. 2024-09-21T02:05:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:10 smithi136 ceph-mon[34754]: pgmap v366: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:05:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:10 smithi092 ceph-mon[31190]: pgmap v366: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:05:11.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:10 smithi099 ceph-mon[34404]: pgmap v366: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-21T02:05:11.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:05:12.204 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 -- 172.21.15.92:0/3214498929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc81031c0 msgr2=0x7fddc81035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 --2- 172.21.15.92:0/3214498929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc81031c0 0x7fddc81035d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009900 tx=0x7fddb802a410 comp rx=0 tx=0).stop 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 -- 172.21.15.92:0/3214498929 shutdown_connections 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 --2- 172.21.15.92:0/3214498929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8105ae0 0x7fddc810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 --2- 172.21.15.92:0/3214498929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc8105190 0x7fddc81055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 --2- 172.21.15.92:0/3214498929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc81031c0 0x7fddc81035d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.722+0000 7fddcd128640 1 -- 172.21.15.92:0/3214498929 >> 172.21.15.92:0/3214498929 conn(0x7fddc8077c60 msgr2=0x7fddc80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.723+0000 7fddcd128640 1 -- 172.21.15.92:0/3214498929 shutdown_connections 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.723+0000 7fddcd128640 1 -- 172.21.15.92:0/3214498929 wait complete. 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.723+0000 7fddcd128640 1 Processor -- start 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 -- start start 2024-09-21T02:05:12.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc81031c0 0x7fddc819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8105190 0x7fddc819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 0x7fddc819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8112980 con 0x7fddc8105ae0 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8112800 con 0x7fddc81031c0 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddcd128640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8112b00 con 0x7fddc8105190 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddc759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 0x7fddc819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddc759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 0x7fddc819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39282/0 (socket says 172.21.15.92:39282) 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddc759e640 1 -- 172.21.15.92:0/3598792347 learned_addr learned my addr 172.21.15.92:0/3598792347 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.724+0000 7fddc659c640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8105190 0x7fddc819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc759e640 1 -- 172.21.15.92:0/3598792347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8105190 msgr2=0x7fddc819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc759e640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8105190 0x7fddc819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc759e640 1 -- 172.21.15.92:0/3598792347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc81031c0 msgr2=0x7fddc819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc81031c0 0x7fddc819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc759e640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc81031c0 0x7fddc819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc759e640 1 -- 172.21.15.92:0/3598792347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc81a0310 con 0x7fddc8105ae0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddc759e640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 0x7fddc819fca0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fddc000cce0 tx=0x7fddc000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddbffff640 1 -- 172.21.15.92:0/3598792347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc000f450 con 0x7fddc8105ae0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc81a0600 con 0x7fddc8105ae0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.725+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc80fecc0 con 0x7fddc8105ae0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.726+0000 7fddbffff640 1 -- 172.21.15.92:0/3598792347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc000faa0 con 0x7fddc8105ae0 2024-09-21T02:05:12.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.726+0000 7fddbffff640 1 -- 172.21.15.92:0/3598792347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0018b70 con 0x7fddc8105ae0 2024-09-21T02:05:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.729+0000 7fddd369c740 1 Processor -- start 2024-09-21T02:05:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.729+0000 7fddbffff640 1 -- 172.21.15.92:0/3598792347 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fddc0018d10 con 0x7fddc8105ae0 2024-09-21T02:05:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.729+0000 7fddd369c740 1 -- start start 2024-09-21T02:05:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddd369c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad300eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddd369c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad3013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddd369c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad301930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddd369c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558bad26eea0 con 0x558bacf51070 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddd369c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558bad25e2d0 con 0x558bacf30390 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddd369c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558bad2620e0 con 0x558bacf281b0 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddc659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad300eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddc659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad300eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39294/0 (socket says 172.21.15.92:39294) 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddc659c640 1 -- 172.21.15.92:0/2193852395 learned_addr learned my addr 172.21.15.92:0/2193852395 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.730+0000 7fddc659c640 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad301930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad3013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc659c640 1 -- 172.21.15.92:0/2193852395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 msgr2=0x558bad3013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc659c640 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad3013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc659c640 1 -- 172.21.15.92:0/2193852395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 msgr2=0x558bad300eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc659c640 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad300eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc659c640 1 -- 172.21.15.92:0/2193852395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bacfbc650 con 0x558bacf30390 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddbffff640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fddb40b6650 0x7fddb40b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fddb40b6650 0x7fddb40b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.731+0000 7fddbffff640 1 -- 172.21.15.92:0/3598792347 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fddc00e7660 con 0x7fddc8105ae0 2024-09-21T02:05:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.732+0000 7fddc659c640 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad301930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fddb0012790 tx=0x7fddb0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fa0da5b744a2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.732+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fddb40b6650 0x7fddb40b8b60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddb8031440 tx=0x7fddb8005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.732+0000 7fddbd7fa640 1 -- 172.21.15.92:0/2193852395 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb00120c0 con 0x558bacf30390 2024-09-21T02:05:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.732+0000 7fddbd7fa640 1 -- 172.21.15.92:0/2193852395 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb000e750 con 0x558bacf30390 2024-09-21T02:05:12.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.732+0000 7fddbd7fa640 1 -- 172.21.15.92:0/2193852395 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb00044e0 con 0x558bacf30390 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 -- 172.21.15.92:0/2193852395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 msgr2=0x558bad301930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad301930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fddb0012790 tx=0x7fddb0004730 comp rx=0 tx=0).stop 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 -- 172.21.15.92:0/2193852395 shutdown_connections 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad301930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad3013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 --2- 172.21.15.92:0/2193852395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad300eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 -- 172.21.15.92:0/2193852395 >> 172.21.15.92:0/2193852395 conn(0x558bacf45460 msgr2=0x558bad2feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 -- 172.21.15.92:0/2193852395 shutdown_connections 2024-09-21T02:05:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.733+0000 7fddd369c740 1 -- 172.21.15.92:0/2193852395 wait complete. 2024-09-21T02:05:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 -- start start 2024-09-21T02:05:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 Processor -- start 2024-09-21T02:05:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad30c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad30cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf51070 0x558bad318a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558bad0bf700 con 0x558bacf30390 2024-09-21T02:05:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558bad1f9160 con 0x558bacf51070 2024-09-21T02:05:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.734+0000 7fddd369c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558bad30f6e0 con 0x558bacf281b0 2024-09-21T02:05:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad30cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad30c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddd369c740 10 client.0 _ll_register_callbacks cb 0x558bad2129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad30cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39296/0 (socket says 172.21.15.92:39296) 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc659c640 1 -- 172.21.15.92:0/2563764329 learned_addr learned my addr 172.21.15.92:0/2563764329 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf51070 0x558bad318a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 -- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 msgr2=0x558bad30c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad30c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 -- 172.21.15.92:0/2563764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 msgr2=0x558bad30cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad30cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 -- 172.21.15.92:0/2563764329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bad226c90 con 0x558bacf51070 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad30c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddc759e640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf51070 0x558bad318a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddc00e4ff0 tx=0x7fddc00a9b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ea179d1fc1cad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddae7fc640 1 -- 172.21.15.92:0/2563764329 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00add70 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb000e750 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.735+0000 7fddae7fc640 1 -- 172.21.15.92:0/2563764329 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fddc00ac8d0 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddae7fc640 1 -- 172.21.15.92:0/2563764329 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddc00b2150 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 10 client.25181 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bad31b460 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddae7fc640 1 -- 172.21.15.92:0/2563764329 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb000e750 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddae7fc640 1 -- 172.21.15.92:0/2563764329 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fddc0020030 con 0x558bacf51070 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 10 client.25181 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 10 client.25181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 20 client.25181 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:12.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacf0f9f0 0x558bad31e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 4 client.25181 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.736+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bad270df0 con 0x558bacf0f9f0 2024-09-21T02:05:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.737+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacf0f9f0 0x558bad31e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.737+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacf0f9f0 0x558bad31e9a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b56e3173d5d7c3 server_cookie=b4ce35abc9154e5f in_seq=0 out_seq=0 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.737+0000 7fddae7fc640 10 client.25181 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddae7fc640 1 -- 172.21.15.92:0/2563764329 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558bad270df0 con 0x558bacf0f9f0 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddae7fc640 10 client.25181 handle_command_reply: tid=0 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 client.25181 shutdown 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf51070 msgr2=0x558bad318a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf51070 0x558bad318a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddc00e4ff0 tx=0x7fddc00a9b90 comp rx=0 tx=0).stop 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 shutdown_connections 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacf0f9f0 0x558bad31e9a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf51070 0x558bad318a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad30cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 --2- 172.21.15.92:0/2563764329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad30c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 >> 172.21.15.92:0/2563764329 conn(0x558bacf45460 msgr2=0x558bad304040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddae7fc640 0 client.25181 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.738+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 shutdown_connections 2024-09-21T02:05:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.739+0000 7fddd369c740 1 -- 172.21.15.92:0/2563764329 wait complete. 2024-09-21T02:05:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.739+0000 7fddd369c740 20 client.25181 trim_cache size 0 max 16384 2024-09-21T02:05:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 Processor -- start 2024-09-21T02:05:12.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 -- start start 2024-09-21T02:05:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad318930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad30c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad30c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdda800e430 con 0x558bacf51070 2024-09-21T02:05:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdda800ff70 con 0x558bacf30390 2024-09-21T02:05:12.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.752+0000 7fddd369c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdda800eb00 con 0x558bacf281b0 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad30c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad30c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39300/0 (socket says 172.21.15.92:39300) 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 -- 172.21.15.92:0/1027468707 learned_addr learned my addr 172.21.15.92:0/1027468707 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad318930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 -- 172.21.15.92:0/1027468707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 msgr2=0x558bad318930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad318930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 -- 172.21.15.92:0/1027468707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 msgr2=0x558bad30c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad30c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 -- 172.21.15.92:0/1027468707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bad30ab10 con 0x558bacf51070 2024-09-21T02:05:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddc759e640 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad30c900 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fddc00aa8e0 tx=0x7fddc0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c898d1632f558dde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.753+0000 7fddbcff9640 1 -- 172.21.15.92:0/1027468707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00c7070 con 0x558bacf51070 2024-09-21T02:05:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.754+0000 7fddbcff9640 1 -- 172.21.15.92:0/1027468707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc00add70 con 0x558bacf51070 2024-09-21T02:05:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.754+0000 7fddbcff9640 1 -- 172.21.15.92:0/1027468707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00b6040 con 0x558bacf51070 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.754+0000 7fddd369c740 1 -- 172.21.15.92:0/1027468707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 msgr2=0x558bad30c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.754+0000 7fddd369c740 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad30c900 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fddc00aa8e0 tx=0x7fddc0012000 comp rx=0 tx=0).stop 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 -- 172.21.15.92:0/1027468707 shutdown_connections 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf51070 0x558bad30c900 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf30390 0x558bad30c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 --2- 172.21.15.92:0/1027468707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf281b0 0x558bad318930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 -- 172.21.15.92:0/1027468707 >> 172.21.15.92:0/1027468707 conn(0x558bacf45460 msgr2=0x558bad30ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 -- 172.21.15.92:0/1027468707 shutdown_connections 2024-09-21T02:05:12.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 -- 172.21.15.92:0/1027468707 wait complete. 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.755+0000 7fddd369c740 1 -- start start 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 Processor -- start 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf281b0 0x558bad3152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad3107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf51070 0x558bad310e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdda800f8a0 con 0x558bacf30390 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdda800f1d0 con 0x558bacf281b0 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdda800c410 con 0x558bacf51070 2024-09-21T02:05:12.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddd369c740 10 client.0 _ll_register_callbacks cb 0x7fddc0005a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddc659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad3107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddc659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad3107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39308/0 (socket says 172.21.15.92:39308) 2024-09-21T02:05:12.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.756+0000 7fddc659c640 1 -- 172.21.15.92:0/1760023227 learned_addr learned my addr 172.21.15.92:0/1760023227 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf281b0 0x558bad3152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc759e640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf51070 0x558bad310e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc659c640 1 -- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf51070 msgr2=0x558bad310e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc659c640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf51070 0x558bad310e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc659c640 1 -- 172.21.15.92:0/1760023227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf281b0 msgr2=0x558bad3152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc659c640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf281b0 0x558bad3152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc659c640 1 -- 172.21.15.92:0/1760023227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bad388360 con 0x558bacf30390 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddc659c640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad3107a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddb0005940 tx=0x7fddb000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d96ae44a8bc0ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddae7fc640 1 -- 172.21.15.92:0/1760023227 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0016050 con 0x558bacf30390 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.757+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc00add70 con 0x558bacf30390 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddae7fc640 1 -- 172.21.15.92:0/1760023227 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fddb0004040 con 0x558bacf30390 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddae7fc640 1 -- 172.21.15.92:0/1760023227 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddb002f050 con 0x558bacf30390 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddd369c740 10 client.15672 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bad389420 con 0x558bacf30390 2024-09-21T02:05:12.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddae7fc640 1 -- 172.21.15.92:0/1760023227 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc00add70 con 0x558bacf30390 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddae7fc640 1 -- 172.21.15.92:0/1760023227 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fddb0027020 con 0x558bacf30390 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.758+0000 7fddd369c740 10 client.15672 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddd369c740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddd369c740 20 client.15672 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddd369c740 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacee4da0 0x558bad38b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddd369c740 4 client.15672 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bad26c8a0 con 0x558bacee4da0 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacee4da0 0x558bad38b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddae7fc640 10 client.15672 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.759+0000 7fddc6d9d640 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacee4da0 0x558bad38b9f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5474a5521a521cee server_cookie=f84bdd66183d024f in_seq=0 out_seq=0 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddae7fc640 1 -- 172.21.15.92:0/1760023227 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bad26c8a0 con 0x558bacee4da0 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddae7fc640 10 client.15672 handle_command_reply: tid=0 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 client.15672 shutdown 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 msgr2=0x558bad3107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad3107a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddb0005940 tx=0x7fddb000b7e0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 shutdown_connections 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558bacee4da0 0x558bad38b9f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddae7fc640 0 client.15672 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558bacf51070 0x558bad310e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bacf30390 0x558bad3107a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 --2- 172.21.15.92:0/1760023227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558bacf281b0 0x558bad3152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.760+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 >> 172.21.15.92:0/1760023227 conn(0x558bacf45460 msgr2=0x558bad30c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 shutdown_connections 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddd369c740 1 -- 172.21.15.92:0/1760023227 wait complete. 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddd369c740 20 client.15672 trim_cache size 0 max 16384 2024-09-21T02:05:12.764 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fddb40b6650 msgr2=0x7fddb40b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddcd128640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fddb40b6650 0x7fddb40b8b60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fddb8031440 tx=0x7fddb8005d50 comp rx=0 tx=0).stop 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 msgr2=0x7fddc819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.761+0000 7fddcd128640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 0x7fddc819fca0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fddc000cce0 tx=0x7fddc000a620 comp rx=0 tx=0).stop 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 shutdown_connections 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fddb40b6650 0x7fddb40b8b60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddc8105ae0 0x7fddc819fca0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8105190 0x7fddc819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 --2- 172.21.15.92:0/3598792347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc81031c0 0x7fddc819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:12.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 >> 172.21.15.92:0/3598792347 conn(0x7fddc8077c60 msgr2=0x7fddc81013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:12.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 shutdown_connections 2024-09-21T02:05:12.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:12.762+0000 7fddcd128640 1 -- 172.21.15.92:0/3598792347 wait complete. 2024-09-21T02:05:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:12 smithi136 ceph-mon[34754]: pgmap v367: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:05:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:12 smithi092 ceph-mon[31190]: pgmap v367: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:05:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:12 smithi099 ceph-mon[34404]: pgmap v367: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:05:14.473 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:05:14.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:05:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:15.204 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:14 smithi092 ceph-mon[31190]: pgmap v368: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T02:05:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:14 smithi136 ceph-mon[34754]: pgmap v368: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 -- 172.21.15.92:0/3024819719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c101f70 msgr2=0x7fb02c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 --2- 172.21.15.92:0/3024819719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c101f70 0x7fb02c102380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009ca0 tx=0x7fb01c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 -- 172.21.15.92:0/3024819719 shutdown_connections 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 --2- 172.21.15.92:0/3024819719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02c109e00 0x7fb02c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 --2- 172.21.15.92:0/3024819719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb02c1094b0 0x7fb02c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 --2- 172.21.15.92:0/3024819719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c101f70 0x7fb02c102380 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 -- 172.21.15.92:0/3024819719 >> 172.21.15.92:0/3024819719 conn(0x7fb02c077c60 msgr2=0x7fb02c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 -- 172.21.15.92:0/3024819719 shutdown_connections 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.301+0000 7fb0316bc640 1 -- 172.21.15.92:0/3024819719 wait complete. 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.302+0000 7fb0316bc640 1 Processor -- start 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.302+0000 7fb0316bc640 1 -- start start 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb0316bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02c1094b0 0x7fb02c104920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb0316bc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 0x7fb02c102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb0316bc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb02c103450 0x7fb02c103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb0316bc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c113ac0 con 0x7fb02c109e00 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb0316bc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c113940 con 0x7fb02c1094b0 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb0316bc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c113c40 con 0x7fb02c103450 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 0x7fb02c102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 0x7fb02c102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40148/0 (socket says 172.21.15.92:40148) 2024-09-21T02:05:15.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 -- 172.21.15.92:0/145272154 learned_addr learned my addr 172.21.15.92:0/145272154 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02affd640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02c1094b0 0x7fb02c104920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 -- 172.21.15.92:0/145272154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb02c103450 msgr2=0x7fb02c103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb02c103450 0x7fb02c103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 -- 172.21.15.92:0/145272154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02c1094b0 msgr2=0x7fb02c104920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02c1094b0 0x7fb02c104920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.303+0000 7fb02a7fc640 1 -- 172.21.15.92:0/145272154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c1ad0c0 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.304+0000 7fb02a7fc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 0x7fb02c102f10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d950 tx=0x7fb01400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.304+0000 7fb023fff640 1 -- 172.21.15.92:0/145272154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014017070 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.304+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c1ad3b0 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.304+0000 7fb023fff640 1 -- 172.21.15.92:0/145272154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb014009c30 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.304+0000 7fb023fff640 1 -- 172.21.15.92:0/145272154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140047a0 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.304+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c1ad940 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.308+0000 7fb023fff640 1 -- 172.21.15.92:0/145272154 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fb014004020 con 0x7fb02c109e00 2024-09-21T02:05:15.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.308+0000 7fb037c30740 1 Processor -- start 2024-09-21T02:05:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.308+0000 7fb037c30740 1 -- start start 2024-09-21T02:05:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb037c30740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bd06350 0x56202c0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb037c30740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb037c30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bce5670 0x56202c0adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb037c30740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56202c023e60 con 0x56202bd06350 2024-09-21T02:05:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb037c30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56202c013290 con 0x56202bce5670 2024-09-21T02:05:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb037c30740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56202c0170a0 con 0x56202bcdd490 2024-09-21T02:05:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb023fff640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0180b6620 0x7fb0180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb02a7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bce5670 0x56202c0adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb02affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb02affd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38346/0 (socket says 172.21.15.92:38346) 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb02affd640 1 -- 172.21.15.92:0/1387488291 learned_addr learned my addr 172.21.15.92:0/1387488291 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.309+0000 7fb02b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bd06350 0x56202c0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb023fff640 1 -- 172.21.15.92:0/145272154 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb0140e6de0 con 0x7fb02c109e00 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0180b6620 0x7fb0180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02affd640 1 -- 172.21.15.92:0/1387488291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bce5670 msgr2=0x56202c0adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02affd640 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bce5670 0x56202c0adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02affd640 1 -- 172.21.15.92:0/1387488291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bd06350 msgr2=0x56202c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02affd640 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bd06350 0x56202c0ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02affd640 1 -- 172.21.15.92:0/1387488291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202bd71930 con 0x56202bcdd490 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb02affd640 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0ad970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009ca0 tx=0x7fb01c02f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac523c5a64aa7f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb0217fa640 1 -- 172.21.15.92:0/1387488291 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c031d70 con 0x56202bcdd490 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb0217fa640 1 -- 172.21.15.92:0/1387488291 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01c036070 con 0x56202bcdd490 2024-09-21T02:05:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.310+0000 7fb0217fa640 1 -- 172.21.15.92:0/1387488291 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0042a0 con 0x56202bcdd490 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.311+0000 7fb037c30740 1 -- 172.21.15.92:0/1387488291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 msgr2=0x56202c0ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.311+0000 7fb037c30740 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0ad970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009ca0 tx=0x7fb01c02f4c0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.311+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0180b6620 0x7fb0180b8b30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb02400ae70 tx=0x7fb024005870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 -- 172.21.15.92:0/1387488291 shutdown_connections 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bce5670 0x56202c0adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0ad970 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 --2- 172.21.15.92:0/1387488291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bd06350 0x56202c0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 -- 172.21.15.92:0/1387488291 >> 172.21.15.92:0/1387488291 conn(0x56202bcfa740 msgr2=0x56202c0ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 -- 172.21.15.92:0/1387488291 shutdown_connections 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 -- 172.21.15.92:0/1387488291 wait complete. 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 -- start start 2024-09-21T02:05:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.312+0000 7fb037c30740 1 Processor -- start 2024-09-21T02:05:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bd06350 0x56202c0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56202be74890 con 0x56202bcdd490 2024-09-21T02:05:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56202bfadd40 con 0x56202bd06350 2024-09-21T02:05:15.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56202c0bba30 con 0x56202bce5670 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb037c30740 10 client.0 _ll_register_callbacks cb 0x56202c011c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0b8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40166/0 (socket says 172.21.15.92:40166) 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 -- 172.21.15.92:0/1162214961 learned_addr learned my addr 172.21.15.92:0/1162214961 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02a7fc640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bd06350 0x56202c0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 -- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 msgr2=0x56202c0b9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 -- 172.21.15.92:0/1162214961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bd06350 msgr2=0x56202c0c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bd06350 0x56202c0c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02affd640 1 -- 172.21.15.92:0/1162214961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202c0c52e0 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.313+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bd06350 0x56202c0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb02affd640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0b8fd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb01c031e80 tx=0x7fb01c003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d99da6aa93ef87b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb0127fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb0127fc640 1 -- 172.21.15.92:0/1162214961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c04e060 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb0127fc640 1 -- 172.21.15.92:0/1162214961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb01c02f660 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb01c036070 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb0127fc640 1 -- 172.21.15.92:0/1162214961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb01c0318e0 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb037c30740 10 client.15684 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56202c0c7c10 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.314+0000 7fb0127fc640 1 -- 172.21.15.92:0/1162214961 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01c036070 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.315+0000 7fb0127fc640 1 -- 172.21.15.92:0/1162214961 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb01c043020 con 0x56202bcdd490 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb037c30740 10 client.15684 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:15.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb037c30740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb037c30740 20 client.15684 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb037c30740 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bcc4cd0 0x56202c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb037c30740 4 client.15684 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56202c025db0 con 0x56202bcc4cd0 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.316+0000 7fb02a7fc640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bcc4cd0 0x56202c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.317+0000 7fb0127fc640 10 client.15684 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.317+0000 7fb02a7fc640 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bcc4cd0 0x56202c0caf70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de91c2a2ecdad2f server_cookie=84889beeb74a71ee in_seq=0 out_seq=0 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.317+0000 7fb0127fc640 1 -- 172.21.15.92:0/1162214961 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56202c025db0 con 0x56202bcc4cd0 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.317+0000 7fb0127fc640 10 client.15684 handle_command_reply: tid=0 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.317+0000 7fb037c30740 1 client.15684 shutdown 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 msgr2=0x56202c0b8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0b8fd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb01c031e80 tx=0x7fb01c003d20 comp rx=0 tx=0).stop 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 shutdown_connections 2024-09-21T02:05:15.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bcc4cd0 0x56202c0caf70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bd06350 0x56202c0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb0127fc640 0 client.15684 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 --2- 172.21.15.92:0/1162214961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0b8fd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 >> 172.21.15.92:0/1162214961 conn(0x56202bcfa740 msgr2=0x56202c0ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 shutdown_connections 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 1 -- 172.21.15.92:0/1162214961 wait complete. 2024-09-21T02:05:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.318+0000 7fb037c30740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T02:05:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:14 smithi099 ceph-mon[34404]: pgmap v368: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T02:05:15.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.335+0000 7fb037c30740 1 Processor -- start 2024-09-21T02:05:15.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 -- start start 2024-09-21T02:05:15.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b5f00 con 0x56202bcdd490 2024-09-21T02:05:15.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b7a40 con 0x56202bcc4cd0 2024-09-21T02:05:15.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb037c30740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b65d0 con 0x56202bce5670 2024-09-21T02:05:15.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.336+0000 7fb02a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40172/0 (socket says 172.21.15.92:40172) 2024-09-21T02:05:15.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02a7fc640 1 -- 172.21.15.92:0/1620565109 learned_addr learned my addr 172.21.15.92:0/1620565109 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:15.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 -- 172.21.15.92:0/1620565109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 msgr2=0x56202c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 -- 172.21.15.92:0/1620565109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 msgr2=0x56202c0c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 -- 172.21.15.92:0/1620565109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202c0cb3d0 con 0x56202bcc4cd0 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb02affd640 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0b8660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009ca0 tx=0x7fb01c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1bbd95e008bb944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb020ff9640 1 -- 172.21.15.92:0/1620565109 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c009890 con 0x56202bcc4cd0 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb020ff9640 1 -- 172.21.15.92:0/1620565109 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01c036040 con 0x56202bcc4cd0 2024-09-21T02:05:15.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.337+0000 7fb020ff9640 1 -- 172.21.15.92:0/1620565109 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c02fa20 con 0x56202bcc4cd0 2024-09-21T02:05:15.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 -- 172.21.15.92:0/1620565109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 msgr2=0x56202c0b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0b8660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009ca0 tx=0x7fb01c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 -- 172.21.15.92:0/1620565109 shutdown_connections 2024-09-21T02:05:15.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bce5670 0x56202c0b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bcdd490 0x56202c0c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 --2- 172.21.15.92:0/1620565109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0b8660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.338+0000 7fb037c30740 1 -- 172.21.15.92:0/1620565109 >> 172.21.15.92:0/1620565109 conn(0x56202bcfa740 msgr2=0x56202c0aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:15.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.339+0000 7fb037c30740 1 -- 172.21.15.92:0/1620565109 shutdown_connections 2024-09-21T02:05:15.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.339+0000 7fb037c30740 1 -- 172.21.15.92:0/1620565109 wait complete. 2024-09-21T02:05:15.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.339+0000 7fb037c30740 1 -- start start 2024-09-21T02:05:15.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.339+0000 7fb037c30740 1 Processor -- start 2024-09-21T02:05:15.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0bd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 0x56202c134850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b7370 con 0x56202bce5670 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b6ca0 con 0x56202bcc4cd0 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b3ee0 con 0x56202bcdd490 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb037c30740 10 client.0 _ll_register_callbacks cb 0x7fb024005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 0x56202c134850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 0x56202c134850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40186/0 (socket says 172.21.15.92:40186) 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 -- 172.21.15.92:0/3844730629 learned_addr learned my addr 172.21.15.92:0/3844730629 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02affd640 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 -- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 msgr2=0x56202c0bd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0bd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 -- 172.21.15.92:0/3844730629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 msgr2=0x56202c0bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 -- 172.21.15.92:0/3844730629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202c134d90 con 0x56202bce5670 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.340+0000 7fb02b7fe640 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 0x56202c134850 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb024012470 tx=0x7fb0240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e421126ede03a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb0127fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb0127fc640 1 -- 172.21.15.92:0/3844730629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024020620 con 0x56202bce5670 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb0127fc640 1 -- 172.21.15.92:0/3844730629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb0240207c0 con 0x56202bce5670 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb01c036040 con 0x56202bce5670 2024-09-21T02:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb0127fc640 1 -- 172.21.15.92:0/3844730629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0240209c0 con 0x56202bce5670 2024-09-21T02:05:15.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb037c30740 10 client.15690 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:15.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56202c135f80 con 0x56202bce5670 2024-09-21T02:05:15.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.341+0000 7fb0127fc640 1 -- 172.21.15.92:0/3844730629 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01c036040 con 0x56202bce5670 2024-09-21T02:05:15.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb0127fc640 1 -- 172.21.15.92:0/3844730629 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb024032080 con 0x56202bce5670 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb037c30740 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb037c30740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb037c30740 20 client.15690 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb037c30740 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bd06350 0x56202c1387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb037c30740 4 client.15690 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.342+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56202c021860 con 0x56202bd06350 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.343+0000 7fb02affd640 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bd06350 0x56202c1387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.343+0000 7fb0127fc640 10 client.15690 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:15.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.343+0000 7fb02affd640 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bd06350 0x56202c1387a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41239a438c3c6b6d server_cookie=3eb56b5df662e6cb in_seq=0 out_seq=0 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb0127fc640 1 -- 172.21.15.92:0/3844730629 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56202c021860 con 0x56202bd06350 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb0127fc640 10 client.15690 handle_command_reply: tid=0 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 client.15690 shutdown 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 msgr2=0x56202c134850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 0x56202c134850 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb024012470 tx=0x7fb0240046c0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 shutdown_connections 2024-09-21T02:05:15.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56202bd06350 0x56202c1387a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56202bce5670 0x56202c134850 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56202bcdd490 0x56202c0bd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 --2- 172.21.15.92:0/3844730629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56202bcc4cd0 0x56202c0bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 >> 172.21.15.92:0/3844730629 conn(0x56202bcfa740 msgr2=0x56202c0aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb0127fc640 0 client.15690 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a", 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:05:15.347 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 shutdown_connections 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 1 -- 172.21.15.92:0/3844730629 wait complete. 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.345+0000 7fb037c30740 20 client.15690 trim_cache size 0 max 16384 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0180b6620 msgr2=0x7fb0180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0180b6620 0x7fb0180b8b30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb02400ae70 tx=0x7fb024005870 comp rx=0 tx=0).stop 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 msgr2=0x7fb02c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 0x7fb02c102f10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d950 tx=0x7fb01400de10 comp rx=0 tx=0).stop 2024-09-21T02:05:15.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 shutdown_connections 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0180b6620 0x7fb0180b8b30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb02c103450 0x7fb02c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb02c109e00 0x7fb02c102f10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 --2- 172.21.15.92:0/145272154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02c1094b0 0x7fb02c104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 >> 172.21.15.92:0/145272154 conn(0x7fb02c077c60 msgr2=0x7fb02c107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 shutdown_connections 2024-09-21T02:05:15.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:15.346+0000 7fb0316bc640 1 -- 172.21.15.92:0/145272154 wait complete. 2024-09-21T02:05:15.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:05:16.344 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:16.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3642816060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 msgr2=0x7fc7f410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3642816060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f410f520 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009ca0 tx=0x7fc7ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3642816060 shutdown_connections 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3642816060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f410f520 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3642816060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f41083a0 0x7fc7f41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3642816060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 0x7fc7f4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3642816060 >> 172.21.15.92:0/3642816060 conn(0x7fc7f4079730 msgr2=0x7fc7f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3642816060 shutdown_connections 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.807+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3642816060 wait complete. 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.808+0000 7fc7f9efb640 1 Processor -- start 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.808+0000 7fc7f9efb640 1 -- start start 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f9efb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 0x7fc7f41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f9efb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f41083a0 0x7fc7f4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f9efb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f9efb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41129b0 con 0x7fc7f4108cf0 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40194/0 (socket says 172.21.15.92:40194) 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f3fff640 1 -- 172.21.15.92:0/3487204936 learned_addr learned my addr 172.21.15.92:0/3487204936 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4112830 con 0x7fc7f41083a0 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f41083a0 0x7fc7f4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 0x7fc7f41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4112b30 con 0x7fc7f4102a60 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f37fe640 1 -- 172.21.15.92:0/3487204936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f41083a0 msgr2=0x7fc7f4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f41083a0 0x7fc7f4199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f37fe640 1 -- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 msgr2=0x7fc7f419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.809+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f37fe640 1 -- 172.21.15.92:0/3487204936 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f419f330 con 0x7fc7f4102a60 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:16.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 0x7fc7f41993c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009f10 tx=0x7fc7e400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f0ff9640 1 -- 172.21.15.92:0/3487204936 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4020070 con 0x7fc7f4102a60 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f419f620 con 0x7fc7f4102a60 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f0ff9640 1 -- 172.21.15.92:0/3487204936 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e400ccc0 con 0x7fc7f4102a60 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f0ff9640 1 -- 172.21.15.92:0/3487204936 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e401ba80 con 0x7fc7f4102a60 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.810+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f406ae60 con 0x7fc7f4102a60 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 Processor -- start 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 -- start start 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27ca070 0x55b1d2b71650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b71b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 0x55b1d2b720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b1d2ae7e50 con 0x55b1d27a11b0 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1d2ad7280 con 0x55b1d27ca070 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.813+0000 7fc80046f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b1d2adb090 con 0x55b1d27a9390 2024-09-21T02:05:16.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 0x55b1d2b720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 0x55b1d2b720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38394/0 (socket says 172.21.15.92:38394) 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/1566756771 learned_addr learned my addr 172.21.15.92:0/1566756771 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27ca070 0x55b1d2b71650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/1566756771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27ca070 msgr2=0x55b1d2b71650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27ca070 0x55b1d2b71650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/1566756771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 msgr2=0x55b1d2b71b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b71b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/1566756771 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d2835650 con 0x55b1d27a9390 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.814+0000 7fc7f0ff9640 1 -- 172.21.15.92:0/3487204936 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fc7e4023070 con 0x7fc7f4102a60 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.815+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 0x55b1d2b720d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009ba0 tx=0x7fc7e80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26ea9ee94f4de3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.815+0000 7fc7d9ffb640 1 -- 172.21.15.92:0/1566756771 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8009d50 con 0x55b1d27a9390 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.815+0000 7fc7d9ffb640 1 -- 172.21.15.92:0/1566756771 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e80035f0 con 0x55b1d27a9390 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.815+0000 7fc7d9ffb640 1 -- 172.21.15.92:0/1566756771 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f6a0 con 0x55b1d27a9390 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.815+0000 7fc7f0ff9640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7dc0b6620 0x7fc7dc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.816+0000 7fc7f0ff9640 1 -- 172.21.15.92:0/3487204936 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc7e40eb480 con 0x7fc7f4102a60 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.816+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7dc0b6620 0x7fc7dc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.816+0000 7fc80046f740 1 -- 172.21.15.92:0/1566756771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 msgr2=0x55b1d2b720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.816+0000 7fc80046f740 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 0x55b1d2b720d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009ba0 tx=0x7fc7e80107a0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7dc0b6620 0x7fc7dc0b8b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec004af0 tx=0x7fc7ec005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 -- 172.21.15.92:0/1566756771 shutdown_connections 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a9390 0x55b1d2b720d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b71b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 --2- 172.21.15.92:0/1566756771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27ca070 0x55b1d2b71650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 -- 172.21.15.92:0/1566756771 >> 172.21.15.92:0/1566756771 conn(0x55b1d27be460 msgr2=0x55b1d2b6f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 -- 172.21.15.92:0/1566756771 shutdown_connections 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.817+0000 7fc80046f740 1 -- 172.21.15.92:0/1566756771 wait complete. 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 -- start start 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 Processor -- start 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b7cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b7d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 0x55b1d2b89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b1d2938700 con 0x55b1d27a11b0 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1d2a72110 con 0x55b1d27a9390 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.818+0000 7fc80046f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b1d2b7fe30 con 0x55b1d27ca070 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc80046f740 10 client.0 _ll_register_callbacks cb 0x55b1d2a8b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 0x55b1d2b89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b7cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 0x55b1d2b89150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38408/0 (socket says 172.21.15.92:38408) 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 -- 172.21.15.92:0/1043438530 learned_addr learned my addr 172.21.15.92:0/1043438530 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b7d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 -- 172.21.15.92:0/1043438530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 msgr2=0x55b1d2b7d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b7d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 -- 172.21.15.92:0/1043438530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 msgr2=0x55b1d2b7cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b7cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.819+0000 7fc7f3fff640 1 -- 172.21.15.92:0/1043438530 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d2a9fc40 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 0x55b1d2b89150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0319e0 tx=0x7fc7ec02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968b233e37439188 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc7d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/1043438530 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec004250 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/1043438530 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc7ec0043f0 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e80035f0 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/1043438530 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec0045f0 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc80046f740 10 client.25246 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.820+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1d2b8bb70 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/1043438530 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ec036040 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/1043438530 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc7ec052070 con 0x55b1d27ca070 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc80046f740 10 client.25246 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc80046f740 10 client.25246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc80046f740 20 client.25246 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc80046f740 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27889f0 0x55b1d2b8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc80046f740 4 client.25246 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.821+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc7e80035f0 con 0x55b1d27889f0 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.822+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27889f0 0x55b1d2b8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.822+0000 7fc7d2ffd640 10 client.25246 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.822+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27889f0 0x55b1d2b8f090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f0fcd1c90c070e server_cookie=9c36e4173e986113 in_seq=0 out_seq=0 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/1043438530 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc7e80035f0 con 0x55b1d27889f0 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc7d2ffd640 10 client.25246 handle_command_reply: tid=0 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 client.25246 shutdown 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 msgr2=0x55b1d2b89150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 0x55b1d2b89150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0319e0 tx=0x7fc7ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 shutdown_connections 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27889f0 0x55b1d2b8f090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc7d2ffd640 0 client.25246 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27ca070 0x55b1d2b89150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b7d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 --2- 172.21.15.92:0/1043438530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b7cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 >> 172.21.15.92:0/1043438530 conn(0x55b1d27be460 msgr2=0x55b1d2b747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.823+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 shutdown_connections 2024-09-21T02:05:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.824+0000 7fc80046f740 1 -- 172.21.15.92:0/1043438530 wait complete. 2024-09-21T02:05:16.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.824+0000 7fc80046f740 20 client.25246 trim_cache size 0 max 16384 2024-09-21T02:05:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.841+0000 7fc80046f740 1 Processor -- start 2024-09-21T02:05:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.841+0000 7fc80046f740 1 -- start start 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.841+0000 7fc80046f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27889f0 0x55b1d2b7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc80046f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b89050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc80046f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc80046f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ad190 con 0x55b1d27a11b0 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc80046f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41aecd0 con 0x55b1d27a9390 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc80046f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ad860 con 0x55b1d27889f0 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b89050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b89050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40200/0 (socket says 172.21.15.92:40200) 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/204673708 learned_addr learned my addr 172.21.15.92:0/204673708 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27889f0 0x55b1d2b7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/204673708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27889f0 msgr2=0x55b1d2b7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27889f0 0x55b1d2b7d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/204673708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 msgr2=0x55b1d2b79f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.842+0000 7fc7f2ffd640 1 -- 172.21.15.92:0/204673708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d2b7f930 con 0x55b1d27a11b0 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.843+0000 7fc7f2ffd640 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b89050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8000c00 tx=0x7fc7e8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886d2e6f4e549091 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:16.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.843+0000 7fc7d97fa640 1 -- 172.21.15.92:0/204673708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e802a070 con 0x55b1d27a11b0 2024-09-21T02:05:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.843+0000 7fc7d97fa640 1 -- 172.21.15.92:0/204673708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e8009dd0 con 0x55b1d27a11b0 2024-09-21T02:05:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.843+0000 7fc7d97fa640 1 -- 172.21.15.92:0/204673708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8003bf0 con 0x55b1d27a11b0 2024-09-21T02:05:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 -- 172.21.15.92:0/204673708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 msgr2=0x55b1d2b89050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b89050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8000c00 tx=0x7fc7e8009820 comp rx=0 tx=0).stop 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 -- 172.21.15.92:0/204673708 shutdown_connections 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27a9390 0x55b1d2b79f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a11b0 0x55b1d2b89050 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 --2- 172.21.15.92:0/204673708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27889f0 0x55b1d2b7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 -- 172.21.15.92:0/204673708 >> 172.21.15.92:0/204673708 conn(0x55b1d27be460 msgr2=0x55b1d2b7cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 -- 172.21.15.92:0/204673708 shutdown_connections 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.844+0000 7fc80046f740 1 -- 172.21.15.92:0/204673708 wait complete. 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.845+0000 7fc80046f740 1 -- start start 2024-09-21T02:05:16.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.845+0000 7fc80046f740 1 Processor -- start 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc80046f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 0x55b1d2b81930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc80046f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a11b0 0x55b1d2b81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc80046f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 0x55b1d2bf2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 0x55b1d2b81930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 0x55b1d2b81930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:35070/0 (socket says 172.21.15.92:35070) 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a11b0 0x55b1d2b81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a11b0 0x55b1d2b81f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38430/0 (socket says 172.21.15.92:38430) 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 0x55b1d2bf2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 0x55b1d2bf2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40212/0 (socket says 172.21.15.92:40212) 2024-09-21T02:05:16.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc7f37fe640 1 -- 172.21.15.92:0/692749947 learned_addr learned my addr 172.21.15.92:0/692749947 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ae600 con 0x55b1d27a9390 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41adf30 con 0x55b1d27889f0 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.846+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41ab170 con 0x55b1d27a11b0 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f3fff640 1 -- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a11b0 msgr2=0x55b1d2b81f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a11b0 0x55b1d2b81f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f3fff640 1 -- 172.21.15.92:0/692749947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 msgr2=0x55b1d2b81930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 0x55b1d2b81930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f3fff640 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d2bf3090 con 0x55b1d27a9390 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 0x55b1d2b81930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc80046f740 10 client.0 _ll_register_callbacks cb 0x7fc7ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:16.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e8009dd0 con 0x55b1d27a9390 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7f3fff640 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 0x55b1d2bf2a20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec030710 tx=0x7fc7ec030b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c3f1c57b5d91d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.847+0000 7fc7d2ffd640 10 client.15705 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.848+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/692749947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec063060 con 0x55b1d27a9390 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.848+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/692749947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc7ec004d40 con 0x55b1d27a9390 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.848+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/692749947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec0317f0 con 0x55b1d27a9390 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.848+0000 7fc80046f740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:16.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.848+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b1d2bfa170 con 0x55b1d27a9390 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/692749947 <== mon.0 v2:172.21.15.92:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc7ec052040 con 0x55b1d27a9390 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc80046f740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc80046f740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc80046f740 20 client.15705 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc80046f740 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27ca070 0x55b1d2bfc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc80046f740 4 client.15705 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1d2ae9da0 con 0x55b1d27ca070 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27ca070 0x55b1d2bfc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc7f37fe640 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27ca070 0x55b1d2bfc940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f181838f15cdf15 server_cookie=f11fd35d7c5fbaff in_seq=0 out_seq=0 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.849+0000 7fc7d2ffd640 10 client.15705 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc7d2ffd640 1 -- 172.21.15.92:0/692749947 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x55b1d2ae9da0 con 0x55b1d27ca070 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc7d2ffd640 10 client.15705 handle_command_reply: tid=0 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 client.15705 shutdown 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 msgr2=0x55b1d2bf2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 0x55b1d2bf2a20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec030710 tx=0x7fc7ec030b90 comp rx=0 tx=0).stop 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 shutdown_connections 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1d27ca070 0x55b1d2bfc940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1d27a9390 0x55b1d2bf2a20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1d27a11b0 0x55b1d2b81f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 --2- 172.21.15.92:0/692749947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d27889f0 0x55b1d2b81930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 >> 172.21.15.92:0/692749947 conn(0x55b1d27be460 msgr2=0x55b1d2b72680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc7d2ffd640 0 client.15705 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 shutdown_connections 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 1 -- 172.21.15.92:0/692749947 wait complete. 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.850+0000 7fc80046f740 20 client.15705 trim_cache size 0 max 16384 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:05:16.853 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a", 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:05:16.854 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:16.855 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7dc0b6620 msgr2=0x7fc7dc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7dc0b6620 0x7fc7dc0b8b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec004af0 tx=0x7fc7ec005d80 comp rx=0 tx=0).stop 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 msgr2=0x7fc7f41993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 0x7fc7f41993c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009f10 tx=0x7fc7e400ed10 comp rx=0 tx=0).stop 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 shutdown_connections 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc7dc0b6620 0x7fc7dc0b8b30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7f4108cf0 0x7fc7f419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f41083a0 0x7fc7f4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 --2- 172.21.15.92:0/3487204936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7f4102a60 0x7fc7f41993c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 >> 172.21.15.92:0/3487204936 conn(0x7fc7f4079730 msgr2=0x7fc7f4104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 shutdown_connections 2024-09-21T02:05:16.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:16.851+0000 7fc7f9efb640 1 -- 172.21.15.92:0/3487204936 wait complete. 2024-09-21T02:05:17.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:16 smithi092 ceph-mon[31190]: pgmap v369: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-21T02:05:17.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:16 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:05:17.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:05:17.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:16 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:05:17.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:05:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:16 smithi136 ceph-mon[34754]: pgmap v369: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-21T02:05:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:16 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:05:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:05:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:16 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:05:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:05:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:16 smithi099 ceph-mon[34404]: pgmap v369: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-21T02:05:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:16 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:05:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:05:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:16 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:05:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:05:17.595 INFO:tasks.cephfs.filesystem:scrub status for tag:1c962d51-81e7-4a2e-bb15-e30d91c9dc6a - {'path': '/', 'tag': '1c962d51-81e7-4a2e-bb15-e30d91c9dc6a', 'options': 'recursive,force'} 2024-09-21T02:05:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:18 smithi136 ceph-mon[34754]: pgmap v370: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T02:05:19.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:18 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:05:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:18 smithi092 ceph-mon[31190]: pgmap v370: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T02:05:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:18 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:05:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:18 smithi099 ceph-mon[34404]: pgmap v370: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T02:05:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:19 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:05:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:20 smithi136 ceph-mon[34754]: pgmap v371: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-09-21T02:05:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:20 smithi092 ceph-mon[31190]: pgmap v371: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-09-21T02:05:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:20 smithi099 ceph-mon[34404]: pgmap v371: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 -- 172.21.15.92:0/2353737439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103fc0 msgr2=0x7fb07810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 --2- 172.21.15.92:0/2353737439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103fc0 0x7fb07810a960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb070009c90 tx=0x7fb07002a860 comp rx=0 tx=0).stop 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 -- 172.21.15.92:0/2353737439 shutdown_connections 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 --2- 172.21.15.92:0/2353737439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103fc0 0x7fb07810a960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 --2- 172.21.15.92:0/2353737439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb078103620 0x7fb078103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 --2- 172.21.15.92:0/2353737439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0781025a0 0x7fb078102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 -- 172.21.15.92:0/2353737439 >> 172.21.15.92:0/2353737439 conn(0x7fb0780fdee0 msgr2=0x7fb078100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 -- 172.21.15.92:0/2353737439 shutdown_connections 2024-09-21T02:05:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.362+0000 7fb080207640 1 -- 172.21.15.92:0/2353737439 wait complete. 2024-09-21T02:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.363+0000 7fb080207640 1 Processor -- start 2024-09-21T02:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.363+0000 7fb080207640 1 -- start start 2024-09-21T02:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.363+0000 7fb080207640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0781025a0 0x7fb078197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.363+0000 7fb080207640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 0x7fb078198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 0x7fb078198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 0x7fb078198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40220/0 (socket says 172.21.15.92:40220) 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 -- 172.21.15.92:0/3487572582 learned_addr learned my addr 172.21.15.92:0/3487572582 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb078103fc0 0x7fb078198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810d160 con 0x7fb078103620 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810cfe0 con 0x7fb078103fc0 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810d2e0 con 0x7fb0781025a0 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0781025a0 0x7fb078197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb078103fc0 0x7fb078198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 -- 172.21.15.92:0/3487572582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0781025a0 msgr2=0x7fb078197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0781025a0 0x7fb078197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 -- 172.21.15.92:0/3487572582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb078103fc0 msgr2=0x7fb078198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb078103fc0 0x7fb078198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 -- 172.21.15.92:0/3487572582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb078199730 con 0x7fb078103620 2024-09-21T02:05:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.364+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 0x7fb078198360 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d950 tx=0x7fb06800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.365+0000 7fb06effd640 1 -- 172.21.15.92:0/3487572582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068017030 con 0x7fb078103620 2024-09-21T02:05:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.365+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb078199a20 con 0x7fb078103620 2024-09-21T02:05:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.365+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0781a0110 con 0x7fb078103620 2024-09-21T02:05:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.365+0000 7fb06effd640 1 -- 172.21.15.92:0/3487572582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb068004020 con 0x7fb078103620 2024-09-21T02:05:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.365+0000 7fb06effd640 1 -- 172.21.15.92:0/3487572582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0680047a0 con 0x7fb078103620 2024-09-21T02:05:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 Processor -- start 2024-09-21T02:05:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 -- start start 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 0x56450373d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450336c4d0 0x56450373e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5645036a76f0 con 0x56450338d1b0 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564503696b20 con 0x56450336c4d0 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.369+0000 7fb087248740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56450369a930 con 0x5645033642f0 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 0x56450373d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 0x56450373d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40236/0 (socket says 172.21.15.92:40236) 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 -- 172.21.15.92:0/1942747823 learned_addr learned my addr 172.21.15.92:0/1942747823 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb06effd640 1 -- 172.21.15.92:0/3487572582 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7fb068009a30 con 0x7fb078103620 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 -- 172.21.15.92:0/1942747823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 msgr2=0x56450373dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 -- 172.21.15.92:0/1942747823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450336c4d0 msgr2=0x56450373e820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450336c4d0 0x56450373e820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 -- 172.21.15.92:0/1942747823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56450365c530 con 0x56450338d1b0 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.370+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 0x56450373d660 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb070009c90 tx=0x7fb070009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960003bb5dec6c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.371+0000 7fb063fff640 1 -- 172.21.15.92:0/1942747823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700313f0 con 0x56450338d1b0 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.371+0000 7fb063fff640 1 -- 172.21.15.92:0/1942747823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb070036030 con 0x56450338d1b0 2024-09-21T02:05:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.371+0000 7fb063fff640 1 -- 172.21.15.92:0/1942747823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070030ab0 con 0x56450338d1b0 2024-09-21T02:05:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.371+0000 7fb06effd640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0640b6620 0x7fb0640b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.371+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0640b6620 0x7fb0640b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.371+0000 7fb06effd640 1 -- 172.21.15.92:0/3487572582 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb0680e6e40 con 0x7fb078103620 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 -- 172.21.15.92:0/1942747823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 msgr2=0x56450373d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 0x56450373d660 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb070009c90 tx=0x7fb070009940 comp rx=0 tx=0).stop 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0640b6620 0x7fb0640b8b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb07400fd50 tx=0x7fb074005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 -- 172.21.15.92:0/1942747823 shutdown_connections 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450336c4d0 0x56450373e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 --2- 172.21.15.92:0/1942747823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450338d1b0 0x56450373d660 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 -- 172.21.15.92:0/1942747823 >> 172.21.15.92:0/1942747823 conn(0x564503381300 msgr2=0x5645037390f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 -- 172.21.15.92:0/1942747823 shutdown_connections 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.372+0000 7fb087248740 1 -- 172.21.15.92:0/1942747823 wait complete. 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.373+0000 7fb087248740 1 -- start start 2024-09-21T02:05:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.373+0000 7fb087248740 1 Processor -- start 2024-09-21T02:05:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb087248740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x564503746e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb087248740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x5645037474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb087248740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503748060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb087248740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5645035ceab0 con 0x5645033642f0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb07dfa3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x564503746e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb07dfa3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x564503746e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40246/0 (socket says 172.21.15.92:40246) 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb07dfa3640 1 -- 172.21.15.92:0/969830426 learned_addr learned my addr 172.21.15.92:0/969830426 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb087248740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56450363c730 con 0x56450338d1b0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56450373f2e0 con 0x56450336c4d0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x5645037474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.374+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503748060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb087248740 10 client.0 _ll_register_callbacks cb 0x5645036906b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07d7a2640 1 -- 172.21.15.92:0/969830426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 msgr2=0x564503748060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503748060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07d7a2640 1 -- 172.21.15.92:0/969830426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 msgr2=0x564503746e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x564503746e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07d7a2640 1 -- 172.21.15.92:0/969830426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645033f8790 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503748060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x564503746e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x5645037474f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb0680e47d0 tx=0x7fb0680e4c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b8cb825f8fed71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb0617fa640 1 -- 172.21.15.92:0/969830426 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0680cee60 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb0617fa640 1 -- 172.21.15.92:0/969830426 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb068094070 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.375+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb070036030 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb0617fa640 1 -- 172.21.15.92:0/969830426 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0680ac960 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb087248740 10 client.25261 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564503471620 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb0617fa640 1 -- 172.21.15.92:0/969830426 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb070036030 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb0617fa640 1 -- 172.21.15.92:0/969830426 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb06805c150 con 0x56450336c4d0 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb087248740 10 client.25261 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb087248740 10 client.25261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb087248740 20 client.25261 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.376+0000 7fb087248740 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56450334bd80 0x5645037545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.377+0000 7fb087248740 4 client.25261 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.377+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645036a9640 con 0x56450334bd80 2024-09-21T02:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.377+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56450334bd80 0x5645037545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.378+0000 7fb0617fa640 10 client.25261 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.378+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56450334bd80 0x5645037545d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1d7fc4e2ea13a5 server_cookie=b4aead6d9d3ff0ad in_seq=0 out_seq=0 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.378+0000 7fb0617fa640 1 -- 172.21.15.92:0/969830426 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645036a9640 con 0x56450334bd80 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.378+0000 7fb0617fa640 10 client.25261 handle_command_reply: tid=0 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.378+0000 7fb087248740 1 client.25261 shutdown 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 msgr2=0x5645037474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x5645037474f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb0680e47d0 tx=0x7fb0680e4c50 comp rx=0 tx=0).stop 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 shutdown_connections 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56450334bd80 0x5645037545d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503748060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x5645037474f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 --2- 172.21.15.92:0/969830426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x564503746e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 >> 172.21.15.92:0/969830426 conn(0x564503381300 msgr2=0x56450372eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb0617fa640 0 client.25261 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 shutdown_connections 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 1 -- 172.21.15.92:0/969830426 wait complete. 2024-09-21T02:05:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.379+0000 7fb087248740 20 client.25261 trim_cache size 0 max 16384 2024-09-21T02:05:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.407+0000 7fb087248740 1 Processor -- start 2024-09-21T02:05:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.407+0000 7fb087248740 1 -- start start 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb087248740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb087248740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 0x56450374ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07d7a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 0x56450374ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07d7a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 0x56450374ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40254/0 (socket says 172.21.15.92:40254) 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb087248740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503742170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07d7a2640 1 -- 172.21.15.92:0/3724272927 learned_addr learned my addr 172.21.15.92:0/3724272927 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781a8d10 con 0x56450336c4d0 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781aa850 con 0x56450338d1b0 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781a93e0 con 0x5645033642f0 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503742170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3724272927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 msgr2=0x564503742170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503742170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3724272927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 msgr2=0x56450374ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 0x56450374ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3724272927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564503750eb0 con 0x5645033642f0 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.408+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 0x56450374ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.409+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373e800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb074004620 tx=0x7fb074017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81f9512c26c487d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.409+0000 7fb0637fe640 1 -- 172.21.15.92:0/3724272927 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07401d1d0 con 0x5645033642f0 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.409+0000 7fb0637fe640 1 -- 172.21.15.92:0/3724272927 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07402d030 con 0x5645033642f0 2024-09-21T02:05:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.409+0000 7fb0637fe640 1 -- 172.21.15.92:0/3724272927 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074004a90 con 0x5645033642f0 2024-09-21T02:05:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 msgr2=0x56450373e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373e800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb074004620 tx=0x7fb074017000 comp rx=0 tx=0).stop 2024-09-21T02:05:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 shutdown_connections 2024-09-21T02:05:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x564503742170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56450336c4d0 0x56450374ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 --2- 172.21.15.92:0/3724272927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645033642f0 0x56450373e800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 >> 172.21.15.92:0/3724272927 conn(0x564503381300 msgr2=0x564503738aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 shutdown_connections 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.410+0000 7fb087248740 1 -- 172.21.15.92:0/3724272927 wait complete. 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.411+0000 7fb087248740 1 -- start start 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.411+0000 7fb087248740 1 Processor -- start 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb087248740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x56450374c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb087248740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x56450374c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb087248740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x56450374d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb07dfa3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x56450374c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb07dfa3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x56450374c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40260/0 (socket says 172.21.15.92:40260) 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3990860654 learned_addr learned my addr 172.21.15.92:0/3990860654 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781aa180 con 0x5645033642f0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781a9ab0 con 0x56450338d1b0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x56450374c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781a6cf0 con 0x56450336c4d0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.412+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x56450374d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 msgr2=0x56450374c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x56450374c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3990860654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 msgr2=0x56450374d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x56450374d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb087248740 10 client.0 _ll_register_callbacks cb 0x7fb0680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07dfa3640 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56450374da30 con 0x5645033642f0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07e7a4640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x56450374d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x56450374c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07402d030 con 0x5645033642f0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb07dfa3640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x56450374c0f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb07400ff60 tx=0x7fb07401d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f88c05d380eb8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb0617fa640 10 client.15723 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb0617fa640 1 -- 172.21.15.92:0/3990860654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074039670 con 0x5645033642f0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb0617fa640 1 -- 172.21.15.92:0/3990860654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb074039810 con 0x5645033642f0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb0617fa640 1 -- 172.21.15.92:0/3990860654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb074039a50 con 0x5645033642f0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb087248740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.413+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56450374ec10 con 0x5645033642f0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb0617fa640 1 -- 172.21.15.92:0/3990860654 <== mon.0 v2:172.21.15.92:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb074039bf0 con 0x5645033642f0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb087248740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb087248740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb087248740 20 client.15723 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb087248740 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5645033211c0 0x5645037c1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb087248740 4 client.15723 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645036a50f0 con 0x5645033211c0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.414+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5645033211c0 0x5645037c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.415+0000 7fb0617fa640 10 client.15723 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.415+0000 7fb07d7a2640 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5645033211c0 0x5645037c1020 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b84437882dde83 server_cookie=59420f021ac7bebb in_seq=0 out_seq=0 2024-09-21T02:05:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb0617fa640 1 -- 172.21.15.92:0/3990860654 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645036a50f0 con 0x5645033211c0 2024-09-21T02:05:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb0617fa640 10 client.15723 handle_command_reply: tid=0 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 client.15723 shutdown 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 msgr2=0x56450374c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x56450374c0f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb07400ff60 tx=0x7fb07401d4c0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 shutdown_connections 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5645033211c0 0x5645037c1020 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56450338d1b0 0x56450374d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56450336c4d0 0x56450374c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 --2- 172.21.15.92:0/3990860654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5645033642f0 0x56450374c0f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 >> 172.21.15.92:0/3990860654 conn(0x564503381300 msgr2=0x564503739a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.422+0000 7fb0617fa640 0 client.15723 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.423+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 shutdown_connections 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.423+0000 7fb087248740 1 -- 172.21.15.92:0/3990860654 wait complete. 2024-09-21T02:05:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.423+0000 7fb087248740 20 client.15723 trim_cache size 0 max 16384 2024-09-21T02:05:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:05:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:05:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:05:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.423+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0640b6620 msgr2=0x7fb0640b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.423+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0640b6620 0x7fb0640b8b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb07400fd50 tx=0x7fb074005f90 comp rx=0 tx=0).stop 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 msgr2=0x7fb078198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 0x7fb078198360 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d950 tx=0x7fb06800de10 comp rx=0 tx=0).stop 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 shutdown_connections 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0640b6620 0x7fb0640b8b30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb078103fc0 0x7fb078198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb078103620 0x7fb078198360 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 --2- 172.21.15.92:0/3487572582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0781025a0 0x7fb078197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 >> 172.21.15.92:0/3487572582 conn(0x7fb0780fdee0 msgr2=0x7fb078106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 shutdown_connections 2024-09-21T02:05:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:21.424+0000 7fb080207640 1 -- 172.21.15.92:0/3487572582 wait complete. 2024-09-21T02:05:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:22 smithi136 ceph-mon[34754]: pgmap v372: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T02:05:23.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:22 smithi092 ceph-mon[31190]: pgmap v372: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T02:05:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:22 smithi099 ceph-mon[34404]: pgmap v372: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T02:05:25.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:24 smithi136 ceph-mon[34754]: pgmap v373: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T02:05:25.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:24 smithi092 ceph-mon[31190]: pgmap v373: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T02:05:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:24 smithi099 ceph-mon[34404]: pgmap v373: 97 pgs: 97 active+clean; 676 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T02:05:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:26 smithi136 ceph-mon[34754]: pgmap v374: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T02:05:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:26 smithi092 ceph-mon[31190]: pgmap v374: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T02:05:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:26 smithi099 ceph-mon[34404]: pgmap v374: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T02:05:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:28 smithi136 ceph-mon[34754]: pgmap v375: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:29.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:28 smithi092 ceph-mon[31190]: pgmap v375: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:28 smithi099 ceph-mon[34404]: pgmap v375: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:30.177 INFO:tasks.workunit.client.0.smithi092.stderr:Updating files: 99% (12623/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T02:05:30.600 INFO:tasks.workunit.client.0.smithi092.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T02:05:30.600 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: git switch -c 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr:Or undo this operation with: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: git switch - 2024-09-21T02:05:30.601 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.602 INFO:tasks.workunit.client.0.smithi092.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T02:05:30.602 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-09-21T02:05:30.602 INFO:tasks.workunit.client.0.smithi092.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:05:30.606 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T02:05:30.637 INFO:tasks.workunit.client.0.smithi092.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T02:05:30.638 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T02:05:30.638 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T02:05:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:31 smithi136 ceph-mon[34754]: pgmap v376: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T02:05:31.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:31 smithi092 ceph-mon[31190]: pgmap v376: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T02:05:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:31 smithi099 ceph-mon[34404]: pgmap v376: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T02:05:31.381 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T02:05:31.449 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T02:05:31.490 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T02:05:31.492 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T02:05:31.492 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T02:05:31.530 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T02:05:31.534 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T02:05:31.534 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T02:05:31.589 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-21T02:05:31.590 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-21T02:05:31.591 DEBUG:teuthology.orchestra.run.smithi092:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-21T02:05:31.650 INFO:tasks.workunit.client.0.smithi092.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-21T02:05:31.651 INFO:tasks.workunit.client.0.smithi092.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-21T02:05:31.651 INFO:tasks.workunit.client.0.smithi092.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-21T02:05:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:33 smithi136 ceph-mon[34754]: pgmap v377: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:33.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:33 smithi092 ceph-mon[31190]: pgmap v377: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:33 smithi099 ceph-mon[34404]: pgmap v377: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T02:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 -- 172.21.15.92:0/1731112204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8109410 msgr2=0x7f49e810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 --2- 172.21.15.92:0/1731112204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8109410 0x7f49e810b920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009900 tx=0x7f49e002a410 comp rx=0 tx=0).stop 2024-09-21T02:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 -- 172.21.15.92:0/1731112204 shutdown_connections 2024-09-21T02:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 --2- 172.21.15.92:0/1731112204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8109410 0x7f49e810b920 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 --2- 172.21.15.92:0/1731112204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8106850 0x7f49e8108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 --2- 172.21.15.92:0/1731112204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e806c4b0 0x7f49e8106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 -- 172.21.15.92:0/1731112204 >> 172.21.15.92:0/1731112204 conn(0x7f49e80fdf30 msgr2=0x7f49e81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 -- 172.21.15.92:0/1731112204 shutdown_connections 2024-09-21T02:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.535+0000 7f49f0007640 1 -- 172.21.15.92:0/1731112204 wait complete. 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 Processor -- start 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 -- start start 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e806c4b0 0x7f49e8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 0x7f49e819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8109410 0x7f49e819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49e810f330 con 0x7f49e8106850 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49e810f1b0 con 0x7f49e8109410 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49f0007640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49e810f4b0 con 0x7f49e806c4b0 2024-09-21T02:05:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49ed5a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 0x7f49e819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49ed5a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 0x7f49e819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55492/0 (socket says 172.21.15.92:55492) 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.536+0000 7f49ed5a2640 1 -- 172.21.15.92:0/698134610 learned_addr learned my addr 172.21.15.92:0/698134610 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49edda3640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e806c4b0 0x7f49e8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8109410 0x7f49e819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ed5a2640 1 -- 172.21.15.92:0/698134610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e806c4b0 msgr2=0x7f49e8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e806c4b0 0x7f49e8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ed5a2640 1 -- 172.21.15.92:0/698134610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8109410 msgr2=0x7f49e819b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8109410 0x7f49e819b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ed5a2640 1 -- 172.21.15.92:0/698134610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e819b830 con 0x7f49e8106850 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8109410 0x7f49e819b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 0x7f49e819a4f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00c970 tx=0x7f49dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49e6ffd640 1 -- 172.21.15.92:0/698134610 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc014070 con 0x7f49e8106850 2024-09-21T02:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49e6ffd640 1 -- 172.21.15.92:0/698134610 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49dc004440 con 0x7f49e8106850 2024-09-21T02:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49e6ffd640 1 -- 172.21.15.92:0/698134610 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc003e50 con 0x7f49e8106850 2024-09-21T02:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e819bb20 con 0x7f49e8106850 2024-09-21T02:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.537+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e81a22a0 con 0x7f49e8106850 2024-09-21T02:05:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 Processor -- start 2024-09-21T02:05:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 -- start start 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49e6ffd640 1 -- 172.21.15.92:0/698134610 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 154563+0+0 (secure 0 0 0) 0x7f49dc017070 con 0x7f49e8106850 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cbae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754cbb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 0x563754cbc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563754c2d6f0 con 0x5637548f24d0 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563754c1cb20 con 0x5637549131b0 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49f704b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563754c20930 con 0x5637548ea2f0 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49ed5a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 0x563754cbc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49ed5a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 0x563754cbc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55494/0 (socket says 172.21.15.92:55494) 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49ed5a2640 1 -- 172.21.15.92:0/2855864607 learned_addr learned my addr 172.21.15.92:0/2855864607 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.540+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cbae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49edda3640 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754cbb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ed5a2640 1 -- 172.21.15.92:0/2855864607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 msgr2=0x563754cbb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754cbb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ed5a2640 1 -- 172.21.15.92:0/2855864607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 msgr2=0x563754cbae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cbae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ed5a2640 1 -- 172.21.15.92:0/2855864607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563754be2530 con 0x5637548f24d0 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 0x563754cbc010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0e4c00 tx=0x7f49dc0efa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896edb93748ba20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49e6ffd640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f49d00b6740 0x7f49d00b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49cffff640 1 -- 172.21.15.92:0/2855864607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0fb070 con 0x5637548f24d0 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f49d00b6740 0x7f49d00b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49cffff640 1 -- 172.21.15.92:0/2855864607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49dc0ace50 con 0x5637548f24d0 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49cffff640 1 -- 172.21.15.92:0/2855864607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0acab0 con 0x5637548f24d0 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.541+0000 7f49e6ffd640 1 -- 172.21.15.92:0/698134610 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f49dc0e7330 con 0x7f49e8106850 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 -- 172.21.15.92:0/2855864607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 msgr2=0x563754cbc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f49d00b6740 0x7f49d00b8c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f49e0031440 tx=0x7f49e00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 0x563754cbc010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0e4c00 tx=0x7f49dc0efa30 comp rx=0 tx=0).stop 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 -- 172.21.15.92:0/2855864607 shutdown_connections 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548f24d0 0x563754cbc010 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754cbb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 --2- 172.21.15.92:0/2855864607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cbae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 -- 172.21.15.92:0/2855864607 >> 172.21.15.92:0/2855864607 conn(0x563754907300 msgr2=0x563754cb68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 -- 172.21.15.92:0/2855864607 shutdown_connections 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.542+0000 7f49f704b740 1 -- 172.21.15.92:0/2855864607 wait complete. 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 -- start start 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 Processor -- start 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548f24d0 0x563754cc4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563754b54ab0 con 0x5637548ea2f0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563754bc2730 con 0x5637549131b0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563754cbcb30 con 0x5637548f24d0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.543+0000 7f49f704b740 10 client.0 _ll_register_callbacks cb 0x563754c166b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cc4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55504/0 (socket says 172.21.15.92:55504) 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 -- 172.21.15.92:0/2589797421 learned_addr learned my addr 172.21.15.92:0/2589797421 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548f24d0 0x563754cc4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 -- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548f24d0 msgr2=0x563754cc4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548f24d0 0x563754cc4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 -- 172.21.15.92:0/2589797421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 msgr2=0x563754cc57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cc57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 -- 172.21.15.92:0/2589797421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375497e790 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cc57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49edda3640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cc4570 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f49d8011b80 tx=0x7f49d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5986f4d3f04ca3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49cd7fa640 1 -- 172.21.15.92:0/2589797421 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d801e060 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49cd7fa640 1 -- 172.21.15.92:0/2589797421 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f49d800a750 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49dc0ace50 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49cd7fa640 1 -- 172.21.15.92:0/2589797421 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f49d800a950 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49f704b740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.544+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637549f7620 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49cd7fa640 1 -- 172.21.15.92:0/2589797421 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49dc0ace50 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49cd7fa640 1 -- 172.21.15.92:0/2589797421 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f49d8018030 con 0x5637548ea2f0 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49f704b740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49f704b740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49f704b740 20 client.15735 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49f704b740 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637548d1d80 0x563754cd1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49f704b740 4 client.15735 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563754c2f640 con 0x5637548d1d80 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.545+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637548d1d80 0x563754cd1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.546+0000 7f49cd7fa640 10 client.15735 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.546+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637548d1d80 0x563754cd1d80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89edd4b292c64e4 server_cookie=4a630a9eaf118037 in_seq=0 out_seq=0 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.546+0000 7f49cd7fa640 1 -- 172.21.15.92:0/2589797421 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563754c2f640 con 0x5637548d1d80 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.546+0000 7f49cd7fa640 10 client.15735 handle_command_reply: tid=0 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 client.15735 shutdown 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 msgr2=0x563754cc4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cc4570 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f49d8011b80 tx=0x7f49d800f790 comp rx=0 tx=0).stop 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 shutdown_connections 2024-09-21T02:05:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637548d1d80 0x563754cd1d80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637549131b0 0x563754cc57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548f24d0 0x563754cc4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49cd7fa640 0 client.15735 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 --2- 172.21.15.92:0/2589797421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cc4570 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 >> 172.21.15.92:0/2589797421 conn(0x563754907300 msgr2=0x563754cb4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 shutdown_connections 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 1 -- 172.21.15.92:0/2589797421 wait complete. 2024-09-21T02:05:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.547+0000 7f49f704b740 20 client.15735 trim_cache size 0 max 16384 2024-09-21T02:05:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.561+0000 7f49f704b740 1 Processor -- start 2024-09-21T02:05:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.561+0000 7f49f704b740 1 -- start start 2024-09-21T02:05:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49f704b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 0x563754cb6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49f704b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754ccc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49f704b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754cbf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49f704b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81aafe0 con 0x5637548d1d80 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49f704b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81acb20 con 0x5637548f24d0 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49f704b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81ab6b0 con 0x5637548ea2f0 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 0x563754cb6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 0x563754cb6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55512/0 (socket says 172.21.15.92:55512) 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 -- 172.21.15.92:0/2584196930 learned_addr learned my addr 172.21.15.92:0/2584196930 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754ccc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 -- 172.21.15.92:0/2584196930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 msgr2=0x563754ccc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754ccc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 -- 172.21.15.92:0/2584196930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 msgr2=0x563754cbf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754cbf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 -- 172.21.15.92:0/2584196930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563754cc5b10 con 0x5637548d1d80 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754ccc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49edda3640 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 0x563754cb6220 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000f80 tx=0x7f49d8005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568bd41591f3ae16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49cf7fe640 1 -- 172.21.15.92:0/2584196930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d800a750 con 0x5637548d1d80 2024-09-21T02:05:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49cf7fe640 1 -- 172.21.15.92:0/2584196930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49d800aaf0 con 0x5637548d1d80 2024-09-21T02:05:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.562+0000 7f49cf7fe640 1 -- 172.21.15.92:0/2584196930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80234b0 con 0x5637548d1d80 2024-09-21T02:05:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 -- 172.21.15.92:0/2584196930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 msgr2=0x563754cb6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 0x563754cb6220 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000f80 tx=0x7f49d8005b80 comp rx=0 tx=0).stop 2024-09-21T02:05:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 -- 172.21.15.92:0/2584196930 shutdown_connections 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754cbf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548ea2f0 0x563754ccc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 --2- 172.21.15.92:0/2584196930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548d1d80 0x563754cb6220 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 -- 172.21.15.92:0/2584196930 >> 172.21.15.92:0/2584196930 conn(0x563754907300 msgr2=0x563754cbef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 -- 172.21.15.92:0/2584196930 shutdown_connections 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.563+0000 7f49f704b740 1 -- 172.21.15.92:0/2584196930 wait complete. 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 -- start start 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 Processor -- start 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548d1d80 0x563754cc9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754ccabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81ac450 con 0x5637548ea2f0 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81abd80 con 0x5637548f24d0 2024-09-21T02:05:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81a8fc0 con 0x5637548d1d80 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49f704b740 10 client.0 _ll_register_callbacks cb 0x7f49d80079f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55524/0 (socket says 172.21.15.92:55524) 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 -- 172.21.15.92:0/1279338018 learned_addr learned my addr 172.21.15.92:0/1279338018 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49edda3640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548d1d80 0x563754cc9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754ccabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 -- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548d1d80 msgr2=0x563754cc9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548d1d80 0x563754cc9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 -- 172.21.15.92:0/1279338018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 msgr2=0x563754ccabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754ccabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ed5a2640 1 -- 172.21.15.92:0/1279338018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563754ccb320 con 0x5637548ea2f0 2024-09-21T02:05:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.564+0000 7f49ee5a4640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754ccabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49ed5a2640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cca050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0e5240 tx=0x7f49dc0ad110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedc3c807ea2a352 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49cd7fa640 1 -- 172.21.15.92:0/1279338018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc09dae0 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49cd7fa640 1 -- 172.21.15.92:0/1279338018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f49dc097120 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d800aaf0 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49cd7fa640 1 -- 172.21.15.92:0/1279338018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f49dc09ac20 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563754ccc5f0 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49cd7fa640 1 -- 172.21.15.92:0/1279338018 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49d800aaf0 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49cd7fa640 1 -- 172.21.15.92:0/1279338018 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f49dc0a95b0 con 0x5637548ea2f0 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 20 client.15741 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637549131b0 0x563754d3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 4 client.15741 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:05:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.565+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563754c2b0f0 con 0x5637549131b0 2024-09-21T02:05:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.566+0000 7f49edda3640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637549131b0 0x563754d3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.566+0000 7f49cd7fa640 10 client.15741 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.566+0000 7f49edda3640 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637549131b0 0x563754d3e750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec39e6af69cc41ad server_cookie=da085e0ace8e4051 in_seq=0 out_seq=0 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49cd7fa640 1 -- 172.21.15.92:0/1279338018 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563754c2b0f0 con 0x5637549131b0 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49cd7fa640 10 client.15741 handle_command_reply: tid=0 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 client.15741 shutdown 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 msgr2=0x563754cca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cca050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0e5240 tx=0x7f49dc0ad110 comp rx=0 tx=0).stop 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 shutdown_connections 2024-09-21T02:05:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637549131b0 0x563754d3e750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637548f24d0 0x563754ccabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637548ea2f0 0x563754cca050 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 --2- 172.21.15.92:0/1279338018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637548d1d80 0x563754cc9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49cd7fa640 0 client.15741 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 >> 172.21.15.92:0/1279338018 conn(0x563754907300 msgr2=0x563754cbf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 shutdown_connections 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 1 -- 172.21.15.92:0/1279338018 wait complete. 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.575+0000 7f49f704b740 20 client.15741 trim_cache size 0 max 16384 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:05:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:05:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:05:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f49d00b6740 msgr2=0x7f49d00b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f49d00b6740 0x7f49d00b8c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f49e0031440 tx=0x7f49e00058d0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 msgr2=0x7f49e819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 0x7f49e819a4f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00c970 tx=0x7f49dc00ce30 comp rx=0 tx=0).stop 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 shutdown_connections 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f49d00b6740 0x7f49d00b8c50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49e8109410 0x7f49e819b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49e8106850 0x7f49e819a4f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 --2- 172.21.15.92:0/698134610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e806c4b0 0x7f49e8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 >> 172.21.15.92:0/698134610 conn(0x7f49e80fdf30 msgr2=0x7f49e8104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 shutdown_connections 2024-09-21T02:05:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:34.576+0000 7f49f0007640 1 -- 172.21.15.92:0/698134610 wait complete. 2024-09-21T02:05:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:35 smithi136 ceph-mon[34754]: pgmap v378: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T02:05:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:35 smithi092 ceph-mon[31190]: pgmap v378: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T02:05:35.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:35 smithi099 ceph-mon[34404]: pgmap v378: 97 pgs: 97 active+clean; 677 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T02:05:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:37 smithi136 ceph-mon[34754]: pgmap v379: 97 pgs: 97 active+clean; 708 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-21T02:05:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:37 smithi092 ceph-mon[31190]: pgmap v379: 97 pgs: 97 active+clean; 708 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-21T02:05:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:37 smithi099 ceph-mon[34404]: pgmap v379: 97 pgs: 97 active+clean; 708 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-21T02:05:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:39 smithi136 ceph-mon[34754]: pgmap v380: 97 pgs: 97 active+clean; 708 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-21T02:05:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:39 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:05:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:39 smithi092 ceph-mon[31190]: pgmap v380: 97 pgs: 97 active+clean; 708 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-21T02:05:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:39 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:05:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:39 smithi099 ceph-mon[34404]: pgmap v380: 97 pgs: 97 active+clean; 708 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-21T02:05:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:39 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:05:40.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: pgmap v381: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 17 op/s 2024-09-21T02:05:40.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:05:40.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: pgmap v381: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 17 op/s 2024-09-21T02:05:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:05:40.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: pgmap v381: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 17 op/s 2024-09-21T02:05:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:40.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:05:40.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:05:40.582 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:05:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:42 smithi136 ceph-mon[34754]: pgmap v382: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-09-21T02:05:42.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:42 smithi092 ceph-mon[31190]: pgmap v382: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-09-21T02:05:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:42 smithi099 ceph-mon[34404]: pgmap v382: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-09-21T02:05:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:44 smithi136 ceph-mon[34754]: pgmap v383: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-21T02:05:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:44 smithi136 ceph-mon[34754]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-21T02:05:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:44 smithi092 ceph-mon[31190]: pgmap v383: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-21T02:05:44.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:44 smithi092 ceph-mon[31190]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-21T02:05:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:44 smithi099 ceph-mon[34404]: pgmap v383: 97 pgs: 97 active+clean; 767 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-21T02:05:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:44 smithi099 ceph-mon[34404]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-21T02:05:46.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:46 smithi136 ceph-mon[34754]: pgmap v384: 97 pgs: 97 active+clean; 828 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-09-21T02:05:46.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:46 smithi092 ceph-mon[31190]: pgmap v384: 97 pgs: 97 active+clean; 828 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-09-21T02:05:46.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:46 smithi099 ceph-mon[34404]: pgmap v384: 97 pgs: 97 active+clean; 828 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-09-21T02:05:47.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 -- 172.21.15.92:0/1332966239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0109480 msgr2=0x7fd5e010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/1332966239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0109480 0x7fd5e010b920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009ca0 tx=0x7fd5d802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 -- 172.21.15.92:0/1332966239 shutdown_connections 2024-09-21T02:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/1332966239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0109480 0x7fd5e010b920 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/1332966239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0106b20 0x7fd5e0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/1332966239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 0x7fd5e0101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 -- 172.21.15.92:0/1332966239 >> 172.21.15.92:0/1332966239 conn(0x7fd5e00fd6d0 msgr2=0x7fd5e00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 -- 172.21.15.92:0/1332966239 shutdown_connections 2024-09-21T02:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.688+0000 7fd5e4abd640 1 -- 172.21.15.92:0/1332966239 wait complete. 2024-09-21T02:05:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 Processor -- start 2024-09-21T02:05:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 -- start start 2024-09-21T02:05:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 0x7fd5e0199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 0x7fd5e019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0109480 0x7fd5e019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e010f320 con 0x7fd5e0106b20 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e010f1a0 con 0x7fd5e0101a90 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5e4abd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e010f4a0 con 0x7fd5e0109480 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.689+0000 7fd5ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 0x7fd5e019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 0x7fd5e019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49596/0 (socket says 172.21.15.92:49596) 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/441220736 learned_addr learned my addr 172.21.15.92:0/441220736 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0109480 0x7fd5e019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5de59c640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 0x7fd5e0199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/441220736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0109480 msgr2=0x7fd5e019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0109480 0x7fd5e019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/441220736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 msgr2=0x7fd5e0199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 0x7fd5e0199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/441220736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e019b950 con 0x7fd5e0106b20 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0109480 0x7fd5e019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5de59c640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 0x7fd5e0199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 0x7fd5e019a580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400d770 tx=0x7fd5d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5cf7fe640 1 -- 172.21.15.92:0/441220736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4017070 con 0x7fd5e0106b20 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5cf7fe640 1 -- 172.21.15.92:0/441220736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d40054d0 con 0x7fd5e0106b20 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e019bc40 con 0x7fd5e0106b20 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5cf7fe640 1 -- 172.21.15.92:0/441220736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4004c30 con 0x7fd5e0106b20 2024-09-21T02:05:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.690+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01a2330 con 0x7fd5e0106b20 2024-09-21T02:05:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.693+0000 7fd5cf7fe640 1 -- 172.21.15.92:0/441220736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd5d4004e10 con 0x7fd5e0106b20 2024-09-21T02:05:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.694+0000 7fd5cf7fe640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5c80b6510 0x7fd5c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.694+0000 7fd5de59c640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5c80b6510 0x7fd5c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.694+0000 7fd5eba2f740 1 Processor -- start 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.694+0000 7fd5cf7fe640 1 -- 172.21.15.92:0/441220736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd5d40e6ec0 con 0x7fd5e0106b20 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.694+0000 7fd5eba2f740 1 -- start start 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.694+0000 7fd5de59c640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5c80b6510 0x7fd5c80b8a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009c20 tx=0x7fd5d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d388b1b0 0x5627d3c32e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c34010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627d3ba56f0 con 0x5627d38622f0 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5627d3b94b20 con 0x5627d386a4d0 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5627d3b98930 con 0x5627d388b1b0 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c334c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49600/0 (socket says 172.21.15.92:49600) 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1849960290 learned_addr learned my addr 172.21.15.92:0/1849960290 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5de59c640 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d388b1b0 0x5627d3c32e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c34010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1849960290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d388b1b0 msgr2=0x5627d3c32e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d388b1b0 0x5627d3c32e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1849960290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 msgr2=0x5627d3c34010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c34010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1849960290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d3b5a530 con 0x5627d38622f0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c334c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40a9600 tx=0x7fd5d40acc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a636612c5f745266 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.695+0000 7fd5ccff9640 1 -- 172.21.15.92:0/1849960290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40e4e60 con 0x5627d38622f0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.696+0000 7fd5ccff9640 1 -- 172.21.15.92:0/1849960290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d4092e50 con 0x5627d38622f0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.696+0000 7fd5ccff9640 1 -- 172.21.15.92:0/1849960290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4020070 con 0x5627d38622f0 2024-09-21T02:05:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.696+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1849960290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 msgr2=0x5627d3c334c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.696+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c334c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40a9600 tx=0x7fd5d40acc00 comp rx=0 tx=0).stop 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1849960290 shutdown_connections 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c34010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c334c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1849960290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d388b1b0 0x5627d3c32e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1849960290 >> 172.21.15.92:0/1849960290 conn(0x5627d387f300 msgr2=0x5627d3c2e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1849960290 shutdown_connections 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1849960290 wait complete. 2024-09-21T02:05:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 -- start start 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.697+0000 7fd5eba2f740 1 Processor -- start 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d38622f0 0x5627d3c3c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 0x5627d3c3d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5627d3accab0 con 0x5627d388b1b0 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5627d3b3a730 con 0x5627d386a4d0 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5627d3c34b30 con 0x5627d38622f0 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5eba2f740 10 client.0 _ll_register_callbacks cb 0x5627d3b8e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 0x5627d3c3d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 0x5627d3c3d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49614/0 (socket says 172.21.15.92:49614) 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 -- 172.21.15.92:0/944538850 learned_addr learned my addr 172.21.15.92:0/944538850 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5de59c640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d38622f0 0x5627d3c3c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 -- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d38622f0 msgr2=0x5627d3c3c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d38622f0 0x5627d3c3c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 -- 172.21.15.92:0/944538850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 msgr2=0x5627d3c3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c3cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 -- 172.21.15.92:0/944538850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d38f6790 con 0x5627d388b1b0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 0x5627d3c3d850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80094c0 tx=0x7fd5d8031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f1f5e6cac8ac0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.698+0000 7fd5c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/944538850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8003b80 con 0x5627d388b1b0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/944538850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd5d8003d20 con 0x5627d388b1b0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d4092e50 con 0x5627d388b1b0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/944538850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5d8004b40 con 0x5627d388b1b0 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 10 client.15753 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627d396f620 con 0x5627d388b1b0 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/944538850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d4092e50 con 0x5627d388b1b0 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/944538850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5d802f660 con 0x5627d388b1b0 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 10 client.15753 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 20 client.15753 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d3849d80 0x5627d3c49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 4 client.15753 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.699+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d3ba7640 con 0x5627d3849d80 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.700+0000 7fd5de59c640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d3849d80 0x5627d3c49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.700+0000 7fd5de59c640 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d3849d80 0x5627d3c49d80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ce46a55dc92c69 server_cookie=1fda76ac7fe0f5b0 in_seq=0 out_seq=0 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.700+0000 7fd5c5ffb640 10 client.15753 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/944538850 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627d3ba7640 con 0x5627d3849d80 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5c5ffb640 10 client.15753 handle_command_reply: tid=0 2024-09-21T02:05:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 client.15753 shutdown 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 msgr2=0x5627d3c3d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 0x5627d3c3d850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80094c0 tx=0x7fd5d8031910 comp rx=0 tx=0).stop 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 shutdown_connections 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d3849d80 0x5627d3c49d80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d388b1b0 0x5627d3c3d850 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d386a4d0 0x5627d3c3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/944538850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d38622f0 0x5627d3c3c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 >> 172.21.15.92:0/944538850 conn(0x5627d387f300 msgr2=0x5627d3c2cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5c5ffb640 0 client.15753 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 shutdown_connections 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 1 -- 172.21.15.92:0/944538850 wait complete. 2024-09-21T02:05:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.701+0000 7fd5eba2f740 20 client.15753 trim_cache size 0 max 16384 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 Processor -- start 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 -- start start 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c32df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c2e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01aafa0 con 0x5627d38622f0 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01acae0 con 0x5627d387f300 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01ab670 con 0x5627d386a4d0 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.716+0000 7fd5ddd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c33db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49622/0 (socket says 172.21.15.92:49622) 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1125115945 learned_addr learned my addr 172.21.15.92:0/1125115945 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5de59c640 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c32df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c2e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1125115945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 msgr2=0x5627d3c2e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c2e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1125115945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 msgr2=0x5627d3c32df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c32df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 -- 172.21.15.92:0/1125115945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d3c3a7e0 con 0x5627d38622f0 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c33db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40a97f0 tx=0x7fd5d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b484a764105980 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5c7fff640 1 -- 172.21.15.92:0/1125115945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40d4030 con 0x5627d38622f0 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5c7fff640 1 -- 172.21.15.92:0/1125115945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d40b1b10 con 0x5627d38622f0 2024-09-21T02:05:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.717+0000 7fd5c7fff640 1 -- 172.21.15.92:0/1125115945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4020040 con 0x5627d38622f0 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.718+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1125115945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 msgr2=0x5627d3c33db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.718+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c33db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40a97f0 tx=0x7fd5d4009f90 comp rx=0 tx=0).stop 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.718+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1125115945 shutdown_connections 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.718+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c2e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.718+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c33db0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.718+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/1125115945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1125115945 >> 172.21.15.92:0/1125115945 conn(0x5627d3c4a0b0 msgr2=0x5627d3c3afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1125115945 shutdown_connections 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- 172.21.15.92:0/1125115945 wait complete. 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- start start 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 Processor -- start 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c40c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c41e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01ac410 con 0x5627d38622f0 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01abd40 con 0x5627d387f300 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.719+0000 7fd5eba2f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01a8f80 con 0x5627d386a4d0 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5eba2f740 10 client.0 _ll_register_callbacks cb 0x7fd5d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c40c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c40c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49632/0 (socket says 172.21.15.92:49632) 2024-09-21T02:05:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 -- 172.21.15.92:0/3997923753 learned_addr learned my addr 172.21.15.92:0/3997923753 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c41e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 -- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 msgr2=0x5627d3c41350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c41350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 -- 172.21.15.92:0/3997923753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 msgr2=0x5627d3c41e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c41e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 -- 172.21.15.92:0/3997923753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d3c42560 con 0x5627d38622f0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5ded9d640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c41e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5de59c640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c40c90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0011c40 tx=0x7fd5d0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe9d15197b697b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/3997923753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0037060 con 0x5627d38622f0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/3997923753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd5d0003d80 con 0x5627d38622f0 2024-09-21T02:05:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627d3ba30f0 con 0x5627d38622f0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/3997923753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5d0012450 con 0x5627d38622f0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5eba2f740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.720+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627d3c43950 con 0x5627d38622f0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/3997923753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5627d3ba30f0 con 0x5627d38622f0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/3997923753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5d00325a0 con 0x5627d38622f0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5eba2f740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5eba2f740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5eba2f740 20 client.15759 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d388b1b0 0x5627d3cb6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5eba2f740 4 client.15759 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627d3ba0520 con 0x5627d388b1b0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.721+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d388b1b0 0x5627d3cb6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.722+0000 7fd5ddd9b640 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d388b1b0 0x5627d3cb6790 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4abd5d0a8259c0 server_cookie=15a762ff6a0bb438 in_seq=0 out_seq=0 2024-09-21T02:05:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.722+0000 7fd5c5ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5c5ffb640 1 -- 172.21.15.92:0/3997923753 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627d3ba0520 con 0x5627d388b1b0 2024-09-21T02:05:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5c5ffb640 10 client.15759 handle_command_reply: tid=0 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 client.15759 shutdown 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 msgr2=0x5627d3c40c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c40c90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0011c40 tx=0x7fd5d0014070 comp rx=0 tx=0).stop 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 shutdown_connections 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5627d388b1b0 0x5627d3cb6790 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627d387f300 0x5627d3c41e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5627d386a4d0 0x5627d3c41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 --2- 172.21.15.92:0/3997923753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627d38622f0 0x5627d3c40c90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5c5ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 >> 172.21.15.92:0/3997923753 conn(0x5627d3c4a0b0 msgr2=0x5627d3c3aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 shutdown_connections 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.734+0000 7fd5eba2f740 1 -- 172.21.15.92:0/3997923753 wait complete. 2024-09-21T02:05:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5eba2f740 20 client.15759 trim_cache size 0 max 16384 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5c80b6510 msgr2=0x7fd5c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5c80b6510 0x7fd5c80b8a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009c20 tx=0x7fd5d0005f90 comp rx=0 tx=0).stop 2024-09-21T02:05:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 msgr2=0x7fd5e019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 0x7fd5e019a580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400d770 tx=0x7fd5d400dc30 comp rx=0 tx=0).stop 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 shutdown_connections 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd5c80b6510 0x7fd5c80b8a20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e0109480 0x7fd5e019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5e0106b20 0x7fd5e019a580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 --2- 172.21.15.92:0/441220736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e0101a90 0x7fd5e0199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 >> 172.21.15.92:0/441220736 conn(0x7fd5e00fd6d0 msgr2=0x7fd5e01078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 shutdown_connections 2024-09-21T02:05:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:05:47.735+0000 7fd5e4abd640 1 -- 172.21.15.92:0/441220736 wait complete. 2024-09-21T02:05:47.867 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 -- 172.21.15.92:0/3551033110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c102a60 msgr2=0x7fc52c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 --2- 172.21.15.92:0/3551033110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c102a60 0x7fc52c102e70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009900 tx=0x7fc51c02a410 comp rx=0 tx=0).stop 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 -- 172.21.15.92:0/3551033110 shutdown_connections 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 --2- 172.21.15.92:0/3551033110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c108cf0 0x7fc52c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 --2- 172.21.15.92:0/3551033110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c1083a0 0x7fc52c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 --2- 172.21.15.92:0/3551033110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c102a60 0x7fc52c102e70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 -- 172.21.15.92:0/3551033110 >> 172.21.15.92:0/3551033110 conn(0x7fc52c079730 msgr2=0x7fc52c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 -- 172.21.15.92:0/3551033110 shutdown_connections 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.350+0000 7fc5321e5640 1 -- 172.21.15.92:0/3551033110 wait complete. 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.351+0000 7fc5321e5640 1 Processor -- start 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.351+0000 7fc5321e5640 1 -- start start 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.351+0000 7fc5321e5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c102a60 0x7fc52c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c1083a0 0x7fc52c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 0x7fc52c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c112960 con 0x7fc52c108cf0 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1127e0 con 0x7fc52c102a60 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c112ae0 con 0x7fc52c1083a0 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 0x7fc52c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 0x7fc52c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49664/0 (socket says 172.21.15.92:49664) 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 -- 172.21.15.92:0/2495243482 learned_addr learned my addr 172.21.15.92:0/2495243482 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c102a60 0x7fc52c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52affd640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c1083a0 0x7fc52c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 -- 172.21.15.92:0/2495243482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c1083a0 msgr2=0x7fc52c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c1083a0 0x7fc52c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 -- 172.21.15.92:0/2495243482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c102a60 msgr2=0x7fc52c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c102a60 0x7fc52c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 -- 172.21.15.92:0/2495243482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c19f320 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc52bfff640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 0x7fc52c19ebf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc52400cad0 tx=0x7fc52400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc528ff9640 1 -- 172.21.15.92:0/2495243482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400ae20 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c19f610 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc528ff9640 1 -- 172.21.15.92:0/2495243482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc524005020 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc528ff9640 1 -- 172.21.15.92:0/2495243482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400f6b0 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.352+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c1a5d70 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.355+0000 7fc528ff9640 1 -- 172.21.15.92:0/2495243482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc52400f850 con 0x7fc52c108cf0 2024-09-21T02:05:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.355+0000 7fc538759740 1 Processor -- start 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.355+0000 7fc538759740 1 -- start start 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc538759740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x5576808e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc538759740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55768050d1b0 0x5576808e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc538759740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc538759740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557680853e50 con 0x557680515390 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc538759740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557680843280 con 0x55768050d1b0 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc538759740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557680847090 con 0x557680536070 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808e94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49680/0 (socket says 172.21.15.92:49680) 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52affd640 1 -- 172.21.15.92:0/46115726 learned_addr learned my addr 172.21.15.92:0/46115726 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55768050d1b0 0x5576808e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52affd640 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x5576808e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc528ff9640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc5140b6510 0x7fc5140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52b7fe640 1 -- 172.21.15.92:0/46115726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 msgr2=0x5576808e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x5576808e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52b7fe640 1 -- 172.21.15.92:0/46115726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 msgr2=0x5576808e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52b7fe640 1 -- 172.21.15.92:0/46115726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576805a1650 con 0x55768050d1b0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52bfff640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc5140b6510 0x7fc5140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.356+0000 7fc52affd640 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.357+0000 7fc528ff9640 1 -- 172.21.15.92:0/2495243482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc5240e73b0 con 0x7fc52c108cf0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.357+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55768050d1b0 0x5576808e8c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009230 tx=0x7fc51c031e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf5a8d494693286 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.357+0000 7fc52bfff640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc5140b6510 0x7fc5140b8a20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5240ad600 tx=0x7fc524011000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.357+0000 7fc511ffb640 1 -- 172.21.15.92:0/46115726 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c040070 con 0x55768050d1b0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.357+0000 7fc511ffb640 1 -- 172.21.15.92:0/46115726 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc51c0316a0 con 0x55768050d1b0 2024-09-21T02:05:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.357+0000 7fc511ffb640 1 -- 172.21.15.92:0/46115726 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c035040 con 0x55768050d1b0 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 -- 172.21.15.92:0/46115726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55768050d1b0 msgr2=0x5576808e8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55768050d1b0 0x5576808e8c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009230 tx=0x7fc51c031e50 comp rx=0 tx=0).stop 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 -- 172.21.15.92:0/46115726 shutdown_connections 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55768050d1b0 0x5576808e8c70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 --2- 172.21.15.92:0/46115726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x5576808e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 -- 172.21.15.92:0/46115726 >> 172.21.15.92:0/46115726 conn(0x55768052a460 msgr2=0x5576808daeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 -- 172.21.15.92:0/46115726 shutdown_connections 2024-09-21T02:05:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.358+0000 7fc538759740 1 -- 172.21.15.92:0/46115726 wait complete. 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 -- start start 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 Processor -- start 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680536070 0x5576808f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5576806a4700 con 0x557680515390 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5576807de110 con 0x557680536070 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5576808e9fb0 con 0x55768050d1b0 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc538759740 10 client.0 _ll_register_callbacks cb 0x5576807f7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.359+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49690/0 (socket says 172.21.15.92:49690) 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 -- 172.21.15.92:0/3154344121 learned_addr learned my addr 172.21.15.92:0/3154344121 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52affd640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680536070 0x5576808f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 -- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 msgr2=0x5576808ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 -- 172.21.15.92:0/3154344121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680536070 msgr2=0x5576808f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680536070 0x5576808f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 -- 172.21.15.92:0/3154344121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768080bc40 con 0x557680515390 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52affd640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680536070 0x5576808f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808ede90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc51c031a80 tx=0x7fc51c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df17b5479ea1b424 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc50affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc50affd640 1 -- 172.21.15.92:0/3154344121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c040030 con 0x557680515390 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc50affd640 1 -- 172.21.15.92:0/3154344121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc51c004050 con 0x557680515390 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51c0316a0 con 0x557680515390 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc50affd640 1 -- 172.21.15.92:0/3154344121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc51c035040 con 0x557680515390 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc538759740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.360+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576808fbe00 con 0x557680515390 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc50affd640 1 -- 172.21.15.92:0/3154344121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc51c0316a0 con 0x557680515390 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc50affd640 1 -- 172.21.15.92:0/3154344121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc51c0035f0 con 0x557680515390 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc538759740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc538759740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc538759740 20 client.15771 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc538759740 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804f49f0 0x5576808ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc538759740 4 client.15771 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557680855da0 con 0x5576804f49f0 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.361+0000 7fc52affd640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804f49f0 0x5576808ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.362+0000 7fc50affd640 10 client.15771 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.362+0000 7fc52affd640 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804f49f0 0x5576808ff160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0ae235a2e9d2a9 server_cookie=66c387ca4303500b in_seq=0 out_seq=0 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc50affd640 1 -- 172.21.15.92:0/3154344121 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557680855da0 con 0x5576804f49f0 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc50affd640 10 client.15771 handle_command_reply: tid=0 2024-09-21T02:05:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 client.15771 shutdown 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 msgr2=0x5576808ede90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808ede90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc51c031a80 tx=0x7fc51c02a410 comp rx=0 tx=0).stop 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 shutdown_connections 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804f49f0 0x5576808ff160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680536070 0x5576808f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680515390 0x5576808ede90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc50affd640 0 client.15771 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 --2- 172.21.15.92:0/3154344121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 >> 172.21.15.92:0/3154344121 conn(0x55768052a460 msgr2=0x5576808dae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 shutdown_connections 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.363+0000 7fc538759740 1 -- 172.21.15.92:0/3154344121 wait complete. 2024-09-21T02:05:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.364+0000 7fc538759740 20 client.15771 trim_cache size 0 max 16384 2024-09-21T02:05:48.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 Processor -- start 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 -- start start 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 0x5576808f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1ad1d0 con 0x557680536070 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1aed10 con 0x557680515390 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc538759740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1ad8a0 con 0x55768050d1b0 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 0x5576808f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 0x5576808f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49704/0 (socket says 172.21.15.92:49704) 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 -- 172.21.15.92:0/3281642874 learned_addr learned my addr 172.21.15.92:0/3281642874 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 -- 172.21.15.92:0/3281642874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 msgr2=0x5576808edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808edfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 -- 172.21.15.92:0/3281642874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 msgr2=0x5576808dab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.381+0000 7fc52affd640 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808dab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.382+0000 7fc52affd640 1 -- 172.21.15.92:0/3281642874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576808ebbd0 con 0x557680536070 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.382+0000 7fc52affd640 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 0x5576808f2bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc5200133b0 tx=0x7fc520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37686fd07594168 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.382+0000 7fc5117fa640 1 -- 172.21.15.92:0/3281642874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52001c040 con 0x557680536070 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.382+0000 7fc5117fa640 1 -- 172.21.15.92:0/3281642874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc520005020 con 0x557680536070 2024-09-21T02:05:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.382+0000 7fc5117fa640 1 -- 172.21.15.92:0/3281642874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520011b70 con 0x557680536070 2024-09-21T02:05:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 -- 172.21.15.92:0/3281642874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 msgr2=0x5576808f2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 0x5576808f2bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc5200133b0 tx=0x7fc520004730 comp rx=0 tx=0).stop 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 -- 172.21.15.92:0/3281642874 shutdown_connections 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557680536070 0x5576808f2bf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 --2- 172.21.15.92:0/3281642874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55768050d1b0 0x5576808edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 -- 172.21.15.92:0/3281642874 >> 172.21.15.92:0/3281642874 conn(0x55768052a460 msgr2=0x5576808ed890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 -- 172.21.15.92:0/3281642874 shutdown_connections 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 -- 172.21.15.92:0/3281642874 wait complete. 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 -- start start 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.383+0000 7fc538759740 1 Processor -- start 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 0x5576808f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x557680961fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1ae640 con 0x55768050d1b0 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1adf70 con 0x557680515390 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1ab1b0 con 0x557680536070 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc538759740 10 client.0 _ll_register_callbacks cb 0x7fc52000e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 0x5576808f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 0x5576808f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49708/0 (socket says 172.21.15.92:49708) 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 -- 172.21.15.92:0/3969768649 learned_addr learned my addr 172.21.15.92:0/3969768649 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 -- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 msgr2=0x557680961fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x557680961fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 -- 172.21.15.92:0/3969768649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 msgr2=0x5576808f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 -- 172.21.15.92:0/3969768649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557680962610 con 0x55768050d1b0 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc52affd640 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 0x5576808f93e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc5200135d0 tx=0x7fc520013600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e1dc9a752e2db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:48.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.384+0000 7fc50affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc50affd640 1 -- 172.21.15.92:0/3969768649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200053d0 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc50affd640 1 -- 172.21.15.92:0/3969768649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc520005570 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc520005020 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc50affd640 1 -- 172.21.15.92:0/3969768649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5200118f0 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557680969a10 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc50affd640 1 -- 172.21.15.92:0/3969768649 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc520005020 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc50affd640 1 -- 172.21.15.92:0/3969768649 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc520025410 con 0x55768050d1b0 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 20 client.15777 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804c9da0 0x55768096c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 4 client.15777 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:05:48.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.385+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557680851850 con 0x5576804c9da0 2024-09-21T02:05:48.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.387+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804c9da0 0x55768096c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:48.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.389+0000 7fc52b7fe640 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804c9da0 0x55768096c0f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b205d57471fd3ce server_cookie=8bd11e7eed1028ac in_seq=0 out_seq=0 2024-09-21T02:05:48.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.389+0000 7fc50affd640 10 client.15777 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.390+0000 7fc50affd640 1 -- 172.21.15.92:0/3969768649 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x557680851850 con 0x5576804c9da0 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.390+0000 7fc50affd640 10 client.15777 handle_command_reply: tid=0 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.390+0000 7fc538759740 1 client.15777 shutdown 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 msgr2=0x5576808f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 0x5576808f93e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc5200135d0 tx=0x7fc520013600 comp rx=0 tx=0).stop 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 shutdown_connections 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5576804c9da0 0x55768096c0f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557680536070 0x557680961fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557680515390 0x5576808f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 --2- 172.21.15.92:0/3969768649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55768050d1b0 0x5576808f93e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 >> 172.21.15.92:0/3969768649 conn(0x55768052a460 msgr2=0x5576808e53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc50affd640 0 client.15777 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:48.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 shutdown_connections 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 1 -- 172.21.15.92:0/3969768649 wait complete. 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc538759740 20 client.15777 trim_cache size 0 max 16384 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:05:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:05:48.395 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:05:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc5140b6510 msgr2=0x7fc5140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc5321e5640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc5140b6510 0x7fc5140b8a20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5240ad600 tx=0x7fc524011000 comp rx=0 tx=0).stop 2024-09-21T02:05:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 msgr2=0x7fc52c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.391+0000 7fc5321e5640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 0x7fc52c19ebf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc52400cad0 tx=0x7fc52400cf90 comp rx=0 tx=0).stop 2024-09-21T02:05:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 shutdown_connections 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc5140b6510 0x7fc5140b8a20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc52c108cf0 0x7fc52c19ebf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c1083a0 0x7fc52c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 --2- 172.21.15.92:0/2495243482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c102a60 0x7fc52c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 >> 172.21.15.92:0/2495243482 conn(0x7fc52c079730 msgr2=0x7fc52c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 shutdown_connections 2024-09-21T02:05:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:48.392+0000 7fc5321e5640 1 -- 172.21.15.92:0/2495243482 wait complete. 2024-09-21T02:05:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:48 smithi136 ceph-mon[34754]: pgmap v385: 97 pgs: 97 active+clean; 828 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:05:48.543 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:48 smithi092 ceph-mon[31190]: pgmap v385: 97 pgs: 97 active+clean; 828 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:05:48.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:48 smithi099 ceph-mon[34404]: pgmap v385: 97 pgs: 97 active+clean; 828 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:05:49.115 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:05:49.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:05:49.373 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 -- 172.21.15.92:0/3547939750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105b90 msgr2=0x7fa85810f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 --2- 172.21.15.92:0/3547939750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105b90 0x7fa85810f5e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa850009ca0 tx=0x7fa85002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 -- 172.21.15.92:0/3547939750 shutdown_connections 2024-09-21T02:05:49.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 --2- 172.21.15.92:0/3547939750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105b90 0x7fa85810f5e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 --2- 172.21.15.92:0/3547939750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858105240 0x7fa858105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 --2- 172.21.15.92:0/3547939750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa858103270 0x7fa858103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 -- 172.21.15.92:0/3547939750 >> 172.21.15.92:0/3547939750 conn(0x7fa858079ec0 msgr2=0x7fa85807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.855+0000 7fa85ddad640 1 -- 172.21.15.92:0/3547939750 shutdown_connections 2024-09-21T02:05:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.856+0000 7fa85ddad640 1 -- 172.21.15.92:0/3547939750 wait complete. 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.856+0000 7fa85ddad640 1 Processor -- start 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.856+0000 7fa85ddad640 1 -- start start 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa85ddad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858103270 0x7fa85819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa85ddad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 0x7fa85819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa85ddad640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa858105b90 0x7fa85819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa85ddad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa858112a30 con 0x7fa858105240 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa85ddad640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581128b0 con 0x7fa858103270 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa85ddad640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa858112bb0 con 0x7fa858105b90 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 0x7fa85819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 0x7fa85819b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49734/0 (socket says 172.21.15.92:49734) 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 -- 172.21.15.92:0/1798797781 learned_addr learned my addr 172.21.15.92:0/1798797781 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa8577fe640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858103270 0x7fa85819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa857fff640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa858105b90 0x7fa85819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 -- 172.21.15.92:0/1798797781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa858105b90 msgr2=0x7fa85819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa858105b90 0x7fa85819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 -- 172.21.15.92:0/1798797781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858103270 msgr2=0x7fa85819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858103270 0x7fa85819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 -- 172.21.15.92:0/1798797781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a6540 con 0x7fa858105240 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa8577fe640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858103270 0x7fa85819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.857+0000 7fa856ffd640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 0x7fa85819b7c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00d770 tx=0x7fa84c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.858+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c017070 con 0x7fa858105240 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.858+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84c009c30 con 0x7fa858105240 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.858+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a6830 con 0x7fa858105240 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.858+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00f620 con 0x7fa858105240 2024-09-21T02:05:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.858+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581a6dc0 con 0x7fa858105240 2024-09-21T02:05:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.859+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa858103680 con 0x7fa858105240 2024-09-21T02:05:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.861+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa84c01a070 con 0x7fa858105240 2024-09-21T02:05:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.862+0000 7fa854ff9640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa8400b6590 0x7fa8400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.862+0000 7fa8577fe640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa8400b6590 0x7fa8400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.862+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa84c0e6d50 con 0x7fa858105240 2024-09-21T02:05:49.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.863+0000 7fa8577fe640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa8400b6590 0x7fa8400b8aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa858102a50 tx=0x7fa848005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:49.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:49.866+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa84c009de0 con 0x7fa858105240 2024-09-21T02:05:50.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.081+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa85810ac00 con 0x7fa858105240 2024-09-21T02:05:50.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.081+0000 7fa854ff9640 1 -- 172.21.15.92:0/1798797781 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7fa84c0b19c0 con 0x7fa858105240 2024-09-21T02:05:50.082 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:05:50.082 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:05:50.083 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:05:50.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa8400b6590 msgr2=0x7fa8400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:50.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa8400b6590 0x7fa8400b8aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa858102a50 tx=0x7fa848005f90 comp rx=0 tx=0).stop 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 msgr2=0x7fa85819b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 0x7fa85819b7c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00d770 tx=0x7fa84c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 shutdown_connections 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa8400b6590 0x7fa8400b8aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa858105b90 0x7fa85819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858105240 0x7fa85819b7c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 --2- 172.21.15.92:0/1798797781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa858103270 0x7fa85819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 >> 172.21.15.92:0/1798797781 conn(0x7fa858079ec0 msgr2=0x7fa858106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 shutdown_connections 2024-09-21T02:05:50.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:50.084+0000 7fa85ddad640 1 -- 172.21.15.92:0/1798797781 wait complete. 2024-09-21T02:05:50.401 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:50 smithi092 ceph-mon[31190]: pgmap v386: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 24 op/s 2024-09-21T02:05:50.401 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:50 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1798797781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:05:50.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:50 smithi136 ceph-mon[34754]: pgmap v386: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 24 op/s 2024-09-21T02:05:50.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:50 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1798797781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:05:50.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:50 smithi099 ceph-mon[34404]: pgmap v386: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 24 op/s 2024-09-21T02:05:50.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:50 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1798797781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:05:50.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:05:51.099 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 -- 172.21.15.92:0/3290792213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd48108cf0 msgr2=0x7fdd4810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 --2- 172.21.15.92:0/3290792213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd48108cf0 0x7fdd4810f520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009900 tx=0x7fdd4002a410 comp rx=0 tx=0).stop 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 -- 172.21.15.92:0/3290792213 shutdown_connections 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 --2- 172.21.15.92:0/3290792213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd48108cf0 0x7fdd4810f520 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 --2- 172.21.15.92:0/3290792213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd481083a0 0x7fdd481087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 --2- 172.21.15.92:0/3290792213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd48102a60 0x7fdd48102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 -- 172.21.15.92:0/3290792213 >> 172.21.15.92:0/3290792213 conn(0x7fdd48079700 msgr2=0x7fdd48079b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 -- 172.21.15.92:0/3290792213 shutdown_connections 2024-09-21T02:05:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.588+0000 7fdd50780640 1 -- 172.21.15.92:0/3290792213 wait complete. 2024-09-21T02:05:51.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 Processor -- start 2024-09-21T02:05:51.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 -- start start 2024-09-21T02:05:51.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd48102a60 0x7fdd48110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 0x7fdd48110540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd48108cf0 0x7fdd48110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48113ad0 con 0x7fdd481083a0 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48113950 con 0x7fdd48108cf0 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd50780640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48113c50 con 0x7fdd48102a60 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 0x7fdd48110540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 0x7fdd48110540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49760/0 (socket says 172.21.15.92:49760) 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/2392201112 learned_addr learned my addr 172.21.15.92:0/2392201112 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.589+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd48102a60 0x7fdd48110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd48108cf0 0x7fdd48110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/2392201112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd48102a60 msgr2=0x7fdd48110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd48102a60 0x7fdd48110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/2392201112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd48108cf0 msgr2=0x7fdd48110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd48108cf0 0x7fdd48110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.592 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/2392201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd481a8720 con 0x7fdd481083a0 2024-09-21T02:05:51.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 0x7fdd48110540 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c970 tx=0x7fdd3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:51.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd477fe640 1 -- 172.21.15.92:0/2392201112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004220 con 0x7fdd481083a0 2024-09-21T02:05:51.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd481a8a10 con 0x7fdd481083a0 2024-09-21T02:05:51.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd477fe640 1 -- 172.21.15.92:0/2392201112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c0043c0 con 0x7fdd481083a0 2024-09-21T02:05:51.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd477fe640 1 -- 172.21.15.92:0/2392201112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004a00 con 0x7fdd481083a0 2024-09-21T02:05:51.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.590+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd481a8f50 con 0x7fdd481083a0 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.592+0000 7fdd56cf4740 1 Processor -- start 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 -- start start 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd477fe640 1 -- 172.21.15.92:0/2392201112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fdd3c0079b0 con 0x7fdd481083a0 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 0x55dc80c426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc808721b0 0x55dc80c42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dc80bb8ea0 con 0x55dc8087a390 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dc80ba82d0 con 0x55dc808721b0 2024-09-21T02:05:51.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc80bac0e0 con 0x55dc8089b070 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd4ed1d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 0x55dc80c426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd4ed1d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 0x55dc80c426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:57542/0 (socket says 172.21.15.92:57542) 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd4ed1d640 1 -- 172.21.15.92:0/2828105203 learned_addr learned my addr 172.21.15.92:0/2828105203 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.593+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc808721b0 0x55dc80c42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4ed1d640 1 -- 172.21.15.92:0/2828105203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc808721b0 msgr2=0x55dc80c42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc808721b0 0x55dc80c42be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4ed1d640 1 -- 172.21.15.92:0/2828105203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 msgr2=0x55dc80c43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4ed1d640 1 -- 172.21.15.92:0/2828105203 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc80906650 con 0x55dc8089b070 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd477fe640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd300b6590 0x7fdd300b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd300b6590 0x7fdd300b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 0x55dc80c426a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd40005940 tx=0x7fdd40032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c1b7492cc9fd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:51.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd477fe640 1 -- 172.21.15.92:0/2392201112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fdd3c0e7000 con 0x7fdd481083a0 2024-09-21T02:05:51.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd44ff9640 1 -- 172.21.15.92:0/2828105203 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4003f070 con 0x55dc8089b070 2024-09-21T02:05:51.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.594+0000 7fdd44ff9640 1 -- 172.21.15.92:0/2828105203 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd400316a0 con 0x55dc8089b070 2024-09-21T02:05:51.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd44ff9640 1 -- 172.21.15.92:0/2828105203 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40035070 con 0x55dc8089b070 2024-09-21T02:05:51.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 -- 172.21.15.92:0/2828105203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 msgr2=0x55dc80c426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd300b6590 0x7fdd300b8aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55dc80c41320 tx=0x7fdd3800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:51.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 0x55dc80c426a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd40005940 tx=0x7fdd40032ce0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 -- 172.21.15.92:0/2828105203 shutdown_connections 2024-09-21T02:05:51.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc808721b0 0x55dc80c42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/2828105203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc8089b070 0x55dc80c426a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.595+0000 7fdd56cf4740 1 -- 172.21.15.92:0/2828105203 >> 172.21.15.92:0/2828105203 conn(0x55dc8088f460 msgr2=0x55dc80c40590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 -- 172.21.15.92:0/2828105203 shutdown_connections 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 -- 172.21.15.92:0/2828105203 wait complete. 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 -- start start 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 Processor -- start 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c4dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c4e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c5a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dc80a09700 con 0x55dc8087a390 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dc80b43160 con 0x55dc8089b070 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc80c50e20 con 0x55dc808721b0 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd56cf4740 10 client.0 _ll_register_callbacks cb 0x55dc80b5c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:51.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.596+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c4e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4e51c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c4dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c4e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49786/0 (socket says 172.21.15.92:49786) 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/3871067581 learned_addr learned my addr 172.21.15.92:0/3871067581 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c5a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/3871067581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 msgr2=0x55dc80c4dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c4dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/3871067581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 msgr2=0x55dc80c5a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c5a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/3871067581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc80b70c90 con 0x55dc8087a390 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c4e230 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0aa3e0 tx=0x7fdd3c00a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987529b806399f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:51.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd2dffb640 1 -- 172.21.15.92:0/3871067581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01d040 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd400316a0 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd2dffb640 1 -- 172.21.15.92:0/3871067581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fdd3c0a8ac0 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd2dffb640 1 -- 172.21.15.92:0/3871067581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd3c0acf40 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd56cf4740 10 client.15795 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.597+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc80c5cb50 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd2dffb640 1 -- 172.21.15.92:0/3871067581 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd400316a0 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd2dffb640 1 -- 172.21.15.92:0/3871067581 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdd3c0a8cc0 con 0x55dc8087a390 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd56cf4740 10 client.15795 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd56cf4740 10 client.15795 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:05:51.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd56cf4740 20 client.15795 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc808599f0 0x55dc80c5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd56cf4740 4 client.15795 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc80bbadf0 con 0x55dc808599f0 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc808599f0 0x55dc80c5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc808599f0 0x55dc80c5feb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c7ec451fb1c0ac4 server_cookie=67da3937b05da350 in_seq=0 out_seq=0 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.598+0000 7fdd2dffb640 10 client.15795 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd2dffb640 1 -- 172.21.15.92:0/3871067581 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc80bbadf0 con 0x55dc808599f0 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd2dffb640 10 client.15795 handle_command_reply: tid=0 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 client.15795 shutdown 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 msgr2=0x55dc80c4e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c4e230 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0aa3e0 tx=0x7fdd3c00a540 comp rx=0 tx=0).stop 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 shutdown_connections 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc808599f0 0x55dc80c5feb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c5a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c4e230 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/3871067581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c4dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 >> 172.21.15.92:0/3871067581 conn(0x55dc8088f460 msgr2=0x55dc80c45830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd2dffb640 0 client.15795 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 shutdown_connections 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 1 -- 172.21.15.92:0/3871067581 wait complete. 2024-09-21T02:05:51.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.599+0000 7fdd56cf4740 20 client.15795 trim_cache size 0 max 16384 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.612+0000 7fdd56cf4740 1 Processor -- start 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.612+0000 7fdd56cf4740 1 -- start start 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c43de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c5a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c4aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481b15f0 con 0x55dc8087a390 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481b3130 con 0x55dc8089b070 2024-09-21T02:05:51.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481b1cc0 con 0x55dc808721b0 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c5a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c5a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49792/0 (socket says 172.21.15.92:49792) 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/1482982922 learned_addr learned my addr 172.21.15.92:0/1482982922 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c4aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c43de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/1482982922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 msgr2=0x55dc80c43de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c43de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/1482982922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 msgr2=0x55dc80c4aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c4aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/1482982922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc80c4c1f0 con 0x55dc8087a390 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c5a040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0058c0 tx=0x7fdd3c0b1d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed910af3e4964ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.613+0000 7fdd2ffff640 1 -- 172.21.15.92:0/1482982922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01d040 con 0x55dc8087a390 2024-09-21T02:05:51.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd2ffff640 1 -- 172.21.15.92:0/1482982922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c0d0040 con 0x55dc8087a390 2024-09-21T02:05:51.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd2ffff640 1 -- 172.21.15.92:0/1482982922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0ac8e0 con 0x55dc8087a390 2024-09-21T02:05:51.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 -- 172.21.15.92:0/1482982922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 msgr2=0x55dc80c5a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c5a040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0058c0 tx=0x7fdd3c0b1d70 comp rx=0 tx=0).stop 2024-09-21T02:05:51.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 -- 172.21.15.92:0/1482982922 shutdown_connections 2024-09-21T02:05:51.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80c4aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c5a040 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/1482982922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c43de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 -- 172.21.15.92:0/1482982922 >> 172.21.15.92:0/1482982922 conn(0x55dc8088f460 msgr2=0x55dc80c436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 -- 172.21.15.92:0/1482982922 shutdown_connections 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.614+0000 7fdd56cf4740 1 -- 172.21.15.92:0/1482982922 wait complete. 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 -- start start 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 Processor -- start 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c52bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c53190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80cc3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481b2a60 con 0x55dc8087a390 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481b2390 con 0x55dc8089b070 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481af5d0 con 0x55dc808721b0 2024-09-21T02:05:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd56cf4740 10 client.0 _ll_register_callbacks cb 0x7fdd3c00c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c53190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd4dd1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c53190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49802/0 (socket says 172.21.15.92:49802) 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.615+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/544305501 learned_addr learned my addr 172.21.15.92:0/544305501 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80cc3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/544305501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 msgr2=0x55dc80c52bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c52bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/544305501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 msgr2=0x55dc80cc3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80cc3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4dd1b640 1 -- 172.21.15.92:0/544305501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc80cc3fc0 con 0x55dc8087a390 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4ed1d640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80cc3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd4dd1b640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c53190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0ad100 tx=0x7fdd3c098420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b1302208da0619 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd2dffb640 1 -- 172.21.15.92:0/544305501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01d070 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd2dffb640 1 -- 172.21.15.92:0/544305501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fdd3c0abe40 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3c0d0040 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd2dffb640 1 -- 172.21.15.92:0/544305501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd3c0acac0 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd56cf4740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc80ccb390 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.616+0000 7fdd2dffb640 1 -- 172.21.15.92:0/544305501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c0d0040 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd2dffb640 1 -- 172.21.15.92:0/544305501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdd3c0b2de0 con 0x55dc8087a390 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 10 client.15801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 20 client.15801 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc8082eda0 0x55dc80ccd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 4 client.15801 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc80bb68a0 con 0x55dc8082eda0 2024-09-21T02:05:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc8082eda0 0x55dc80ccd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd2dffb640 10 client.15801 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd4e51c640 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc8082eda0 0x55dc80ccd920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9535b2f79c1cc6a3 server_cookie=4f27ca463e03a206 in_seq=0 out_seq=0 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd2dffb640 1 -- 172.21.15.92:0/544305501 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc80bb68a0 con 0x55dc8082eda0 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd2dffb640 10 client.15801 handle_command_reply: tid=0 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 client.15801 shutdown 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 msgr2=0x55dc80c53190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c53190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0ad100 tx=0x7fdd3c098420 comp rx=0 tx=0).stop 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 shutdown_connections 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55dc8082eda0 0x55dc80ccd920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd2dffb640 0 client.15801 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.617+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc8089b070 0x55dc80cc3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc8087a390 0x55dc80c53190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd56cf4740 1 --2- 172.21.15.92:0/544305501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc808721b0 0x55dc80c52bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 >> 172.21.15.92:0/544305501 conn(0x55dc8088f460 msgr2=0x55dc80c40560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 shutdown_connections 2024-09-21T02:05:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd56cf4740 1 -- 172.21.15.92:0/544305501 wait complete. 2024-09-21T02:05:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd56cf4740 20 client.15801 trim_cache size 0 max 16384 2024-09-21T02:05:51.624 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd300b6590 msgr2=0x7fdd300b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd300b6590 0x7fdd300b8aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55dc80c41320 tx=0x7fdd3800a590 comp rx=0 tx=0).stop 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 msgr2=0x7fdd48110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 0x7fdd48110540 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c970 tx=0x7fdd3c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 shutdown_connections 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdd300b6590 0x7fdd300b8aa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd48108cf0 0x7fdd48110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd481083a0 0x7fdd48110540 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 --2- 172.21.15.92:0/2392201112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd48102a60 0x7fdd48110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 >> 172.21.15.92:0/2392201112 conn(0x7fdd48079700 msgr2=0x7fdd481063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:51.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.618+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 shutdown_connections 2024-09-21T02:05:51.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:51.619+0000 7fdd50780640 1 -- 172.21.15.92:0/2392201112 wait complete. 2024-09-21T02:05:52.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:05:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:52.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:52 smithi136 ceph-mon[34754]: pgmap v387: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:05:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:52 smithi092 ceph-mon[31190]: pgmap v387: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:05:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:52 smithi099 ceph-mon[34404]: pgmap v387: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:05:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3484576877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f38108d10 msgr2=0x7f4f3810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3484576877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f38108d10 0x7f4f3810f540 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009ca0 tx=0x7f4f3002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3484576877 shutdown_connections 2024-09-21T02:05:52.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3484576877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f38108d10 0x7f4f3810f540 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3484576877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f381083c0 0x7f4f381087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3484576877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38102a80 0x7f4f38102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3484576877 >> 172.21.15.92:0/3484576877 conn(0x7f4f38079740 msgr2=0x7f4f38079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3484576877 shutdown_connections 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.996+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3484576877 wait complete. 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 Processor -- start 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 -- start start 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f38102a80 0x7f4f3819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 0x7f4f3819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38108d10 0x7f4f3819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381129e0 con 0x7f4f381083c0 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38112860 con 0x7f4f38108d10 2024-09-21T02:05:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3fcd6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38112b60 con 0x7f4f38102a80 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3d271640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 0x7f4f3819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3d271640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 0x7f4f3819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49820/0 (socket says 172.21.15.92:49820) 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.997+0000 7f4f3d271640 1 -- 172.21.15.92:0/3419076803 learned_addr learned my addr 172.21.15.92:0/3419076803 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3e273640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38108d10 0x7f4f3819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3da72640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f38102a80 0x7f4f3819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3d271640 1 -- 172.21.15.92:0/3419076803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f38102a80 msgr2=0x7f4f3819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3d271640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f38102a80 0x7f4f3819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3d271640 1 -- 172.21.15.92:0/3419076803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38108d10 msgr2=0x7f4f3819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3d271640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38108d10 0x7f4f3819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3d271640 1 -- 172.21.15.92:0/3419076803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f381a6550 con 0x7f4f381083c0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3d271640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 0x7f4f3819b7d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b450 tx=0x7f4f2800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f2effd640 1 -- 172.21.15.92:0/3419076803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28015020 con 0x7f4f381083c0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f2effd640 1 -- 172.21.15.92:0/3419076803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f28011d40 con 0x7f4f381083c0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f2effd640 1 -- 172.21.15.92:0/3419076803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800f340 con 0x7f4f381083c0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f381a6840 con 0x7f4f381083c0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:52.998+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381a6db0 con 0x7f4f381083c0 2024-09-21T02:05:53.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 Processor -- start 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f2effd640 1 -- 172.21.15.92:0/3419076803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4f28018020 con 0x7f4f381083c0 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 -- start start 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c0506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c050be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 0x55d11c051120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d11bfc6ea0 con 0x55d11bc88390 2024-09-21T02:05:53.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d11bfb62d0 con 0x55d11bca9070 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f4624a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d11bfba0e0 con 0x55d11bc801b0 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f3d271640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 0x55d11c051120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f3e273640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c0506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f3e273640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c0506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:50640/0 (socket says 172.21.15.92:50640) 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f3d271640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 0x55d11c051120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49826/0 (socket says 172.21.15.92:49826) 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.001+0000 7f4f3e273640 1 -- 172.21.15.92:0/479393620 learned_addr learned my addr 172.21.15.92:0/479393620 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3da72640 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c050be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 -- 172.21.15.92:0/479393620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 msgr2=0x55d11c050be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c050be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 -- 172.21.15.92:0/479393620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 msgr2=0x55d11c051120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f2effd640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f240b6510 0x7f4f240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 0x55d11c051120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 -- 172.21.15.92:0/479393620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11bd14650 con 0x55d11bca9070 2024-09-21T02:05:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3d271640 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 0x55d11c051120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f2effd640 1 -- 172.21.15.92:0/3419076803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4f280e6c70 con 0x7f4f381083c0 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f240b6510 0x7f4f240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.002+0000 7f4f3e273640 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c0506a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009920 tx=0x7f4f30009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34a98143f65dca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.003+0000 7f4f23fff640 1 -- 172.21.15.92:0/479393620 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30031e10 con 0x55d11bca9070 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.003+0000 7f4f23fff640 1 -- 172.21.15.92:0/479393620 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30036030 con 0x55d11bca9070 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.003+0000 7f4f23fff640 1 -- 172.21.15.92:0/479393620 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3002f660 con 0x55d11bca9070 2024-09-21T02:05:53.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.003+0000 7f4f3e273640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f240b6510 0x7f4f240b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4f3004a000 tx=0x7f4f3004b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 -- 172.21.15.92:0/479393620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 msgr2=0x55d11c0506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c0506a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009920 tx=0x7f4f30009950 comp rx=0 tx=0).stop 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 -- 172.21.15.92:0/479393620 shutdown_connections 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc88390 0x55d11c051120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c050be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 --2- 172.21.15.92:0/479393620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c0506a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 -- 172.21.15.92:0/479393620 >> 172.21.15.92:0/479393620 conn(0x55d11bc9d460 msgr2=0x55d11c04e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 -- 172.21.15.92:0/479393620 shutdown_connections 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 -- 172.21.15.92:0/479393620 wait complete. 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 -- start start 2024-09-21T02:05:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.004+0000 7f4f4624a740 1 Processor -- start 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c05bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc88390 0x55d11c05c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d11be17700 con 0x55d11bca9070 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d11bf51160 con 0x55d11bc88390 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d11c05ed00 con 0x55d11bc801b0 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f4624a740 10 client.0 _ll_register_callbacks cb 0x55d11bf6a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3e273640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3da72640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c05bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3d271640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc88390 0x55d11c05c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3da72640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c05bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:57592/0 (socket says 172.21.15.92:57592) 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3da72640 1 -- 172.21.15.92:0/2453049323 learned_addr learned my addr 172.21.15.92:0/2453049323 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3d271640 1 -- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 msgr2=0x55d11c05bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3d271640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c05bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3d271640 1 -- 172.21.15.92:0/2453049323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 msgr2=0x55d11c067ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3d271640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3d271640 1 -- 172.21.15.92:0/2453049323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d11bf7ec90 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3da72640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c05bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.005+0000 7f4f3e273640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f3d271640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc88390 0x55d11c05c280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f280e47a0 tx=0x7f4f28010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd543c8c158e723 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f217fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f217fa640 1 -- 172.21.15.92:0/2453049323 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280acbb0 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f30036030 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f217fa640 1 -- 172.21.15.92:0/2453049323 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4f280ac040 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f217fa640 1 -- 172.21.15.92:0/2453049323 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f280b1030 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f4624a740 10 client.25286 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d11c06aa40 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f217fa640 1 -- 172.21.15.92:0/2453049323 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30036030 con 0x55d11bc88390 2024-09-21T02:05:53.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f217fa640 1 -- 172.21.15.92:0/2453049323 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f280ee040 con 0x55d11bc88390 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f4624a740 10 client.25286 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f4624a740 10 client.25286 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.006+0000 7f4f4624a740 20 client.25286 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.007+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc679f0 0x55d11c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.007+0000 7f4f4624a740 4 client.25286 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.007+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d11bfc8df0 con 0x55d11bc679f0 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.007+0000 7f4f3da72640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc679f0 0x55d11c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.007+0000 7f4f217fa640 10 client.25286 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.007+0000 7f4f3da72640 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc679f0 0x55d11c06de60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=164694040e252da8 server_cookie=5d43eef9b8ee6888 in_seq=0 out_seq=0 2024-09-21T02:05:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f217fa640 1 -- 172.21.15.92:0/2453049323 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d11bfc8df0 con 0x55d11bc679f0 2024-09-21T02:05:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f217fa640 10 client.25286 handle_command_reply: tid=0 2024-09-21T02:05:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 client.25286 shutdown 2024-09-21T02:05:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc88390 msgr2=0x55d11c05c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc88390 0x55d11c05c280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f280e47a0 tx=0x7f4f28010f00 comp rx=0 tx=0).stop 2024-09-21T02:05:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 shutdown_connections 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc679f0 0x55d11c06de60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc88390 0x55d11c05c280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2453049323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc801b0 0x55d11c05bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f217fa640 0 client.25286 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.008+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 >> 172.21.15.92:0/2453049323 conn(0x55d11bc9d460 msgr2=0x55d11c053880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.009+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 shutdown_connections 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.009+0000 7f4f4624a740 1 -- 172.21.15.92:0/2453049323 wait complete. 2024-09-21T02:05:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.009+0000 7f4f4624a740 20 client.25286 trim_cache size 0 max 16384 2024-09-21T02:05:53.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 Processor -- start 2024-09-21T02:05:53.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 -- start start 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc801b0 0x55d11c052090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381af4c0 con 0x55d11bca9070 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b1000 con 0x55d11bc801b0 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f4624a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381afb90 con 0x55d11bc88390 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49844/0 (socket says 172.21.15.92:49844) 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 -- 172.21.15.92:0/3814974372 learned_addr learned my addr 172.21.15.92:0/3814974372 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3d271640 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3da72640 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc801b0 0x55d11c052090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 -- 172.21.15.92:0/3814974372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 msgr2=0x55d11c05d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 -- 172.21.15.92:0/3814974372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc801b0 msgr2=0x55d11c052090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc801b0 0x55d11c052090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3e273640 1 -- 172.21.15.92:0/3814974372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d11c052990 con 0x55d11bca9070 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.022+0000 7f4f3d271640 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.023+0000 7f4f3e273640 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ef0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4f300045a0 tx=0x7f4f300045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2e88f7938e5794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.023+0000 7f4f237fe640 1 -- 172.21.15.92:0/3814974372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30060070 con 0x55d11bca9070 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.023+0000 7f4f237fe640 1 -- 172.21.15.92:0/3814974372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30036070 con 0x55d11bca9070 2024-09-21T02:05:53.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.023+0000 7f4f237fe640 1 -- 172.21.15.92:0/3814974372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3002fdf0 con 0x55d11bca9070 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 -- 172.21.15.92:0/3814974372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 msgr2=0x55d11c067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ef0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4f300045a0 tx=0x7f4f300045d0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 -- 172.21.15.92:0/3814974372 shutdown_connections 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bca9070 0x55d11c067ef0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 --2- 172.21.15.92:0/3814974372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bc801b0 0x55d11c052090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 -- 172.21.15.92:0/3814974372 >> 172.21.15.92:0/3814974372 conn(0x55d11bc9d460 msgr2=0x55d11c05d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 -- 172.21.15.92:0/3814974372 shutdown_connections 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 -- 172.21.15.92:0/3814974372 wait complete. 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 -- start start 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.024+0000 7f4f4624a740 1 Processor -- start 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 0x55d11c0647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c060230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b0930 con 0x55d11bc801b0 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b0260 con 0x55d11bca9070 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381ad4a0 con 0x55d11bc88390 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 10 client.0 _ll_register_callbacks cb 0x7f4f280ea440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 0x55d11c0647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 0x55d11c0647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49852/0 (socket says 172.21.15.92:49852) 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 -- 172.21.15.92:0/2693027355 learned_addr learned my addr 172.21.15.92:0/2693027355 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3d271640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3e273640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c060230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 -- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 msgr2=0x55d11c05fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 -- 172.21.15.92:0/2693027355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 msgr2=0x55d11c060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c060230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 -- 172.21.15.92:0/2693027355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d11c0609c0 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f3da72640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 0x55d11c0647c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4f34000c00 tx=0x7f4f3400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf60cc657a966b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f217fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f217fa640 1 -- 172.21.15.92:0/2693027355 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34005220 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f217fa640 1 -- 172.21.15.92:0/2693027355 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4f340053c0 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.025+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f30036070 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f217fa640 1 -- 172.21.15.92:0/2693027355 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f340055c0 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 10 client.15819 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d11c0d8820 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f217fa640 1 -- 172.21.15.92:0/2693027355 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30036070 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f217fa640 1 -- 172.21.15.92:0/2693027355 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f34011740 con 0x55d11bc801b0 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 10 client.15819 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:05:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 20 client.15819 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc3cda0 0x55d11c0db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 4 client.15819 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d11bfc48a0 con 0x55d11bc3cda0 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.026+0000 7f4f3d271640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc3cda0 0x55d11c0db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f3d271640 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc3cda0 0x55d11c0db010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39c0ececeb02820c server_cookie=fc3abea2f88c176 in_seq=0 out_seq=0 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f217fa640 10 client.15819 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f217fa640 1 -- 172.21.15.92:0/2693027355 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d11bfc48a0 con 0x55d11bc3cda0 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f217fa640 10 client.15819 handle_command_reply: tid=0 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 client.15819 shutdown 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 msgr2=0x55d11c0647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 0x55d11c0647c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4f34000c00 tx=0x7f4f3400bf60 comp rx=0 tx=0).stop 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 shutdown_connections 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d11bc3cda0 0x55d11c0db010 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d11bca9070 0x55d11c060230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d11bc88390 0x55d11c05fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 --2- 172.21.15.92:0/2693027355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d11bc801b0 0x55d11c0647c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f217fa640 0 client.15819 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 >> 172.21.15.92:0/2693027355 conn(0x55d11bc9d460 msgr2=0x55d11c05db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 shutdown_connections 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 1 -- 172.21.15.92:0/2693027355 wait complete. 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.027+0000 7f4f4624a740 20 client.15819 trim_cache size 0 max 16384 2024-09-21T02:05:53.033 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f240b6510 msgr2=0x7f4f240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f240b6510 0x7f4f240b8a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4f3004a000 tx=0x7f4f3004b000 comp rx=0 tx=0).stop 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 msgr2=0x7f4f3819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 0x7f4f3819b7d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b450 tx=0x7f4f2800b910 comp rx=0 tx=0).stop 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 shutdown_connections 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f240b6510 0x7f4f240b8a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38108d10 0x7f4f3819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f381083c0 0x7f4f3819b7d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 --2- 172.21.15.92:0/3419076803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f38102a80 0x7f4f3819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:53.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 >> 172.21.15.92:0/3419076803 conn(0x7f4f38079740 msgr2=0x7f4f38105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:53.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 shutdown_connections 2024-09-21T02:05:53.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:53.028+0000 7f4f3fcd6640 1 -- 172.21.15.92:0/3419076803 wait complete. 2024-09-21T02:05:53.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:05:54.008 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:54.414 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:54 smithi092 ceph-mon[31190]: pgmap v388: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 -- 172.21.15.92:0/2206608442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c109df0 msgr2=0x7f596c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 --2- 172.21.15.92:0/2206608442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c109df0 0x7f596c110620 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5964009ca0 tx=0x7f596402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 -- 172.21.15.92:0/2206608442 shutdown_connections 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 --2- 172.21.15.92:0/2206608442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c109df0 0x7f596c110620 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 --2- 172.21.15.92:0/2206608442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596c1094a0 0x7f596c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 --2- 172.21.15.92:0/2206608442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c103b60 0x7f596c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 -- 172.21.15.92:0/2206608442 >> 172.21.15.92:0/2206608442 conn(0x7f596c07b850 msgr2=0x7f596c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 -- 172.21.15.92:0/2206608442 shutdown_connections 2024-09-21T02:05:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.506+0000 7f5970a92640 1 -- 172.21.15.92:0/2206608442 wait complete. 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 Processor -- start 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 -- start start 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 0x7f596c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c1094a0 0x7f596c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596c109df0 0x7f596c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f596c113ab0 con 0x7f596c103b60 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f596c113930 con 0x7f596c1094a0 2024-09-21T02:05:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f5970a92640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f596c113c30 con 0x7f596c109df0 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f596a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 0x7f596c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.507+0000 7f596a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 0x7f596c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41384/0 (socket says 172.21.15.92:41384) 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 -- 172.21.15.92:0/4220535892 learned_addr learned my addr 172.21.15.92:0/4220535892 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f5969d9b640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c1094a0 0x7f596c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 -- 172.21.15.92:0/4220535892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596c109df0 msgr2=0x7f596c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596ad9d640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596c109df0 0x7f596c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596c109df0 0x7f596c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 -- 172.21.15.92:0/4220535892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c1094a0 msgr2=0x7f596c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c1094a0 0x7f596c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 -- 172.21.15.92:0/4220535892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1a03d0 con 0x7f596c103b60 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f5969d9b640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c1094a0 0x7f596c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f596a59c640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 0x7f596c19a460 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f595c00d950 tx=0x7f595c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f595b7fe640 1 -- 172.21.15.92:0/4220535892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c017070 con 0x7f596c103b60 2024-09-21T02:05:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1a06c0 con 0x7f596c103b60 2024-09-21T02:05:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f595b7fe640 1 -- 172.21.15.92:0/4220535892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c009c30 con 0x7f596c103b60 2024-09-21T02:05:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f595b7fe640 1 -- 172.21.15.92:0/4220535892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00f620 con 0x7f596c103b60 2024-09-21T02:05:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.508+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c0755f0 con 0x7f596c103b60 2024-09-21T02:05:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 Processor -- start 2024-09-21T02:05:54.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 -- start start 2024-09-21T02:05:54.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f595b7fe640 1 -- 172.21.15.92:0/4220535892 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f595c010460 con 0x7f596c103b60 2024-09-21T02:05:54.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9632aa070 0x55e963651680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e963651bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e963289390 0x55e963652100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e9635c7ea0 con 0x55e9632811b0 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e9635b72d0 con 0x55e963289390 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.511+0000 7f5977006740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e9635bb0e0 con 0x55e9632aa070 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e963651bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e963651bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41392/0 (socket says 172.21.15.92:41392) 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 -- 172.21.15.92:0/3245825561 learned_addr learned my addr 172.21.15.92:0/3245825561 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f5969d9b640 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e963289390 0x55e963652100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596ad9d640 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9632aa070 0x55e963651680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 -- 172.21.15.92:0/3245825561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9632aa070 msgr2=0x55e963651680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9632aa070 0x55e963651680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 -- 172.21.15.92:0/3245825561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e963289390 msgr2=0x55e963652100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e963289390 0x55e963652100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 -- 172.21.15.92:0/3245825561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e963315650 con 0x55e9632811b0 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596ad9d640 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9632aa070 0x55e963651680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f595b7fe640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f59540b6510 0x7f59540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596a59c640 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e963651bc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f595c0ef2f0 tx=0x7f595c0ef7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea41c2515dd799 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f596ad9d640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f59540b6510 0x7f59540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f595b7fe640 1 -- 172.21.15.92:0/4220535892 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f595c0e7010 con 0x7f596c103b60 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.512+0000 7f5958ff9640 1 -- 172.21.15.92:0/3245825561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0a97d0 con 0x55e9632811b0 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.513+0000 7f5958ff9640 1 -- 172.21.15.92:0/3245825561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c0acc90 con 0x55e9632811b0 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.513+0000 7f5958ff9640 1 -- 172.21.15.92:0/3245825561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0ac8f0 con 0x55e9632811b0 2024-09-21T02:05:54.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.513+0000 7f5977006740 1 -- 172.21.15.92:0/3245825561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 msgr2=0x55e963651bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.513+0000 7f596ad9d640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f59540b6510 0x7f59540b8a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f59640094e0 tx=0x7f59640093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.513+0000 7f5977006740 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e963651bc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f595c0ef2f0 tx=0x7f595c0ef7b0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- 172.21.15.92:0/3245825561 shutdown_connections 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e963289390 0x55e963652100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e963651bc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 --2- 172.21.15.92:0/3245825561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9632aa070 0x55e963651680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- 172.21.15.92:0/3245825561 >> 172.21.15.92:0/3245825561 conn(0x55e96329e460 msgr2=0x55e96364f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- 172.21.15.92:0/3245825561 shutdown_connections 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- 172.21.15.92:0/3245825561 wait complete. 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- start start 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 Processor -- start 2024-09-21T02:05:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e96365ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96365d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632aa070 0x55e96365d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e963418700 con 0x55e9632811b0 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e963552160 con 0x55e9632aa070 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.514+0000 7f5977006740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e96365fe60 con 0x55e963289390 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5977006740 10 client.0 _ll_register_callbacks cb 0x55e96356b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e96365ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e96365ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41398/0 (socket says 172.21.15.92:41398) 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 -- 172.21.15.92:0/2714990111 learned_addr learned my addr 172.21.15.92:0/2714990111 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5969d9b640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96365d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 -- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 msgr2=0x55e96365d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596ad9d640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632aa070 0x55e96365d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96365d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 -- 172.21.15.92:0/2714990111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632aa070 msgr2=0x55e96365d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632aa070 0x55e96365d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 -- 172.21.15.92:0/2714990111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96357fc90 con 0x55e9632811b0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5969d9b640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96365d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596ad9d640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632aa070 0x55e96365d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f596a59c640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e96365ccd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f595c0a9750 tx=0x7f595c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c42d0ad7c1ad1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5951ffb640 1 -- 172.21.15.92:0/2714990111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c020030 con 0x55e9632811b0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5951ffb640 1 -- 172.21.15.92:0/2714990111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f595c0ea140 con 0x55e9632811b0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c0acc90 con 0x55e9632811b0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5951ffb640 1 -- 172.21.15.92:0/2714990111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f595c0acad0 con 0x55e9632811b0 2024-09-21T02:05:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5977006740 10 client.15831 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.515+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e96366bae0 con 0x55e9632811b0 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5951ffb640 1 -- 172.21.15.92:0/2714990111 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c0acc90 con 0x55e9632811b0 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5951ffb640 1 -- 172.21.15.92:0/2714990111 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f595c0a8ad0 con 0x55e9632811b0 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5977006740 10 client.15831 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5977006740 10 client.15831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5977006740 20 client.15831 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5977006740 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e9632689f0 0x55e96366ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5977006740 4 client.15831 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9635c9df0 con 0x55e9632689f0 2024-09-21T02:05:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.516+0000 7f5969d9b640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e9632689f0 0x55e96366ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.517+0000 7f5951ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.517+0000 7f5969d9b640 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e9632689f0 0x55e96366ee40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b993f65c5bf9e93 server_cookie=a1f016952af7795e in_seq=0 out_seq=0 2024-09-21T02:05:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.517+0000 7f5951ffb640 1 -- 172.21.15.92:0/2714990111 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9635c9df0 con 0x55e9632689f0 2024-09-21T02:05:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.517+0000 7f5951ffb640 10 client.15831 handle_command_reply: tid=0 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 client.15831 shutdown 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 msgr2=0x55e96365ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e96365ccd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f595c0a9750 tx=0x7f595c005f70 comp rx=0 tx=0).stop 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 shutdown_connections 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e9632689f0 0x55e96366ee40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632aa070 0x55e96365d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96365d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 --2- 172.21.15.92:0/2714990111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632811b0 0x55e96365ccd0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5951ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 >> 172.21.15.92:0/2714990111 conn(0x55e96329e460 msgr2=0x55e963654810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 shutdown_connections 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 1 -- 172.21.15.92:0/2714990111 wait complete. 2024-09-21T02:05:54.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.518+0000 7f5977006740 20 client.15831 trim_cache size 0 max 16384 2024-09-21T02:05:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 Processor -- start 2024-09-21T02:05:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 -- start start 2024-09-21T02:05:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e96365e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96364ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96364f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b1650 con 0x55e9632aa070 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b3190 con 0x55e9632811b0 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5977006740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b1d20 con 0x55e963289390 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96364f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96364f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41402/0 (socket says 172.21.15.92:41402) 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 -- 172.21.15.92:0/2812039651 learned_addr learned my addr 172.21.15.92:0/2812039651 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f5969d9b640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96364ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596a59c640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e96365e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 -- 172.21.15.92:0/2812039651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 msgr2=0x55e96364ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96364ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 -- 172.21.15.92:0/2812039651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 msgr2=0x55e96365e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e96365e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596ad9d640 1 -- 172.21.15.92:0/2812039651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96365f0a0 con 0x55e9632aa070 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.531+0000 7f596a59c640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e96365e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.532+0000 7f5969d9b640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96364ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.532+0000 7f596ad9d640 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96364f100 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5964031e10 tx=0x7f596403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b0baaf029e38d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.532+0000 7f5953fff640 1 -- 172.21.15.92:0/2812039651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964031e40 con 0x55e9632aa070 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.532+0000 7f5953fff640 1 -- 172.21.15.92:0/2812039651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964036040 con 0x55e9632aa070 2024-09-21T02:05:54.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.532+0000 7f5953fff640 1 -- 172.21.15.92:0/2812039651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964030980 con 0x55e9632aa070 2024-09-21T02:05:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 -- 172.21.15.92:0/2812039651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 msgr2=0x55e96364f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96364f100 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5964031e10 tx=0x7f596403f000 comp rx=0 tx=0).stop 2024-09-21T02:05:54.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 -- 172.21.15.92:0/2812039651 shutdown_connections 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96364f100 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96364ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 --2- 172.21.15.92:0/2812039651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e96365e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 -- 172.21.15.92:0/2812039651 >> 172.21.15.92:0/2812039651 conn(0x55e96329e460 msgr2=0x55e963653070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 -- 172.21.15.92:0/2812039651 shutdown_connections 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 -- 172.21.15.92:0/2812039651 wait complete. 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 -- start start 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.533+0000 7f5977006740 1 Processor -- start 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e963672400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96366f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96366fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b2ac0 con 0x55e9632aa070 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1b23f0 con 0x55e9632811b0 2024-09-21T02:05:54.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1af630 con 0x55e963289390 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96366fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96366fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41412/0 (socket says 172.21.15.92:41412) 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 -- 172.21.15.92:0/541698090 learned_addr learned my addr 172.21.15.92:0/541698090 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 10 client.0 _ll_register_callbacks cb 0x7f5960009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5969d9b640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96366f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596a59c640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e963672400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 -- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 msgr2=0x55e96366f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96366f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 -- 172.21.15.92:0/541698090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 msgr2=0x55e963672400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e963672400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 -- 172.21.15.92:0/541698090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9636703a0 con 0x55e9632aa070 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5969d9b640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96366f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f596ad9d640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96366fe60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5964009ca0 tx=0x7f59640305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf99ed55c71f93f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:54.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5951ffb640 1 -- 172.21.15.92:0/541698090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964031e40 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5951ffb640 1 -- 172.21.15.92:0/541698090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f59640306c0 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.534+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5964036040 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5951ffb640 1 -- 172.21.15.92:0/541698090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59640308c0 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 10 client.15837 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9636d9190 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5951ffb640 1 -- 172.21.15.92:0/541698090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964036040 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5951ffb640 1 -- 172.21.15.92:0/541698090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5964043040 con 0x55e9632aa070 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 10 client.15837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 20 client.15837 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e96323dda0 0x55e9636db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 4 client.15837 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9635c58a0 con 0x55e96323dda0 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.535+0000 7f596a59c640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e96323dda0 0x55e9636db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:54.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f596a59c640 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e96323dda0 0x55e9636db940 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68769b68e3d15ac0 server_cookie=956d648d2c15de39 in_seq=0 out_seq=0 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5951ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5951ffb640 1 -- 172.21.15.92:0/541698090 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9635c58a0 con 0x55e96323dda0 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5951ffb640 10 client.15837 handle_command_reply: tid=0 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 client.15837 shutdown 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 msgr2=0x55e96366fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96366fe60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5964009ca0 tx=0x7f59640305c0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 shutdown_connections 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e96323dda0 0x55e9636db940 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9632aa070 0x55e96366fe60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e963289390 0x55e96366f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 --2- 172.21.15.92:0/541698090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9632811b0 0x55e963672400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5951ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.536+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 >> 172.21.15.92:0/541698090 conn(0x55e96329e460 msgr2=0x55e963668aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 shutdown_connections 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5977006740 1 -- 172.21.15.92:0/541698090 wait complete. 2024-09-21T02:05:54.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5977006740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T02:05:54.542 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:54.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f59540b6510 msgr2=0x7f59540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f59540b6510 0x7f59540b8a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f59640094e0 tx=0x7f59640093f0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 msgr2=0x7f596c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 0x7f596c19a460 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f595c00d950 tx=0x7f595c00de10 comp rx=0 tx=0).stop 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 shutdown_connections 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f59540b6510 0x7f59540b8a20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f596c109df0 0x7f596c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f596c1094a0 0x7f596c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 --2- 172.21.15.92:0/4220535892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c103b60 0x7f596c19a460 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 >> 172.21.15.92:0/4220535892 conn(0x7f596c07b850 msgr2=0x7f596c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 shutdown_connections 2024-09-21T02:05:54.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:54.537+0000 7f5970a92640 1 -- 172.21.15.92:0/4220535892 wait complete. 2024-09-21T02:05:54.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:54 smithi099 ceph-mon[34404]: pgmap v388: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:05:54.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:54 smithi136 ceph-mon[34754]: pgmap v388: 97 pgs: 97 active+clean; 896 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:05:55.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:05:55.452 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 -- 172.21.15.92:0/329968244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca0105320 msgr2=0x7faca010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 --2- 172.21.15.92:0/329968244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca0105320 0x7faca010d870 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fac88009900 tx=0x7fac8802a410 comp rx=0 tx=0).stop 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 -- 172.21.15.92:0/329968244 shutdown_connections 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 --2- 172.21.15.92:0/329968244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca010de30 0x7faca0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 --2- 172.21.15.92:0/329968244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca0105320 0x7faca010d870 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 --2- 172.21.15.92:0/329968244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faca01049d0 0x7faca0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 -- 172.21.15.92:0/329968244 >> 172.21.15.92:0/329968244 conn(0x7faca0079740 msgr2=0x7faca0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 -- 172.21.15.92:0/329968244 shutdown_connections 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.928+0000 7faca5c67640 1 -- 172.21.15.92:0/329968244 wait complete. 2024-09-21T02:05:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 Processor -- start 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 -- start start 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faca01049d0 0x7faca019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca0105320 0x7faca019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 0x7faca019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faca0113a90 con 0x7faca010de30 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faca0113910 con 0x7faca0105320 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7faca5c67640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faca0113c10 con 0x7faca01049d0 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.929+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 0x7faca019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 0x7faca019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41422/0 (socket says 172.21.15.92:41422) 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 -- 172.21.15.92:0/1215785022 learned_addr learned my addr 172.21.15.92:0/1215785022 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faca01049d0 0x7faca019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca0105320 0x7faca019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 -- 172.21.15.92:0/1215785022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faca01049d0 msgr2=0x7faca019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faca01049d0 0x7faca019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 -- 172.21.15.92:0/1215785022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca0105320 msgr2=0x7faca019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca0105320 0x7faca019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 -- 172.21.15.92:0/1215785022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca01a0430 con 0x7faca010de30 2024-09-21T02:05:55.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 0x7faca019fd00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fac9800cad0 tx=0x7fac9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9d7fa640 1 -- 172.21.15.92:0/1215785022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9800ae20 con 0x7faca010de30 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9d7fa640 1 -- 172.21.15.92:0/1215785022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac98005020 con 0x7faca010de30 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca01a0720 con 0x7faca010de30 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7fac9d7fa640 1 -- 172.21.15.92:0/1215785022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9800f6b0 con 0x7faca010de30 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.930+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01a6e80 con 0x7faca010de30 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 Processor -- start 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7fac9d7fa640 1 -- 172.21.15.92:0/1215785022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fac980051d0 con 0x7faca010de30 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 -- start start 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42359070 0x562a427002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a42700800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a42700d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562a42676ea0 con 0x562a423301b0 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a426662d0 con 0x562a42338390 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7facac1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562a4266a0e0 con 0x562a42359070 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a42700800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a42700800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41432/0 (socket says 172.21.15.92:41432) 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7fac9f7fe640 1 -- 172.21.15.92:0/2908706169 learned_addr learned my addr 172.21.15.92:0/2908706169 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.933+0000 7fac96ffd640 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42359070 0x562a427002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9f7fe640 1 -- 172.21.15.92:0/2908706169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42359070 msgr2=0x562a427002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42359070 0x562a427002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9f7fe640 1 -- 172.21.15.92:0/2908706169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 msgr2=0x562a42700d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac96ffd640 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a42700d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a42700d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9f7fe640 1 -- 172.21.15.92:0/2908706169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a423c4650 con 0x562a423301b0 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac96ffd640 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a42700d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9d7fa640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac8c0b6510 0x7fac8c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a42700800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fac9000bcb0 tx=0x7fac900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d76e42f7b2d382 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac8c0b6510 0x7fac8c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac9d7fa640 1 -- 172.21.15.92:0/1215785022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac980e7290 con 0x7faca010de30 2024-09-21T02:05:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac95ffb640 1 -- 172.21.15.92:0/2908706169 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90004020 con 0x562a423301b0 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac95ffb640 1 -- 172.21.15.92:0/2908706169 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac900054d0 con 0x562a423301b0 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.934+0000 7fac95ffb640 1 -- 172.21.15.92:0/2908706169 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90004c80 con 0x562a423301b0 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 -- 172.21.15.92:0/2908706169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 msgr2=0x562a42700800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a42700800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fac9000bcb0 tx=0x7fac900107a0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac8c0b6510 0x7fac8c0b8a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fac88009230 tx=0x7fac88031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 -- 172.21.15.92:0/2908706169 shutdown_connections 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a42700d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a42700800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 --2- 172.21.15.92:0/2908706169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42359070 0x562a427002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 -- 172.21.15.92:0/2908706169 >> 172.21.15.92:0/2908706169 conn(0x562a4234d460 msgr2=0x562a426fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 -- 172.21.15.92:0/2908706169 shutdown_connections 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 -- 172.21.15.92:0/2908706169 wait complete. 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.935+0000 7facac1db740 1 -- start start 2024-09-21T02:05:55.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 Processor -- start 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a4270b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42338390 0x562a4270bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42359070 0x562a42717dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562a424c7700 con 0x562a423301b0 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a42601160 con 0x562a42359070 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562a4270eaa0 con 0x562a42338390 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7facac1db740 10 client.0 _ll_register_callbacks cb 0x562a4261a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a4270b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a4270b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41438/0 (socket says 172.21.15.92:41438) 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7fac9f7fe640 1 -- 172.21.15.92:0/1425598298 learned_addr learned my addr 172.21.15.92:0/1425598298 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42338390 0x562a4270bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.936+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42359070 0x562a42717dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9f7fe640 1 -- 172.21.15.92:0/1425598298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42338390 msgr2=0x562a4270bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42338390 0x562a4270bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9f7fe640 1 -- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42359070 msgr2=0x562a42717dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42359070 0x562a42717dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9f7fe640 1 -- 172.21.15.92:0/1425598298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4262ec90 con 0x562a423301b0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42359070 0x562a42717dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a4270b990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac90000c00 tx=0x7fac9000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d209ffe1d59a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac7effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac7effd640 1 -- 172.21.15.92:0/1425598298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9002b060 con 0x562a423301b0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac7effd640 1 -- 172.21.15.92:0/1425598298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac90004bf0 con 0x562a423301b0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac900054d0 con 0x562a423301b0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac7effd640 1 -- 172.21.15.92:0/1425598298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac90004640 con 0x562a423301b0 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7facac1db740 10 client.15849 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a4271a730 con 0x562a423301b0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac7effd640 1 -- 172.21.15.92:0/1425598298 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac900054d0 con 0x562a423301b0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7fac7effd640 1 -- 172.21.15.92:0/1425598298 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac90011ba0 con 0x562a423301b0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7facac1db740 10 client.15849 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7facac1db740 10 client.15849 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.937+0000 7facac1db740 20 client.15849 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.938+0000 7facac1db740 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a423179f0 0x562a4271db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.938+0000 7facac1db740 4 client.15849 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.938+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a42678df0 con 0x562a423179f0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.938+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a423179f0 0x562a4271db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.938+0000 7fac7effd640 10 client.15849 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.938+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a423179f0 0x562a4271db40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75316ec9abfd24d9 server_cookie=5f7786ce428a0bb7 in_seq=0 out_seq=0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7fac7effd640 1 -- 172.21.15.92:0/1425598298 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a42678df0 con 0x562a423179f0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7fac7effd640 10 client.15849 handle_command_reply: tid=0 2024-09-21T02:05:55.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 client.15849 shutdown 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 msgr2=0x562a4270b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a4270b990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac90000c00 tx=0x7fac9000bfd0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 shutdown_connections 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42359070 0x562a42717dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a42338390 0x562a4270bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a423179f0 0x562a4271db40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 --2- 172.21.15.92:0/1425598298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a423301b0 0x562a4270b990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 >> 172.21.15.92:0/1425598298 conn(0x562a4234d460 msgr2=0x562a42703450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7fac7effd640 0 client.15849 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 shutdown_connections 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.939+0000 7facac1db740 1 -- 172.21.15.92:0/1425598298 wait complete. 2024-09-21T02:05:55.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.940+0000 7facac1db740 20 client.15849 trim_cache size 0 max 16384 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.952+0000 7facac1db740 1 Processor -- start 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.952+0000 7facac1db740 1 -- start start 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7facac1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42717d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7facac1db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4270b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7facac1db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4270bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7facac1db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faca01af540 con 0x562a42359070 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7facac1db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faca01b1080 con 0x562a42338390 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7facac1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faca01afc10 con 0x562a423301b0 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4270bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4270bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41446/0 (socket says 172.21.15.92:41446) 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 -- 172.21.15.92:0/1478213810 learned_addr learned my addr 172.21.15.92:0/1478213810 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 -- 172.21.15.92:0/1478213810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 msgr2=0x562a42717d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42717d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4270b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42717d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 -- 172.21.15.92:0/1478213810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 msgr2=0x562a4270b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4270b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 -- 172.21.15.92:0/1478213810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a42317c20 con 0x562a42359070 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4270b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4270bc80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fac980ad1b0 tx=0x7fac980ad3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98ceb609f312680 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac957fa640 1 -- 172.21.15.92:0/1478213810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9801d040 con 0x562a42359070 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac957fa640 1 -- 172.21.15.92:0/1478213810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac980054f0 con 0x562a42359070 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.953+0000 7fac957fa640 1 -- 172.21.15.92:0/1478213810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980ac3e0 con 0x562a42359070 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 -- 172.21.15.92:0/1478213810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 msgr2=0x562a4270bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4270bc80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fac980ad1b0 tx=0x7fac980ad3b0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 -- 172.21.15.92:0/1478213810 shutdown_connections 2024-09-21T02:05:55.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4270bc80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4270b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 --2- 172.21.15.92:0/1478213810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42717d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 -- 172.21.15.92:0/1478213810 >> 172.21.15.92:0/1478213810 conn(0x562a4234d460 msgr2=0x562a4270a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 -- 172.21.15.92:0/1478213810 shutdown_connections 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 -- 172.21.15.92:0/1478213810 wait complete. 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.954+0000 7facac1db740 1 -- start start 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 Processor -- start 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42721330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4271e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4271ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faca01b09b0 con 0x562a42359070 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faca01b02e0 con 0x562a42338390 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faca01ad520 con 0x562a423301b0 2024-09-21T02:05:55.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7facac1db740 10 client.0 _ll_register_callbacks cb 0x7fac9000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4271ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4271ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41450/0 (socket says 172.21.15.92:41450) 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7fac9ffff640 1 -- 172.21.15.92:0/1860878527 learned_addr learned my addr 172.21.15.92:0/1860878527 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4271e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.955+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42721330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9ffff640 1 -- 172.21.15.92:0/1860878527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 msgr2=0x562a42721330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42721330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9ffff640 1 -- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 msgr2=0x562a4271e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4271e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9ffff640 1 -- 172.21.15.92:0/1860878527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4271f4b0 con 0x562a42359070 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac96ffd640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4271e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:55.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42721330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac9ffff640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4271ed40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fac980b20a0 tx=0x7fac980acb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de06636372b8197e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac7effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac7effd640 1 -- 172.21.15.92:0/1860878527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9801d070 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac7effd640 1 -- 172.21.15.92:0/1860878527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac980ac380 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac980054f0 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac7effd640 1 -- 172.21.15.92:0/1860878527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac980b3070 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7facac1db740 10 client.15855 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a4278c590 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac7effd640 1 -- 172.21.15.92:0/1860878527 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac980054f0 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.956+0000 7fac7effd640 1 -- 172.21.15.92:0/1860878527 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac980c8b70 con 0x562a42359070 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 10 client.15855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 20 client.15855 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a422ecda0 0x562a4278ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 4 client.15855 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:55.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a426748a0 con 0x562a422ecda0 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a422ecda0 0x562a4278ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7fac7effd640 10 client.15855 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7fac9f7fe640 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a422ecda0 0x562a4278ec80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c61f2be4b87bf943 server_cookie=e331a66b9486d8c1 in_seq=0 out_seq=0 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7fac7effd640 1 -- 172.21.15.92:0/1860878527 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a426748a0 con 0x562a422ecda0 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7fac7effd640 10 client.15855 handle_command_reply: tid=0 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.957+0000 7facac1db740 1 client.15855 shutdown 2024-09-21T02:05:55.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 msgr2=0x562a4271ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4271ed40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fac980b20a0 tx=0x7fac980acb00 comp rx=0 tx=0).stop 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 shutdown_connections 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x562a422ecda0 0x562a4278ec80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a42359070 0x562a4271ed40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a42338390 0x562a4271e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7fac7effd640 0 client.15855 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 --2- 172.21.15.92:0/1860878527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a423301b0 0x562a42721330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 >> 172.21.15.92:0/1860878527 conn(0x562a4234d460 msgr2=0x562a4270b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 shutdown_connections 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 1 -- 172.21.15.92:0/1860878527 wait complete. 2024-09-21T02:05:55.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7facac1db740 20 client.15855 trim_cache size 0 max 16384 2024-09-21T02:05:55.979 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:55.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac8c0b6510 msgr2=0x7fac8c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7faca5c67640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac8c0b6510 0x7fac8c0b8a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fac88009230 tx=0x7fac88031400 comp rx=0 tx=0).stop 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 msgr2=0x7faca019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.958+0000 7faca5c67640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 0x7faca019fd00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fac9800cad0 tx=0x7fac9800cf90 comp rx=0 tx=0).stop 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 shutdown_connections 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac8c0b6510 0x7fac8c0b8a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faca010de30 0x7faca019fd00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca0105320 0x7faca019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 --2- 172.21.15.92:0/1215785022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faca01049d0 0x7faca019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 >> 172.21.15.92:0/1215785022 conn(0x7faca0079740 msgr2=0x7faca0100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 shutdown_connections 2024-09-21T02:05:55.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:55.959+0000 7faca5c67640 1 -- 172.21.15.92:0/1215785022 wait complete. 2024-09-21T02:05:56.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:56 smithi092 ceph-mon[31190]: pgmap v389: 97 pgs: 97 active+clean; 954 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:05:56.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:56 smithi099 ceph-mon[34404]: pgmap v389: 97 pgs: 97 active+clean; 954 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:05:56.584 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:56 smithi136 ceph-mon[34754]: pgmap v389: 97 pgs: 97 active+clean; 954 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:05:56.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:05:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:05:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 -- 172.21.15.92:0/1268580122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 msgr2=0x7f6ea81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 --2- 172.21.15.92:0/1268580122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea81087d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009900 tx=0x7f6e9c02a410 comp rx=0 tx=0).stop 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 -- 172.21.15.92:0/1268580122 shutdown_connections 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 --2- 172.21.15.92:0/1268580122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ea8108d10 0x7f6ea810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 --2- 172.21.15.92:0/1268580122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea81087d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 --2- 172.21.15.92:0/1268580122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8102a80 0x7f6ea8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 -- 172.21.15.92:0/1268580122 >> 172.21.15.92:0/1268580122 conn(0x7f6ea8079740 msgr2=0x7f6ea8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 -- 172.21.15.92:0/1268580122 shutdown_connections 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.371+0000 7f6ead821640 1 -- 172.21.15.92:0/1268580122 wait complete. 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 Processor -- start 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 -- start start 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ea8102a80 0x7f6ea81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8108d10 0x7f6ea819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8112980 con 0x7f6ea81083c0 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8112800 con 0x7f6ea8108d10 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.372+0000 7f6ead821640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8112b00 con 0x7f6ea8102a80 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea8199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41462/0 (socket says 172.21.15.92:41462) 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 -- 172.21.15.92:0/553989234 learned_addr learned my addr 172.21.15.92:0/553989234 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ea8102a80 0x7f6ea81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8108d10 0x7f6ea819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 -- 172.21.15.92:0/553989234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ea8102a80 msgr2=0x7f6ea81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ea8102a80 0x7f6ea81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 -- 172.21.15.92:0/553989234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8108d10 msgr2=0x7f6ea819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8108d10 0x7f6ea819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 -- 172.21.15.92:0/553989234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea819f360 con 0x7f6ea81083c0 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8108d10 0x7f6ea819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea8199990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c032fd0 tx=0x7f6e9c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6eac81f640 1 -- 172.21.15.92:0/553989234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c03f030 con 0x7f6ea81083c0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6eac81f640 1 -- 172.21.15.92:0/553989234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9c004490 con 0x7f6ea81083c0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea819f5f0 con 0x7f6ea81083c0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6eac81f640 1 -- 172.21.15.92:0/553989234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c035070 con 0x7f6ea81083c0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.373+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a5db0 con 0x7f6ea81083c0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 Processor -- start 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 -- start start 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d6565c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6565c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6565cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557d655d2ea0 con 0x557d652b5070 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557d655c22d0 con 0x557d65294390 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557d655c60e0 con 0x557d6528c1b0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6eac81f640 1 -- 172.21.15.92:0/553989234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f6e9c042030 con 0x7f6ea81083c0 2024-09-21T02:05:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6ea77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d6565c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6ea77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d6565c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41464/0 (socket says 172.21.15.92:41464) 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6ea77fe640 1 -- 172.21.15.92:0/892345375 learned_addr learned my addr 172.21.15.92:0/892345375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.376+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6565c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea77fe640 1 -- 172.21.15.92:0/892345375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 msgr2=0x557d6565c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6565c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea77fe640 1 -- 172.21.15.92:0/892345375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 msgr2=0x557d6565cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6565cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea77fe640 1 -- 172.21.15.92:0/892345375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d65320650 con 0x557d652b5070 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d6565c2c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009f30 tx=0x7f6ea0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c445ee4e8535e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6eac81f640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6e900b6590 0x7f6e900b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6e8dffb640 1 -- 172.21.15.92:0/892345375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0016070 con 0x557d652b5070 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6e900b6590 0x7f6e900b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6eac81f640 1 -- 172.21.15.92:0/553989234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6e9c107780 con 0x7f6ea81083c0 2024-09-21T02:05:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6e8dffb640 1 -- 172.21.15.92:0/892345375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea001bb10 con 0x557d652b5070 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.377+0000 7f6e8dffb640 1 -- 172.21.15.92:0/892345375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0012c30 con 0x557d652b5070 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 -- 172.21.15.92:0/892345375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 msgr2=0x557d6565c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6e900b6590 0x7f6e900b8aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x557d6565af40 tx=0x7f6e9800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d6565c2c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009f30 tx=0x7f6ea0010800 comp rx=0 tx=0).stop 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 -- 172.21.15.92:0/892345375 shutdown_connections 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6565cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6565c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/892345375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d6565c2c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 -- 172.21.15.92:0/892345375 >> 172.21.15.92:0/892345375 conn(0x557d652a9460 msgr2=0x557d6565a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 -- 172.21.15.92:0/892345375 shutdown_connections 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 -- 172.21.15.92:0/892345375 wait complete. 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.378+0000 7f6eb3d95740 1 -- start start 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 Processor -- start 2024-09-21T02:05:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d65669ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6566a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d65675f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557d65423700 con 0x557d652b5070 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557d6555d160 con 0x557d65294390 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557d6566cc00 con 0x557d6528c1b0 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6eb3d95740 10 client.0 _ll_register_callbacks cb 0x557d655769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d65675f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d65675f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41472/0 (socket says 172.21.15.92:41472) 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea77fe640 1 -- 172.21.15.92:0/2308356181 learned_addr learned my addr 172.21.15.92:0/2308356181 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6566a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d65669ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea77fe640 1 -- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 msgr2=0x557d65669ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d65669ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.379+0000 7f6ea77fe640 1 -- 172.21.15.92:0/2308356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 msgr2=0x557d6566a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6566a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6ea77fe640 1 -- 172.21.15.92:0/2308356181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6558ac90 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d65669ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d65675f60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009f00 tx=0x7f6ea0004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d404fc99d6ec11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6e86ffd640 1 -- 172.21.15.92:0/2308356181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0029060 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6e86ffd640 1 -- 172.21.15.92:0/2308356181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6ea00129b0 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea001bb10 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6e86ffd640 1 -- 172.21.15.92:0/2308356181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ea0016070 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6eb3d95740 10 client.15867 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d65678960 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.380+0000 7f6e86ffd640 1 -- 172.21.15.92:0/2308356181 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea001bb10 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6e86ffd640 1 -- 172.21.15.92:0/2308356181 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ea000f040 con 0x557d652b5070 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6eb3d95740 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6eb3d95740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:57.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6eb3d95740 20 client.15867 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d652739f0 0x557d6567bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6eb3d95740 4 client.15867 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d655d4df0 con 0x557d652739f0 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.381+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d652739f0 0x557d6567bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.382+0000 7f6e86ffd640 10 client.15867 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.382+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d652739f0 0x557d6567bd70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a96a84ec7c4438 server_cookie=8878ae77342baaed in_seq=0 out_seq=0 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.382+0000 7f6e86ffd640 1 -- 172.21.15.92:0/2308356181 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d655d4df0 con 0x557d652739f0 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.382+0000 7f6e86ffd640 10 client.15867 handle_command_reply: tid=0 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.382+0000 7f6eb3d95740 1 client.15867 shutdown 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 msgr2=0x557d65675f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d65675f60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009f00 tx=0x7f6ea0004a60 comp rx=0 tx=0).stop 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 shutdown_connections 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d652739f0 0x557d6567bd70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d652b5070 0x557d65675f60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d6566a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/2308356181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d65669ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6e86ffd640 0 client.15867 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 >> 172.21.15.92:0/2308356181 conn(0x557d652a9460 msgr2=0x557d65658750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 shutdown_connections 2024-09-21T02:05:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 1 -- 172.21.15.92:0/2308356181 wait complete. 2024-09-21T02:05:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.383+0000 7f6eb3d95740 20 client.15867 trim_cache size 0 max 16384 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 Processor -- start 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 -- start start 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 0x557d65661ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d65666d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d652b5070 0x557d656672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ad260 con 0x557d6528c1b0 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81aeda0 con 0x557d65294390 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ad930 con 0x557d652b5070 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 0x557d65661ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 0x557d65661ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41480/0 (socket says 172.21.15.92:41480) 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6ea6ffd640 1 -- 172.21.15.92:0/3856271354 learned_addr learned my addr 172.21.15.92:0/3856271354 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d652b5070 0x557d656672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.396+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d65666d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea6ffd640 1 -- 172.21.15.92:0/3856271354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d652b5070 msgr2=0x557d656672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d652b5070 0x557d656672d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea6ffd640 1 -- 172.21.15.92:0/3856271354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 msgr2=0x557d65666d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d65666d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea6ffd640 1 -- 172.21.15.92:0/3856271354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d656555d0 con 0x557d6528c1b0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d652b5070 0x557d656672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d65666d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 0x557d65661ac0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x557d65423700 tx=0x7f6e98015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99e799ca00dfe82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6e8d7fa640 1 -- 172.21.15.92:0/3856271354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9801e0d0 con 0x557d6528c1b0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6e8d7fa640 1 -- 172.21.15.92:0/3856271354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9801e270 con 0x557d6528c1b0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.397+0000 7f6e8d7fa640 1 -- 172.21.15.92:0/3856271354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e980035f0 con 0x557d6528c1b0 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 -- 172.21.15.92:0/3856271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 msgr2=0x557d65661ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 0x557d65661ac0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x557d65423700 tx=0x7f6e98015000 comp rx=0 tx=0).stop 2024-09-21T02:05:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 -- 172.21.15.92:0/3856271354 shutdown_connections 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d652b5070 0x557d656672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d65294390 0x557d65666d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/3856271354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d6528c1b0 0x557d65661ac0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 -- 172.21.15.92:0/3856271354 >> 172.21.15.92:0/3856271354 conn(0x557d652a9460 msgr2=0x557d656613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 -- 172.21.15.92:0/3856271354 shutdown_connections 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 -- 172.21.15.92:0/3856271354 wait complete. 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.398+0000 7f6eb3d95740 1 -- start start 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 Processor -- start 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6567f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 0x557d6567c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d652b5070 0x557d6567cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae6d0 con 0x557d65294390 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae000 con 0x557d652b5070 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ab240 con 0x557d6528c1b0 2024-09-21T02:05:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6eb3d95740 10 client.0 _ll_register_callbacks cb 0x7f6ea0009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 0x557d6567c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 0x557d6567c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41482/0 (socket says 172.21.15.92:41482) 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 -- 172.21.15.92:0/1382616702 learned_addr learned my addr 172.21.15.92:0/1382616702 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6567f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d652b5070 0x557d6567cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 -- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 msgr2=0x557d6567f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6567f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 -- 172.21.15.92:0/1382616702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d652b5070 msgr2=0x557d6567cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d652b5070 0x557d6567cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea67fc640 1 -- 172.21.15.92:0/1382616702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6567d2c0 con 0x557d65294390 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.399+0000 7f6ea77fe640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d652b5070 0x557d6567cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6ea67fc640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 0x557d6567c840 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0c9ec0 tx=0x7f6e9c0cd780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baada632d240f06b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6e86ffd640 1 -- 172.21.15.92:0/1382616702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0d2020 con 0x557d65294390 2024-09-21T02:05:57.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6e86ffd640 1 -- 172.21.15.92:0/1382616702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6e9c0d6e80 con 0x557d65294390 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9801e270 con 0x557d65294390 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6e86ffd640 1 -- 172.21.15.92:0/1382616702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e9c049040 con 0x557d65294390 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d656e61c0 con 0x557d65294390 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6e86ffd640 1 -- 172.21.15.92:0/1382616702 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9801e270 con 0x557d65294390 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6e86ffd640 1 -- 172.21.15.92:0/1382616702 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6e9c0c9240 con 0x557d65294390 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 20 client.15873 populate_metadata read hostname 'smithi092' 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.400+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d65248da0 0x557d656e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6eb3d95740 4 client.15873 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d655d08a0 con 0x557d65248da0 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d65248da0 0x557d656e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6e86ffd640 10 client.15873 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:57.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6ea6ffd640 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d65248da0 0x557d656e89d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df0d120e2a2c417 server_cookie=94360b817f84fb81 in_seq=0 out_seq=0 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6e86ffd640 1 -- 172.21.15.92:0/1382616702 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d655d08a0 con 0x557d65248da0 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6e86ffd640 10 client.15873 handle_command_reply: tid=0 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.401+0000 7f6eb3d95740 1 client.15873 shutdown 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 msgr2=0x557d6567c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 0x557d6567c840 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0c9ec0 tx=0x7f6e9c0cd780 comp rx=0 tx=0).stop 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 shutdown_connections 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557d65248da0 0x557d656e89d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d652b5070 0x557d6567cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d65294390 0x557d6567c840 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6e86ffd640 0 client.15873 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 --2- 172.21.15.92:0/1382616702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557d6528c1b0 0x557d6567f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 >> 172.21.15.92:0/1382616702 conn(0x557d652a9460 msgr2=0x557d65668020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 shutdown_connections 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 1 -- 172.21.15.92:0/1382616702 wait complete. 2024-09-21T02:05:57.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6eb3d95740 20 client.15873 trim_cache size 0 max 16384 2024-09-21T02:05:57.409 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:05:57.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6e900b6590 msgr2=0x7f6e900b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6e900b6590 0x7f6e900b8aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x557d6565af40 tx=0x7f6e9800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.409 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 msgr2=0x7f6ea8199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea8199990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c032fd0 tx=0x7f6e9c004270 comp rx=0 tx=0).stop 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 shutdown_connections 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6e900b6590 0x7f6e900b8aa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ea8108d10 0x7f6ea819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ea81083c0 0x7f6ea8199990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.402+0000 7f6ead821640 1 --2- 172.21.15.92:0/553989234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ea8102a80 0x7f6ea81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.403+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 >> 172.21.15.92:0/553989234 conn(0x7f6ea8079740 msgr2=0x7f6ea81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.403+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 shutdown_connections 2024-09-21T02:05:57.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:05:57.403+0000 7f6ead821640 1 -- 172.21.15.92:0/553989234 wait complete. 2024-09-21T02:05:58.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:05:58 smithi092 ceph-mon[31190]: pgmap v390: 97 pgs: 97 active+clean; 954 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:05:58.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:05:58 smithi099 ceph-mon[34404]: pgmap v390: 97 pgs: 97 active+clean; 954 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:05:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:05:58 smithi136 ceph-mon[34754]: pgmap v390: 97 pgs: 97 active+clean; 954 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:05:59.467 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:05:59.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:05:59.735 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/2020958989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 msgr2=0x7f8f24102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/2020958989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f24102ea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009900 tx=0x7f8f1402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/2020958989 shutdown_connections 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/2020958989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f24108d20 0x7f8f2410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/2020958989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f241083d0 0x7f8f241087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/2020958989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f24102ea0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/2020958989 >> 172.21.15.92:0/2020958989 conn(0x7f8f24079750 msgr2=0x7f8f24079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/2020958989 shutdown_connections 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/2020958989 wait complete. 2024-09-21T02:06:00.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.232+0000 7f8f2bdc5640 1 Processor -- start 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 -- start start 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f2419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f241083d0 0x7f8f2419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f24108d20 0x7f8f2419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24112990 con 0x7f8f24102a90 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24112810 con 0x7f8f241083d0 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2bdc5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24112b10 con 0x7f8f24108d20 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f2419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f2419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41508/0 (socket says 172.21.15.92:41508) 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 -- 172.21.15.92:0/1534667130 learned_addr learned my addr 172.21.15.92:0/1534667130 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29360640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f241083d0 0x7f8f2419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f2a362640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f24108d20 0x7f8f2419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 -- 172.21.15.92:0/1534667130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f24108d20 msgr2=0x7f8f2419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f24108d20 0x7f8f2419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 -- 172.21.15.92:0/1534667130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f241083d0 msgr2=0x7f8f2419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f241083d0 0x7f8f2419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 -- 172.21.15.92:0/1534667130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f241a64e0 con 0x7f8f24102a90 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.233+0000 7f8f29b61640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f2419b2e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f14005940 tx=0x7f8f140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.234+0000 7f8f22ffd640 1 -- 172.21.15.92:0/1534667130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1403f030 con 0x7f8f24102a90 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.234+0000 7f8f22ffd640 1 -- 172.21.15.92:0/1534667130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f14003cb0 con 0x7f8f24102a90 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.234+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241a6770 con 0x7f8f24102a90 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.234+0000 7f8f22ffd640 1 -- 172.21.15.92:0/1534667130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14035070 con 0x7f8f24102a90 2024-09-21T02:06:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.234+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a6c50 con 0x7f8f24102a90 2024-09-21T02:06:00.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 Processor -- start 2024-09-21T02:06:00.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 -- start start 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d3713430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3713970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d3713eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e7d3689e60 con 0x55e7d3343490 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e7d3679290 con 0x55e7d334b670 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f32339740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e7d367d0a0 con 0x55e7d336c350 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.237+0000 7f8f22ffd640 1 -- 172.21.15.92:0/1534667130 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8f14042030 con 0x7f8f24102a90 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3713970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3713970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41516/0 (socket says 172.21.15.92:41516) 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 -- 172.21.15.92:0/2446826786 learned_addr learned my addr 172.21.15.92:0/2446826786 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f2a362640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d3713430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29360640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d3713eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 -- 172.21.15.92:0/2446826786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 msgr2=0x55e7d3713430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d3713430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 -- 172.21.15.92:0/2446826786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 msgr2=0x55e7d3713eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d3713eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 -- 172.21.15.92:0/2446826786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7d33d7930 con 0x55e7d3343490 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f2a362640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d3713430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29360640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d3713eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:00.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f29b61640 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3713970 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f1410dc10 tx=0x7f8f1410dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1983cfa3d3999dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f22ffd640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8f0c0b6540 0x7f8f0c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.238+0000 7f8f0bfff640 1 -- 172.21.15.92:0/2446826786 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14116070 con 0x55e7d3343490 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.239+0000 7f8f2a362640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8f0c0b6540 0x7f8f0c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.239+0000 7f8f0bfff640 1 -- 172.21.15.92:0/2446826786 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f140cd4c0 con 0x55e7d3343490 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.239+0000 7f8f0bfff640 1 -- 172.21.15.92:0/2446826786 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f140cd120 con 0x55e7d3343490 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.239+0000 7f8f22ffd640 1 -- 172.21.15.92:0/1534667130 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8f14107840 con 0x7f8f24102a90 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 -- 172.21.15.92:0/2446826786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 msgr2=0x55e7d3713970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3713970 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f1410dc10 tx=0x7f8f1410dd00 comp rx=0 tx=0).stop 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f2a362640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8f0c0b6540 0x7f8f0c0b8a50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c011550 tx=0x7f8f1c007680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 -- 172.21.15.92:0/2446826786 shutdown_connections 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d3713eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3713970 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 --2- 172.21.15.92:0/2446826786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d3713430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 -- 172.21.15.92:0/2446826786 >> 172.21.15.92:0/2446826786 conn(0x55e7d3360740 msgr2=0x55e7d3711320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 -- 172.21.15.92:0/2446826786 shutdown_connections 2024-09-21T02:06:00.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 -- 172.21.15.92:0/2446826786 wait complete. 2024-09-21T02:06:00.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 -- start start 2024-09-21T02:06:00.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.240+0000 7f8f32339740 1 Processor -- start 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d371efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d371f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d372ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e7d34da890 con 0x55e7d334b670 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e7d3613d40 con 0x55e7d3343490 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e7d37219d0 con 0x55e7d336c350 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f32339740 10 client.0 _ll_register_callbacks cb 0x55e7d3677c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:00.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d371f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d371f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41526/0 (socket says 172.21.15.92:41526) 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 -- 172.21.15.92:0/3558139431 learned_addr learned my addr 172.21.15.92:0/3558139431 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29b61640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d371efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f2a362640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d372ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 -- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 msgr2=0x55e7d372ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d372ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 -- 172.21.15.92:0/3558139431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 msgr2=0x55e7d371efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d371efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 -- 172.21.15.92:0/3558139431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7d372b170 con 0x55e7d334b670 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29b61640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d371efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f2a362640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d372ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.241+0000 7f8f29360640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d371f4f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800bad0 tx=0x7f8f1800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962a0c5aec7c88e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f097fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f097fa640 1 -- 172.21.15.92:0/3558139431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011440 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f097fa640 1 -- 172.21.15.92:0/3558139431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8f180115e0 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f140cd4c0 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f097fa640 1 -- 172.21.15.92:0/3558139431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f180117e0 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7d372dae0 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f097fa640 1 -- 172.21.15.92:0/3558139431 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f140cd4c0 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f097fa640 1 -- 172.21.15.92:0/3558139431 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f180259a0 con 0x55e7d334b670 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 20 client.15885 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d332acd0 0x55e7d3730fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 4 client.15885 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.242+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7d368bdb0 con 0x55e7d332acd0 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.243+0000 7f8f29b61640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d332acd0 0x55e7d3730fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.243+0000 7f8f097fa640 10 client.15885 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.243+0000 7f8f29b61640 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d332acd0 0x55e7d3730fd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249490a598f72929 server_cookie=3d9d55c8526127f5 in_seq=0 out_seq=0 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f097fa640 1 -- 172.21.15.92:0/3558139431 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e7d368bdb0 con 0x55e7d332acd0 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f097fa640 10 client.15885 handle_command_reply: tid=0 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 client.15885 shutdown 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 msgr2=0x55e7d371f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d371f4f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800bad0 tx=0x7f8f1800bf90 comp rx=0 tx=0).stop 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 shutdown_connections 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d332acd0 0x55e7d3730fd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d336c350 0x55e7d372ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d371f4f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f097fa640 0 client.15885 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 --2- 172.21.15.92:0/3558139431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d371efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 >> 172.21.15.92:0/3558139431 conn(0x55e7d3360740 msgr2=0x55e7d37165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 shutdown_connections 2024-09-21T02:06:00.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 1 -- 172.21.15.92:0/3558139431 wait complete. 2024-09-21T02:06:00.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.244+0000 7f8f32339740 20 client.15885 trim_cache size 0 max 16384 2024-09-21T02:06:00.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 Processor -- start 2024-09-21T02:06:00.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 -- start start 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d371a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d372ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d3714e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241af400 con 0x55e7d334b670 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b0f40 con 0x55e7d3343490 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f32339740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241afad0 con 0x55e7d332acd0 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d3714e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d3714e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41532/0 (socket says 172.21.15.92:41532) 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 -- 172.21.15.92:0/4002799256 learned_addr learned my addr 172.21.15.92:0/4002799256 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f29360640 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d372ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f29b61640 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d371a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 -- 172.21.15.92:0/4002799256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 msgr2=0x55e7d371a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d371a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 -- 172.21.15.92:0/4002799256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 msgr2=0x55e7d372ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d372ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.262+0000 7f8f2a362640 1 -- 172.21.15.92:0/4002799256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7d3715720 con 0x55e7d334b670 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.263+0000 7f8f2a362640 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d3714e20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00f6c0 tx=0x7f8f1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9067ec8f6c1a2fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.263+0000 7f8f0b7fe640 1 -- 172.21.15.92:0/4002799256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c02a070 con 0x55e7d334b670 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.263+0000 7f8f0b7fe640 1 -- 172.21.15.92:0/4002799256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0253f0 con 0x55e7d334b670 2024-09-21T02:06:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.263+0000 7f8f0b7fe640 1 -- 172.21.15.92:0/4002799256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c01b680 con 0x55e7d334b670 2024-09-21T02:06:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 -- 172.21.15.92:0/4002799256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 msgr2=0x55e7d3714e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d3714e20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00f6c0 tx=0x7f8f1c014000 comp rx=0 tx=0).stop 2024-09-21T02:06:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 -- 172.21.15.92:0/4002799256 shutdown_connections 2024-09-21T02:06:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d334b670 0x55e7d3714e20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d3343490 0x55e7d372ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 --2- 172.21.15.92:0/4002799256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d371a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 -- 172.21.15.92:0/4002799256 >> 172.21.15.92:0/4002799256 conn(0x55e7d3360740 msgr2=0x55e7d37202d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 -- 172.21.15.92:0/4002799256 shutdown_connections 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 -- 172.21.15.92:0/4002799256 wait complete. 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 -- start start 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.264+0000 7f8f32339740 1 Processor -- start 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d3722d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3723290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d37237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b0870 con 0x55e7d3343490 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241b01a0 con 0x55e7d334b670 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241ad3e0 con 0x55e7d332acd0 2024-09-21T02:06:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f32339740 10 client.0 _ll_register_callbacks cb 0x7f8f18004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3723290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3723290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41546/0 (socket says 172.21.15.92:41546) 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 -- 172.21.15.92:0/526355390 learned_addr learned my addr 172.21.15.92:0/526355390 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f2a362640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d37237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29b61640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d3722d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 -- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 msgr2=0x55e7d3722d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d3722d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 -- 172.21.15.92:0/526355390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 msgr2=0x55e7d37237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d37237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.265+0000 7f8f29360640 1 -- 172.21.15.92:0/526355390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7d379ab40 con 0x55e7d3343490 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f29360640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3723290 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800bad0 tx=0x7f8f180043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da10b851973158c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f097fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f097fa640 1 -- 172.21.15.92:0/526355390 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011b90 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f097fa640 1 -- 172.21.15.92:0/526355390 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8f18011d30 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c0253f0 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f097fa640 1 -- 172.21.15.92:0/526355390 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f18034070 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 10 client.15891 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7d379bde0 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f097fa640 1 -- 172.21.15.92:0/526355390 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0253f0 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f097fa640 1 -- 172.21.15.92:0/526355390 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f1802d030 con 0x55e7d3343490 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 10 client.15891 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 20 client.15891 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d336c350 0x55e7d379e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 4 client.15891 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:06:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.266+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e7d3687860 con 0x55e7d336c350 2024-09-21T02:06:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.267+0000 7f8f29b61640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d336c350 0x55e7d379e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.267+0000 7f8f29b61640 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d336c350 0x55e7d379e5d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c7e565762e9b34 server_cookie=d49d9ef01de82bf4 in_seq=0 out_seq=0 2024-09-21T02:06:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.267+0000 7f8f097fa640 10 client.15891 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.269+0000 7f8f097fa640 1 -- 172.21.15.92:0/526355390 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e7d3687860 con 0x55e7d336c350 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.269+0000 7f8f097fa640 10 client.15891 handle_command_reply: tid=0 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.269+0000 7f8f32339740 1 client.15891 shutdown 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.269+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 msgr2=0x55e7d3723290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.269+0000 7f8f32339740 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3723290 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800bad0 tx=0x7f8f180043b0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 shutdown_connections 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e7d336c350 0x55e7d379e5d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7d334b670 0x55e7d37237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7d3343490 0x55e7d3723290 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 --2- 172.21.15.92:0/526355390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7d332acd0 0x55e7d3722d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f097fa640 0 client.15891 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 >> 172.21.15.92:0/526355390 conn(0x55e7d3360740 msgr2=0x55e7d3720bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 shutdown_connections 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 1 -- 172.21.15.92:0/526355390 wait complete. 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f32339740 20 client.15891 trim_cache size 0 max 16384 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "6efa3f01-9ec1-473a-908e-a63bfd3275d7", 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:06:00.274 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.270+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8f0c0b6540 msgr2=0x7f8f0c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8f0c0b6540 0x7f8f0c0b8a50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c011550 tx=0x7f8f1c007680 comp rx=0 tx=0).stop 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 msgr2=0x7f8f2419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f2419b2e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f14005940 tx=0x7f8f140042f0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 shutdown_connections 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8f0c0b6540 0x7f8f0c0b8a50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f24108d20 0x7f8f2419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f241083d0 0x7f8f2419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 --2- 172.21.15.92:0/1534667130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f24102a90 0x7f8f2419b2e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 >> 172.21.15.92:0/1534667130 conn(0x7f8f24079750 msgr2=0x7f8f24101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 shutdown_connections 2024-09-21T02:06:00.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:00.271+0000 7f8f2bdc5640 1 -- 172.21.15.92:0/1534667130 wait complete. 2024-09-21T02:06:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:00 smithi092 ceph-mon[31190]: pgmap v391: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:00.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:00 smithi099 ceph-mon[34404]: pgmap v391: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:00 smithi136 ceph-mon[34754]: pgmap v391: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 -- 172.21.15.92:0/612165805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 msgr2=0x7f391c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 --2- 172.21.15.92:0/612165805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c103a30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3910009900 tx=0x7f391002a410 comp rx=0 tx=0).stop 2024-09-21T02:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 -- 172.21.15.92:0/612165805 shutdown_connections 2024-09-21T02:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 --2- 172.21.15.92:0/612165805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 0x7f391c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 --2- 172.21.15.92:0/612165805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c103a30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 --2- 172.21.15.92:0/612165805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391c1025a0 0x7f391c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 -- 172.21.15.92:0/612165805 >> 172.21.15.92:0/612165805 conn(0x7f391c0fdee0 msgr2=0x7f391c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 -- 172.21.15.92:0/612165805 shutdown_connections 2024-09-21T02:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.832+0000 7f392167e640 1 -- 172.21.15.92:0/612165805 wait complete. 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 Processor -- start 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 -- start start 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391c1025a0 0x7f391c06b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 0x7f391c1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10e200 con 0x7f391c103620 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10e080 con 0x7f391c1025a0 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f392167e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10e380 con 0x7f391c103fc0 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41566/0 (socket says 172.21.15.92:41566) 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f391a7fc640 1 -- 172.21.15.92:0/2221721366 learned_addr learned my addr 172.21.15.92:0/2221721366 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f391affd640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391c1025a0 0x7f391c06b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.833+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 0x7f391c1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391a7fc640 1 -- 172.21.15.92:0/2221721366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 msgr2=0x7f391c1a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 0x7f391c1a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391a7fc640 1 -- 172.21.15.92:0/2221721366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391c1025a0 msgr2=0x7f391c06b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391c1025a0 0x7f391c06b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391a7fc640 1 -- 172.21.15.92:0/2221721366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c1a1a20 con 0x7f391c103620 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 0x7f391c1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c19ecd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3910006010 tx=0x7f3910004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f3903fff640 1 -- 172.21.15.92:0/2221721366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391003f030 con 0x7f391c103620 2024-09-21T02:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f3903fff640 1 -- 172.21.15.92:0/2221721366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3910003cb0 con 0x7f391c103620 2024-09-21T02:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f3903fff640 1 -- 172.21.15.92:0/2221721366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910035070 con 0x7f391c103620 2024-09-21T02:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c1a1cb0 con 0x7f391c103620 2024-09-21T02:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.834+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a21a0 con 0x7f391c103620 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 Processor -- start 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f3903fff640 1 -- 172.21.15.92:0/2221721366 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f3910042030 con 0x7f391c103620 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 -- start start 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570acdb2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9e22f0 0x5570acdb34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdb4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5570acd256f0 con 0x5570ac9ea4d0 2024-09-21T02:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5570acd14b20 con 0x5570aca0b1b0 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f39285e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5570acd18930 con 0x5570ac9e22f0 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdb4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdb4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41570/0 (socket says 172.21.15.92:41570) 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 -- 172.21.15.92:0/4002142875 learned_addr learned my addr 172.21.15.92:0/4002142875 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391affd640 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9e22f0 0x5570acdb34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391b7fe640 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570acdb2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 -- 172.21.15.92:0/4002142875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9e22f0 msgr2=0x5570acdb34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9e22f0 0x5570acdb34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 -- 172.21.15.92:0/4002142875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 msgr2=0x5570acdb2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570acdb2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f391a7fc640 1 -- 172.21.15.92:0/4002142875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570accda530 con 0x5570ac9ea4d0 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.837+0000 7f3903fff640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39040b6510 0x7f39040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.838+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdb4010 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f391010dc10 tx=0x7f391010dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b103f4ba349d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.838+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39040b6510 0x7f39040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.838+0000 7f3903fff640 1 -- 172.21.15.92:0/2221721366 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f39101079d0 con 0x7f391c103620 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.838+0000 7f39017fa640 1 -- 172.21.15.92:0/4002142875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100ca190 con 0x5570ac9ea4d0 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.838+0000 7f39017fa640 1 -- 172.21.15.92:0/4002142875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39100c92a0 con 0x5570ac9ea4d0 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.838+0000 7f39017fa640 1 -- 172.21.15.92:0/4002142875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100cd2b0 con 0x5570ac9ea4d0 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- 172.21.15.92:0/4002142875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 msgr2=0x5570acdb4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdb4010 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f391010dc10 tx=0x7f391010dd00 comp rx=0 tx=0).stop 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39040b6510 0x7f39040b8a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3914009f70 tx=0x7f3914009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- 172.21.15.92:0/4002142875 shutdown_connections 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdb4010 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9e22f0 0x5570acdb34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- 172.21.15.92:0/4002142875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570acdb2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- 172.21.15.92:0/4002142875 >> 172.21.15.92:0/4002142875 conn(0x5570ac9ff300 msgr2=0x5570acdae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- 172.21.15.92:0/4002142875 shutdown_connections 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- 172.21.15.92:0/4002142875 wait complete. 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- start start 2024-09-21T02:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 Processor -- start 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdbc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570acdbc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 0x5570acdbd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5570acc4cab0 con 0x5570aca0b1b0 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5570accba730 con 0x5570ac9e22f0 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.839+0000 7f39285e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5570acdb48f0 con 0x5570ac9ea4d0 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f39285e7740 10 client.0 _ll_register_callbacks cb 0x5570acd0e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 0x5570acdbd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 0x5570acdbd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41576/0 (socket says 172.21.15.92:41576) 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 -- 172.21.15.92:0/2386879832 learned_addr learned my addr 172.21.15.92:0/2386879832 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570acdbc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391affd640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdbc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 -- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 msgr2=0x5570acdbc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570acdbc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 -- 172.21.15.92:0/2386879832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 msgr2=0x5570acdbc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdbc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 -- 172.21.15.92:0/2386879832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570aca76790 con 0x5570aca0b1b0 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 0x5570acdbd540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3914012960 tx=0x7f3914004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ee817cd7367a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f38feff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f38feff5640 1 -- 172.21.15.92:0/2386879832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400fd60 con 0x5570aca0b1b0 2024-09-21T02:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f38feff5640 1 -- 172.21.15.92:0/2386879832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f391401f420 con 0x5570aca0b1b0 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39100c92a0 con 0x5570aca0b1b0 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f38feff5640 1 -- 172.21.15.92:0/2386879832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f391402d070 con 0x5570aca0b1b0 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f39285e7740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.840+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570acaef620 con 0x5570aca0b1b0 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f38feff5640 1 -- 172.21.15.92:0/2386879832 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39100c92a0 con 0x5570aca0b1b0 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f38feff5640 1 -- 172.21.15.92:0/2386879832 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39140276a0 con 0x5570aca0b1b0 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f39285e7740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f39285e7740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f39285e7740 20 client.15903 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f39285e7740 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac9c9d80 0x5570acdc9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f39285e7740 4 client.15903 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570acd27640 con 0x5570ac9c9d80 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.841+0000 7f391affd640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac9c9d80 0x5570acdc9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.842+0000 7f38feff5640 10 client.15903 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.842+0000 7f391affd640 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac9c9d80 0x5570acdc9b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0c64b8bd22351d server_cookie=c232a9550f4ec942 in_seq=0 out_seq=0 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.842+0000 7f38feff5640 1 -- 172.21.15.92:0/2386879832 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570acd27640 con 0x5570ac9c9d80 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.842+0000 7f38feff5640 10 client.15903 handle_command_reply: tid=0 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.842+0000 7f39285e7740 1 client.15903 shutdown 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 msgr2=0x5570acdbd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 0x5570acdbd540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3914012960 tx=0x7f3914004a90 comp rx=0 tx=0).stop 2024-09-21T02:06:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 shutdown_connections 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac9c9d80 0x5570acdc9b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570aca0b1b0 0x5570acdbd540 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570acdbc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f38feff5640 0 client.15903 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 --2- 172.21.15.92:0/2386879832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdbc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 >> 172.21.15.92:0/2386879832 conn(0x5570ac9ff300 msgr2=0x5570acdacea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 shutdown_connections 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 1 -- 172.21.15.92:0/2386879832 wait complete. 2024-09-21T02:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.843+0000 7f39285e7740 20 client.15903 trim_cache size 0 max 16384 2024-09-21T02:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 Processor -- start 2024-09-21T02:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 -- start start 2024-09-21T02:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdbade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570aca0b1b0 0x5570acdbb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ab050 con 0x5570ac9ea4d0 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1acb90 con 0x5570ac9e22f0 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f39285e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ab720 con 0x5570aca0b1b0 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdbade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdbade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41586/0 (socket says 172.21.15.92:41586) 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f391a7fc640 1 -- 172.21.15.92:0/2121662603 learned_addr learned my addr 172.21.15.92:0/2121662603 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.857+0000 7f391affd640 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391b7fe640 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570aca0b1b0 0x5570acdbb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391a7fc640 1 -- 172.21.15.92:0/2121662603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570aca0b1b0 msgr2=0x5570acdbb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570aca0b1b0 0x5570acdbb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391a7fc640 1 -- 172.21.15.92:0/2121662603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 msgr2=0x5570acdba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdba8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391a7fc640 1 -- 172.21.15.92:0/2121662603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570acdaf260 con 0x5570ac9ea4d0 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f391a7fc640 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdbade0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f39100d0040 tx=0x7f39100b50c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ff8dfe4d88ffb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f3900ff9640 1 -- 172.21.15.92:0/2121662603 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910049070 con 0x5570ac9ea4d0 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f3900ff9640 1 -- 172.21.15.92:0/2121662603 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39100c92a0 con 0x5570ac9ea4d0 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.858+0000 7f3900ff9640 1 -- 172.21.15.92:0/2121662603 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100d2040 con 0x5570ac9ea4d0 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- 172.21.15.92:0/2121662603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 msgr2=0x5570acdbade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdbade0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f39100d0040 tx=0x7f39100b50c0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- 172.21.15.92:0/2121662603 shutdown_connections 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570aca0b1b0 0x5570acdbb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9ea4d0 0x5570acdbade0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- 172.21.15.92:0/2121662603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570ac9e22f0 0x5570acdba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- 172.21.15.92:0/2121662603 >> 172.21.15.92:0/2121662603 conn(0x5570ac9ff300 msgr2=0x5570acdae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- 172.21.15.92:0/2121662603 shutdown_connections 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- 172.21.15.92:0/2121662603 wait complete. 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- start start 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 Processor -- start 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 0x5570ace2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570ace2fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570ace309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ac4c0 con 0x5570ac9e22f0 2024-09-21T02:06:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1abdf0 con 0x5570aca0b1b0 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1a9030 con 0x5570ac9ea4d0 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f39285e7740 10 client.0 _ll_register_callbacks cb 0x7f3914009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f391affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 0x5570ace2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f391affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 0x5570ace2f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41592/0 (socket says 172.21.15.92:41592) 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f391affd640 1 -- 172.21.15.92:0/4277038549 learned_addr learned my addr 172.21.15.92:0/4277038549 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.859+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570ace2fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391b7fe640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570ace309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391affd640 1 -- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 msgr2=0x5570ace2fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391affd640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570ace2fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391affd640 1 -- 172.21.15.92:0/4277038549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 msgr2=0x5570ace309d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391affd640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570ace309d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391affd640 1 -- 172.21.15.92:0/4277038549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ace31130 con 0x5570ac9e22f0 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570ace2fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391b7fe640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570ace309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f391affd640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 0x5570ace2f7a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b840 tx=0x7f390c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c126ba48a68a97f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f38feff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f38feff5640 1 -- 172.21.15.92:0/4277038549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c029060 con 0x5570ac9e22f0 2024-09-21T02:06:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f38feff5640 1 -- 172.21.15.92:0/4277038549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f390c024670 con 0x5570ac9e22f0 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39100c92a0 con 0x5570ac9e22f0 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f38feff5640 1 -- 172.21.15.92:0/4277038549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f390c024870 con 0x5570ac9e22f0 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f39285e7740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570ace38560 con 0x5570ac9e22f0 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f38feff5640 1 -- 172.21.15.92:0/4277038549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39100c92a0 con 0x5570ac9e22f0 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f38feff5640 1 -- 172.21.15.92:0/4277038549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f390c02f520 con 0x5570ac9e22f0 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.860+0000 7f39285e7740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f39285e7740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f39285e7740 20 client.15909 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f39285e7740 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac99f1c0 0x5570ace3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f39285e7740 4 client.15909 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:06:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570acd230f0 con 0x5570ac99f1c0 2024-09-21T02:06:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac99f1c0 0x5570ace3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f38feff5640 10 client.15909 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.861+0000 7f391a7fc640 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac99f1c0 0x5570ace3af80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff5cfa64650fe89 server_cookie=48ccaef7c8b48b50 in_seq=0 out_seq=0 2024-09-21T02:06:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f38feff5640 1 -- 172.21.15.92:0/4277038549 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570acd230f0 con 0x5570ac99f1c0 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f38feff5640 10 client.15909 handle_command_reply: tid=0 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 client.15909 shutdown 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 msgr2=0x5570ace2f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 0x5570ace2f7a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b840 tx=0x7f390c00bf40 comp rx=0 tx=0).stop 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 shutdown_connections 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5570ac99f1c0 0x5570ace3af80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5570aca0b1b0 0x5570ace309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5570ac9ea4d0 0x5570ace2fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f38feff5640 0 client.15909 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 --2- 172.21.15.92:0/4277038549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570ac9e22f0 0x5570ace2f7a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 >> 172.21.15.92:0/4277038549 conn(0x5570ac9ff300 msgr2=0x5570acdaf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 shutdown_connections 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 1 -- 172.21.15.92:0/4277038549 wait complete. 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f39285e7740 20 client.15909 trim_cache size 0 max 16384 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:06:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39040b6510 msgr2=0x7f39040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.873+0000 7f392167e640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39040b6510 0x7f39040b8a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3914009f70 tx=0x7f3914009580 comp rx=0 tx=0).stop 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 msgr2=0x7f391c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c19ecd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3910006010 tx=0x7f3910004340 comp rx=0 tx=0).stop 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 shutdown_connections 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39040b6510 0x7f39040b8a20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c103fc0 0x7f391c1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c103620 0x7f391c19ecd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 --2- 172.21.15.92:0/2221721366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391c1025a0 0x7f391c06b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 >> 172.21.15.92:0/2221721366 conn(0x7f391c0fdee0 msgr2=0x7f391c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 shutdown_connections 2024-09-21T02:06:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:00.874+0000 7f392167e640 1 -- 172.21.15.92:0/2221721366 wait complete. 2024-09-21T02:06:01.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:06:01.250 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:01.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:01 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:06:01.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:01 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:01.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:01 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:06:01.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:01 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:01 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:06:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:01 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:01 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:06:01.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:01 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:01.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 -- 172.21.15.92:0/2481430154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e4103b60 msgr2=0x7f85e4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 --2- 172.21.15.92:0/2481430154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e4103b60 0x7f85e4103f70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009900 tx=0x7f85d402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 -- 172.21.15.92:0/2481430154 shutdown_connections 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 --2- 172.21.15.92:0/2481430154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85e4109df0 0x7f85e4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 --2- 172.21.15.92:0/2481430154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85e41094a0 0x7f85e41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 --2- 172.21.15.92:0/2481430154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e4103b60 0x7f85e4103f70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 -- 172.21.15.92:0/2481430154 >> 172.21.15.92:0/2481430154 conn(0x7f85e407b850 msgr2=0x7f85e407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 -- 172.21.15.92:0/2481430154 shutdown_connections 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.738+0000 7f85e93a1640 1 -- 172.21.15.92:0/2481430154 wait complete. 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 Processor -- start 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 -- start start 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85e4103b60 0x7f85e419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 0x7f85e419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85e4109df0 0x7f85e419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113ab0 con 0x7f85e41094a0 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113930 con 0x7f85e4103b60 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e93a1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113c30 con 0x7f85e4109df0 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 0x7f85e419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 0x7f85e419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41614/0 (socket says 172.21.15.92:41614) 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 -- 172.21.15.92:0/3663569255 learned_addr learned my addr 172.21.15.92:0/3663569255 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e37fe640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85e4109df0 0x7f85e419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85e4103b60 0x7f85e419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 -- 172.21.15.92:0/3663569255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85e4109df0 msgr2=0x7f85e419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85e4109df0 0x7f85e419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 -- 172.21.15.92:0/3663569255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85e4103b60 msgr2=0x7f85e419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.739+0000 7f85e27fc640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85e4103b60 0x7f85e419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85e27fc640 1 -- 172.21.15.92:0/3663569255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e41a0310 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85e27fc640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 0x7f85e419aa00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c8c0 tx=0x7f85d800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85cbfff640 1 -- 172.21.15.92:0/3663569255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8007bb0 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e41a0600 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85cbfff640 1 -- 172.21.15.92:0/3663569255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d8007d50 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85cbfff640 1 -- 172.21.15.92:0/3663569255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80187c0 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.740+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a6e20 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.742+0000 7f85ef915740 1 Processor -- start 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.742+0000 7f85ef915740 1 -- start start 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85cbfff640 1 -- 172.21.15.92:0/3663569255 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f85d800fa20 con 0x7f85e41094a0 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85ef915740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85ef915740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85ef915740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85ef915740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba99ce50 con 0x55f7ba65e390 2024-09-21T02:06:01.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85ef915740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba98c280 con 0x55f7ba67f070 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85ef915740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba990090 con 0x55f7ba6561b0 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa270d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41622/0 (socket says 172.21.15.92:41622) 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 -- 172.21.15.92:0/931254378 learned_addr learned my addr 172.21.15.92:0/931254378 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e37fe640 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 -- 172.21.15.92:0/931254378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 msgr2=0x55f7baa26b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa26b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 -- 172.21.15.92:0/931254378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 msgr2=0x55f7baa26650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa26650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 -- 172.21.15.92:0/931254378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ba6ea650 con 0x55f7ba65e390 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.743+0000 7f85e27fc640 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa270d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ed600 tx=0x7f85d80ed6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffeaa2a70b24870 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.744+0000 7f85cbfff640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f85cc0b6540 0x7f85cc0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.744+0000 7f85c97fa640 1 -- 172.21.15.92:0/931254378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80fb070 con 0x55f7ba65e390 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.744+0000 7f85c97fa640 1 -- 172.21.15.92:0/931254378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d80ad110 con 0x55f7ba65e390 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.744+0000 7f85c97fa640 1 -- 172.21.15.92:0/931254378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80acd70 con 0x55f7ba65e390 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.744+0000 7f85e37fe640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f85cc0b6540 0x7f85cc0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.744+0000 7f85cbfff640 1 -- 172.21.15.92:0/3663569255 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f85d80e7490 con 0x7f85e41094a0 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- 172.21.15.92:0/931254378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 msgr2=0x55f7baa270d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85e37fe640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f85cc0b6540 0x7f85cc0b8a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f85dc007e50 tx=0x7f85dc007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa270d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ed600 tx=0x7f85d80ed6f0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- 172.21.15.92:0/931254378 shutdown_connections 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa270d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa26b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- 172.21.15.92:0/931254378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- 172.21.15.92:0/931254378 >> 172.21.15.92:0/931254378 conn(0x55f7ba673460 msgr2=0x55f7baa24540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- 172.21.15.92:0/931254378 shutdown_connections 2024-09-21T02:06:01.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- 172.21.15.92:0/931254378 wait complete. 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- start start 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 Processor -- start 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa31c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa32190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba7ed700 con 0x55f7ba65e390 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f7ba927110 con 0x55f7ba67f070 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.745+0000 7f85ef915740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f7baa34e30 con 0x55f7ba6561b0 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85ef915740 10 client.0 _ll_register_callbacks cb 0x55f7ba940990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa32190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa32190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41628/0 (socket says 172.21.15.92:41628) 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 -- 172.21.15.92:0/492876495 learned_addr learned my addr 172.21.15.92:0/492876495 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e37fe640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa31c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 -- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 msgr2=0x55f7baa31c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa31c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 -- 172.21.15.92:0/492876495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 msgr2=0x55f7baa326d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa326d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 -- 172.21.15.92:0/492876495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ba954c40 con 0x55f7ba65e390 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85e27fc640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa32190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ad300 tx=0x7f85d80e7790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c526d2edddf9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85c6ff5640 1 -- 172.21.15.92:0/492876495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80bc610 con 0x55f7ba65e390 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85c6ff5640 1 -- 172.21.15.92:0/492876495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f85d801f300 con 0x55f7ba65e390 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d80ad110 con 0x55f7ba65e390 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.746+0000 7f85c6ff5640 1 -- 172.21.15.92:0/492876495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85d80bc860 con 0x55f7ba65e390 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 10 client.15921 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7baa40a80 con 0x55f7ba65e390 2024-09-21T02:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85c6ff5640 1 -- 172.21.15.92:0/492876495 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d80ad110 con 0x55f7ba65e390 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85c6ff5640 1 -- 172.21.15.92:0/492876495 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85d80b6b90 con 0x55f7ba65e390 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 10 client.15921 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 20 client.15921 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba63d9f0 0x55f7baa43de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 4 client.15921 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7ba99eda0 con 0x55f7ba63d9f0 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.747+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba63d9f0 0x55f7baa43de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.748+0000 7f85c6ff5640 10 client.15921 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.748+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba63d9f0 0x55f7baa43de0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2955ef5dec54f8df server_cookie=566c6da6f7758685 in_seq=0 out_seq=0 2024-09-21T02:06:01.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85c6ff5640 1 -- 172.21.15.92:0/492876495 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7ba99eda0 con 0x55f7ba63d9f0 2024-09-21T02:06:01.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85c6ff5640 10 client.15921 handle_command_reply: tid=0 2024-09-21T02:06:01.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 client.15921 shutdown 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 msgr2=0x55f7baa32190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa32190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ad300 tx=0x7f85d80e7790 comp rx=0 tx=0).stop 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 shutdown_connections 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba63d9f0 0x55f7baa43de0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba67f070 0x55f7baa326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba65e390 0x55f7baa32190 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 --2- 172.21.15.92:0/492876495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa31c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 >> 172.21.15.92:0/492876495 conn(0x55f7ba673460 msgr2=0x55f7baa29830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85c6ff5640 0 client.15921 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 shutdown_connections 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 1 -- 172.21.15.92:0/492876495 wait complete. 2024-09-21T02:06:01.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.749+0000 7f85ef915740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 Processor -- start 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 -- start start 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa33880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa2ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 0x55f7baa2f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41af400 con 0x55f7ba67f070 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b0f40 con 0x55f7ba65e390 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.766+0000 7f85ef915740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41afad0 con 0x55f7ba6561b0 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 0x55f7baa2f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 0x55f7baa2f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41632/0 (socket says 172.21.15.92:41632) 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 -- 172.21.15.92:0/634845657 learned_addr learned my addr 172.21.15.92:0/634845657 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa33880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e27fc640 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa2ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 -- 172.21.15.92:0/634845657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 msgr2=0x55f7baa33880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa33880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 -- 172.21.15.92:0/634845657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 msgr2=0x55f7baa2ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa2ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 -- 172.21.15.92:0/634845657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ba63dc20 con 0x55f7ba67f070 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e27fc640 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa2ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85e37fe640 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 0x55f7baa2f380 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f85dc01d600 tx=0x7f85dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c02baf8be7837d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85c8ff9640 1 -- 172.21.15.92:0/634845657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc01c610 con 0x55f7ba67f070 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85c8ff9640 1 -- 172.21.15.92:0/634845657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85dc01cc60 con 0x55f7ba67f070 2024-09-21T02:06:01.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.767+0000 7f85c8ff9640 1 -- 172.21.15.92:0/634845657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc026d60 con 0x55f7ba67f070 2024-09-21T02:06:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 -- 172.21.15.92:0/634845657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 msgr2=0x55f7baa2f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 0x55f7baa2f380 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f85dc01d600 tx=0x7f85dc014000 comp rx=0 tx=0).stop 2024-09-21T02:06:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 -- 172.21.15.92:0/634845657 shutdown_connections 2024-09-21T02:06:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba67f070 0x55f7baa2f380 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa2ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 --2- 172.21.15.92:0/634845657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba6561b0 0x55f7baa33880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 -- 172.21.15.92:0/634845657 >> 172.21.15.92:0/634845657 conn(0x55f7ba673460 msgr2=0x55f7baa33170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 -- 172.21.15.92:0/634845657 shutdown_connections 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 -- 172.21.15.92:0/634845657 wait complete. 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.768+0000 7f85ef915740 1 -- start start 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 Processor -- start 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 0x55f7baa44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa45270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba67f070 0x55f7baa457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b0870 con 0x55f7ba6561b0 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41b01a0 con 0x55f7ba65e390 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41ad3e0 con 0x55f7ba67f070 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85ef915740 10 client.0 _ll_register_callbacks cb 0x7f85d80f9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:01.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 0x55f7baa44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 0x55f7baa44d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41648/0 (socket says 172.21.15.92:41648) 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 -- 172.21.15.92:0/3400173262 learned_addr learned my addr 172.21.15.92:0/3400173262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e37fe640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba67f070 0x55f7baa457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e27fc640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa45270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 -- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba67f070 msgr2=0x55f7baa457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba67f070 0x55f7baa457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 -- 172.21.15.92:0/3400173262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 msgr2=0x55f7baa45270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa45270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.769+0000 7f85e2ffd640 1 -- 172.21.15.92:0/3400173262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7baaad560 con 0x55f7ba6561b0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85e2ffd640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 0x55f7baa44d30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85d402a910 tx=0x7f85d4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240eb40df274cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85c6ff5640 1 -- 172.21.15.92:0/3400173262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4035040 con 0x55f7ba6561b0 2024-09-21T02:06:01.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85c6ff5640 1 -- 172.21.15.92:0/3400173262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f85d40316a0 con 0x55f7ba6561b0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc01cc60 con 0x55f7ba6561b0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85c6ff5640 1 -- 172.21.15.92:0/3400173262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85d4055050 con 0x55f7ba6561b0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 10 client.15927 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7baaae710 con 0x55f7ba6561b0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85c6ff5640 1 -- 172.21.15.92:0/3400173262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85dc01cc60 con 0x55f7ba6561b0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85c6ff5640 1 -- 172.21.15.92:0/3400173262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85d4004830 con 0x55f7ba6561b0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 10 client.15927 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 20 client.15927 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba612da0 0x55f7baab0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 4 client.15927 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.770+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7ba99a850 con 0x55f7ba612da0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85e27fc640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba612da0 0x55f7baab0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:01.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85c6ff5640 10 client.15927 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85e27fc640 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba612da0 0x55f7baab0eb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374012b242177b12 server_cookie=dea2c5a32633409f in_seq=0 out_seq=0 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85c6ff5640 1 -- 172.21.15.92:0/3400173262 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55f7ba99a850 con 0x55f7ba612da0 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85c6ff5640 10 client.15927 handle_command_reply: tid=0 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85ef915740 1 client.15927 shutdown 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 msgr2=0x55f7baa44d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.771+0000 7f85ef915740 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 0x55f7baa44d30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85d402a910 tx=0x7f85d4004180 comp rx=0 tx=0).stop 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 shutdown_connections 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f7ba612da0 0x55f7baab0eb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f7ba67f070 0x55f7baa457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7ba65e390 0x55f7baa45270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 --2- 172.21.15.92:0/3400173262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7ba6561b0 0x55f7baa44d30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 >> 172.21.15.92:0/3400173262 conn(0x55f7ba673460 msgr2=0x55f7baa29750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85c6ff5640 0 client.15927 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:01.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 shutdown_connections 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 1 -- 172.21.15.92:0/3400173262 wait complete. 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85ef915740 20 client.15927 trim_cache size 0 max 16384 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "6efa3f01-9ec1-473a-908e-a63bfd3275d7", 2024-09-21T02:06:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:01.780 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:01.781 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:06:01.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f85cc0b6540 msgr2=0x7f85cc0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f85cc0b6540 0x7f85cc0b8a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f85dc007e50 tx=0x7f85dc007600 comp rx=0 tx=0).stop 2024-09-21T02:06:01.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 msgr2=0x7f85e419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 0x7f85e419aa00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c8c0 tx=0x7f85d800cd80 comp rx=0 tx=0).stop 2024-09-21T02:06:01.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 shutdown_connections 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f85cc0b6540 0x7f85cc0b8a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85e4109df0 0x7f85e419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85e41094a0 0x7f85e419aa00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 --2- 172.21.15.92:0/3663569255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85e4103b60 0x7f85e419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 >> 172.21.15.92:0/3663569255 conn(0x7f85e407b850 msgr2=0x7f85e4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 shutdown_connections 2024-09-21T02:06:01.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:01.772+0000 7f85e93a1640 1 -- 172.21.15.92:0/3663569255 wait complete. 2024-09-21T02:06:01.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:01 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:06:01.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:01 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:01.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:01 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:06:01.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:01 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:02.429 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:02 smithi092 ceph-mon[31190]: pgmap v392: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:06:02.455 INFO:tasks.cephfs.filesystem:scrub status for tag:6efa3f01-9ec1-473a-908e-a63bfd3275d7 - {'path': '/', 'tag': '6efa3f01-9ec1-473a-908e-a63bfd3275d7', 'options': 'recursive,force'} 2024-09-21T02:06:02.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:02 smithi099 ceph-mon[34404]: pgmap v392: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:06:02.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:02 smithi136 ceph-mon[34754]: pgmap v392: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:06:03.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:03 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:06:03.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:03 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:06:03.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:03 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:06:04.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:04 smithi099 ceph-mon[34404]: pgmap v393: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:06:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:04 smithi136 ceph-mon[34754]: pgmap v393: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:06:04.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:04 smithi092 ceph-mon[31190]: pgmap v393: 97 pgs: 97 active+clean; 1011 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:06:06.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:06 smithi099 ceph-mon[34404]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:06 smithi136 ceph-mon[34754]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:06 smithi092 ceph-mon[31190]: pgmap v394: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:08.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:08 smithi099 ceph-mon[34404]: pgmap v395: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:06:08.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:08 smithi136 ceph-mon[34754]: pgmap v395: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:06:08.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:08 smithi092 ceph-mon[31190]: pgmap v395: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:06:10.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:10 smithi099 ceph-mon[34404]: pgmap v396: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:06:10.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:10 smithi136 ceph-mon[34754]: pgmap v396: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:06:10.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:10 smithi092 ceph-mon[31190]: pgmap v396: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:06:12.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:12 smithi099 ceph-mon[34404]: pgmap v397: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:12.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:12 smithi136 ceph-mon[34754]: pgmap v397: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:12.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:12 smithi092 ceph-mon[31190]: pgmap v397: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 -- 172.21.15.92:0/2964052512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2381036e0 msgr2=0x7fb238103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/2964052512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2381036e0 0x7fb238103b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb22802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 -- 172.21.15.92:0/2964052512 shutdown_connections 2024-09-21T02:06:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/2964052512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb238105100 0x7fb23810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/2964052512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb238104760 0x7fb238104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/2964052512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2381036e0 0x7fb238103b50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 -- 172.21.15.92:0/2964052512 >> 172.21.15.92:0/2964052512 conn(0x7fb2380ff020 msgr2=0x7fb2381014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 -- 172.21.15.92:0/2964052512 shutdown_connections 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.987+0000 7fb23e5ba640 1 -- 172.21.15.92:0/2964052512 wait complete. 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 Processor -- start 2024-09-21T02:06:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 -- start start 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2381036e0 0x7fb238199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb238104760 0x7fb23819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 0x7fb23819b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810f300 con 0x7fb238105100 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810f180 con 0x7fb238104760 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23e5ba640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810f480 con 0x7fb2381036e0 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23cb57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 0x7fb23819b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23cb57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 0x7fb23819b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60106/0 (socket says 172.21.15.92:60106) 2024-09-21T02:06:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.988+0000 7fb23cb57640 1 -- 172.21.15.92:0/45309859 learned_addr learned my addr 172.21.15.92:0/45309859 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb237fff640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2381036e0 0x7fb238199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb2377fe640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb238104760 0x7fb23819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23cb57640 1 -- 172.21.15.92:0/45309859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2381036e0 msgr2=0x7fb238199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23cb57640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2381036e0 0x7fb238199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23cb57640 1 -- 172.21.15.92:0/45309859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb238104760 msgr2=0x7fb23819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23cb57640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb238104760 0x7fb23819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23cb57640 1 -- 172.21.15.92:0/45309859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23819b990 con 0x7fb238105100 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb237fff640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2381036e0 0x7fb238199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23cb57640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 0x7fb23819b170 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb23000eb10 tx=0x7fb23000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb2357fa640 1 -- 172.21.15.92:0/45309859 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230004b20 con 0x7fb238105100 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23819bc80 con 0x7fb238105100 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb2357fa640 1 -- 172.21.15.92:0/45309859 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230004cc0 con 0x7fb238105100 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb2357fa640 1 -- 172.21.15.92:0/45309859 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230010670 con 0x7fb238105100 2024-09-21T02:06:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.989+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381a2370 con 0x7fb238105100 2024-09-21T02:06:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb245602740 1 Processor -- start 2024-09-21T02:06:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb2357fa640 1 -- 172.21.15.92:0/45309859 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb230005020 con 0x7fb238105100 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb245602740 1 -- start start 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb245602740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1d41b0 0x560f2c57be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb245602740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c57c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb245602740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c57d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.992+0000 7fb245602740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f2c4ee6f0 con 0x560f2c1b34d0 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb245602740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f2c4ddb20 con 0x560f2c1d41b0 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb245602740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560f2c4e1930 con 0x560f2c1ab2f0 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c57d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c57d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60108/0 (socket says 172.21.15.92:60108) 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb237fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c57c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 -- 172.21.15.92:0/3099360960 learned_addr learned my addr 172.21.15.92:0/3099360960 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1d41b0 0x560f2c57be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 -- 172.21.15.92:0/3099360960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 msgr2=0x560f2c57c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c57c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 -- 172.21.15.92:0/3099360960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1d41b0 msgr2=0x560f2c57be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1d41b0 0x560f2c57be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 -- 172.21.15.92:0/3099360960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2c4a3530 con 0x560f2c1b34d0 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb237fff640 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c57c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2357fa640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2200b6590 0x7fb2200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb237fff640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2200b6590 0x7fb2200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2377fe640 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c57d010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb22c013510 tx=0x7fb22c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a469822f843a8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.993+0000 7fb2357fa640 1 -- 172.21.15.92:0/45309859 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb2300e79f0 con 0x7fb238105100 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.994+0000 7fb21e7fc640 1 -- 172.21.15.92:0/3099360960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c019070 con 0x560f2c1b34d0 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.994+0000 7fb21e7fc640 1 -- 172.21.15.92:0/3099360960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c004220 con 0x560f2c1b34d0 2024-09-21T02:06:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.994+0000 7fb21e7fc640 1 -- 172.21.15.92:0/3099360960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c011700 con 0x560f2c1b34d0 2024-09-21T02:06:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.994+0000 7fb245602740 1 -- 172.21.15.92:0/3099360960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 msgr2=0x560f2c57d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.994+0000 7fb237fff640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2200b6590 0x7fb2200b8aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb228031440 tx=0x7fb2280058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.994+0000 7fb245602740 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c57d010 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb22c013510 tx=0x7fb22c009740 comp rx=0 tx=0).stop 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 -- 172.21.15.92:0/3099360960 shutdown_connections 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c57d010 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c57c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 --2- 172.21.15.92:0/3099360960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1d41b0 0x560f2c57be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 -- 172.21.15.92:0/3099360960 >> 172.21.15.92:0/3099360960 conn(0x560f2c1c8300 msgr2=0x560f2c5778e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 -- 172.21.15.92:0/3099360960 shutdown_connections 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 -- 172.21.15.92:0/3099360960 wait complete. 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 -- start start 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.995+0000 7fb245602740 1 Processor -- start 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1ab2f0 0x560f2c585660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c585d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1d41b0 0x560f2c586890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f2c415ab0 con 0x560f2c1b34d0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f2c483730 con 0x560f2c1ab2f0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560f2c57db30 con 0x560f2c1d41b0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 10 client.0 _ll_register_callbacks cb 0x560f2c4d76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c585d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb237fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1ab2f0 0x560f2c585660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c585d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60112/0 (socket says 172.21.15.92:60112) 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 -- 172.21.15.92:0/1481382191 learned_addr learned my addr 172.21.15.92:0/1481382191 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb23cb57640 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1d41b0 0x560f2c586890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 -- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1d41b0 msgr2=0x560f2c586890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1d41b0 0x560f2c586890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 -- 172.21.15.92:0/1481382191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1ab2f0 msgr2=0x560f2c585660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1ab2f0 0x560f2c585660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 -- 172.21.15.92:0/1481382191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2c23f790 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2377fe640 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c585d20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb22c005bc0 tx=0x7fb22c003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5dc9ddbd6b6e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2177fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2177fe640 1 -- 172.21.15.92:0/1481382191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c019070 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb2177fe640 1 -- 172.21.15.92:0/1481382191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb22c005250 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.996+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb22c004220 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb2177fe640 1 -- 172.21.15.92:0/1481382191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb22c02c070 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 10 client.15939 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f2c2b8620 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb2177fe640 1 -- 172.21.15.92:0/1481382191 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c004220 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb2177fe640 1 -- 172.21.15.92:0/1481382191 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb22c024020 con 0x560f2c1b34d0 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 20 client.15939 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c192d80 0x560f2c592b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 4 client.15939 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.997+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f2c4f0640 con 0x560f2c192d80 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.998+0000 7fb237fff640 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c192d80 0x560f2c592b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.998+0000 7fb237fff640 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c192d80 0x560f2c592b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750773e42476ec31 server_cookie=7873cf6b949fbb5f in_seq=0 out_seq=0 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.998+0000 7fb2177fe640 10 client.15939 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb2177fe640 1 -- 172.21.15.92:0/1481382191 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f2c4f0640 con 0x560f2c192d80 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb2177fe640 10 client.15939 handle_command_reply: tid=0 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 client.15939 shutdown 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 msgr2=0x560f2c585d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c585d20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb22c005bc0 tx=0x7fb22c003b70 comp rx=0 tx=0).stop 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 shutdown_connections 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c192d80 0x560f2c592b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1d41b0 0x560f2c586890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb2177fe640 0 client.15939 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1b34d0 0x560f2c585d20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 --2- 172.21.15.92:0/1481382191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1ab2f0 0x560f2c585660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 >> 172.21.15.92:0/1481382191 conn(0x560f2c1c8300 msgr2=0x560f2c575ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 shutdown_connections 2024-09-21T02:06:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 1 -- 172.21.15.92:0/1481382191 wait complete. 2024-09-21T02:06:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:13.999+0000 7fb245602740 20 client.15939 trim_cache size 0 max 16384 2024-09-21T02:06:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 Processor -- start 2024-09-21T02:06:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 -- start start 2024-09-21T02:06:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c583b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c5840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ab050 con 0x560f2c1d41b0 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381acb90 con 0x560f2c1b34d0 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb245602740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ab720 con 0x560f2c1ab2f0 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c5840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c5840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60128/0 (socket says 172.21.15.92:60128) 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 -- 172.21.15.92:0/934803921 learned_addr learned my addr 172.21.15.92:0/934803921 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb237fff640 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c583b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 -- 172.21.15.92:0/934803921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 msgr2=0x560f2c583b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c583b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 -- 172.21.15.92:0/934803921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 msgr2=0x560f2c58d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb2377fe640 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.014+0000 7fb23cb57640 1 -- 172.21.15.92:0/934803921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2c576fd0 con 0x560f2c1d41b0 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb23cb57640 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c5840b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb2300e5520 tx=0x7fb23000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b3241b09450445 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb21dffb640 1 -- 172.21.15.92:0/934803921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300ad670 con 0x560f2c1d41b0 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb21dffb640 1 -- 172.21.15.92:0/934803921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2300ae260 con 0x560f2c1d41b0 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb21dffb640 1 -- 172.21.15.92:0/934803921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300b2220 con 0x560f2c1d41b0 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb245602740 1 -- 172.21.15.92:0/934803921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 msgr2=0x560f2c5840b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb245602740 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c5840b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb2300e5520 tx=0x7fb23000cf70 comp rx=0 tx=0).stop 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb245602740 1 -- 172.21.15.92:0/934803921 shutdown_connections 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb245602740 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c5840b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb245602740 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.015+0000 7fb245602740 1 --2- 172.21.15.92:0/934803921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c583b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- 172.21.15.92:0/934803921 >> 172.21.15.92:0/934803921 conn(0x560f2c1c8300 msgr2=0x560f2c584860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- 172.21.15.92:0/934803921 shutdown_connections 2024-09-21T02:06:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- 172.21.15.92:0/934803921 wait complete. 2024-09-21T02:06:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- start start 2024-09-21T02:06:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 Processor -- start 2024-09-21T02:06:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c58a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c58bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381ac4c0 con 0x560f2c1d41b0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381abdf0 con 0x560f2c1b34d0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2381a9030 con 0x560f2c1ab2f0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.016+0000 7fb245602740 10 client.0 _ll_register_callbacks cb 0x7fb22c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c58bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c58bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60138/0 (socket says 172.21.15.92:60138) 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 -- 172.21.15.92:0/3542423371 learned_addr learned my addr 172.21.15.92:0/3542423371 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb2377fe640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb237fff640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c58a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 -- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 msgr2=0x560f2c58a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c58a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 -- 172.21.15.92:0/3542423371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 msgr2=0x560f2c58b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 -- 172.21.15.92:0/3542423371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2c58c2c0 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb237fff640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c58a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb23cb57640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c58bb60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb2300aadd0 tx=0x7fb23000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4459523cf6e84957 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb2177fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb2177fe640 1 -- 172.21.15.92:0/3542423371 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300ad670 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb2177fe640 1 -- 172.21.15.92:0/3542423371 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb2300b72f0 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2300ae260 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb2177fe640 1 -- 172.21.15.92:0/3542423371 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2300b2640 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb245602740 10 client.15945 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f2c58d5e0 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.017+0000 7fb2177fe640 1 -- 172.21.15.92:0/3542423371 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2300ae260 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb2177fe640 1 -- 172.21.15.92:0/3542423371 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb230096030 con 0x560f2c1d41b0 2024-09-21T02:06:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb245602740 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb245602740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb245602740 20 client.15945 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb245602740 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c1681c0 0x560f2c5ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb245602740 4 client.15945 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f2c4ec0f0 con 0x560f2c1681c0 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.018+0000 7fb237fff640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c1681c0 0x560f2c5ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.019+0000 7fb2177fe640 10 client.15945 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.019+0000 7fb237fff640 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c1681c0 0x560f2c5ff7c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91a8aba76ad4aa7 server_cookie=f4a12376abbdb0d1 in_seq=0 out_seq=0 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb2177fe640 1 -- 172.21.15.92:0/3542423371 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f2c4ec0f0 con 0x560f2c1681c0 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb2177fe640 10 client.15945 handle_command_reply: tid=0 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 client.15945 shutdown 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 msgr2=0x560f2c58bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c58bb60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb2300aadd0 tx=0x7fb23000cee0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 shutdown_connections 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f2c1681c0 0x560f2c5ff7c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f2c1d41b0 0x560f2c58bb60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2c1b34d0 0x560f2c58b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb2177fe640 0 client.15945 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 --2- 172.21.15.92:0/3542423371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f2c1ab2f0 0x560f2c58a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 >> 172.21.15.92:0/3542423371 conn(0x560f2c1c8300 msgr2=0x560f2c57bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 shutdown_connections 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 1 -- 172.21.15.92:0/3542423371 wait complete. 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.031+0000 7fb245602740 20 client.15945 trim_cache size 0 max 16384 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:06:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:06:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2200b6590 msgr2=0x7fb2200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2200b6590 0x7fb2200b8aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb228031440 tx=0x7fb2280058b0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 msgr2=0x7fb23819b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 0x7fb23819b170 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb23000eb10 tx=0x7fb23000c2d0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 shutdown_connections 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2200b6590 0x7fb2200b8aa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb238105100 0x7fb23819b170 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb238104760 0x7fb23819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 --2- 172.21.15.92:0/45309859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2381036e0 0x7fb238199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 >> 172.21.15.92:0/45309859 conn(0x7fb2380ff020 msgr2=0x7fb238107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 shutdown_connections 2024-09-21T02:06:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:14.032+0000 7fb23e5ba640 1 -- 172.21.15.92:0/45309859 wait complete. 2024-09-21T02:06:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:14 smithi099 ceph-mon[34404]: pgmap v398: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:06:14.606 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:14 smithi092 ceph-mon[31190]: pgmap v398: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:06:14.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:14 smithi136 ceph-mon[34754]: pgmap v398: 97 pgs: 97 active+clean; 1.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:06:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:16 smithi099 ceph-mon[34404]: pgmap v399: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:06:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:16 smithi136 ceph-mon[34754]: pgmap v399: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:06:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:16 smithi092 ceph-mon[31190]: pgmap v399: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:06:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:18 smithi099 ceph-mon[34404]: pgmap v400: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:06:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:18 smithi136 ceph-mon[34754]: pgmap v400: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:06:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:18 smithi092 ceph-mon[31190]: pgmap v400: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:06:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:20 smithi136 ceph-mon[34754]: pgmap v401: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:20.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:20 smithi092 ceph-mon[31190]: pgmap v401: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:20.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:20 smithi099 ceph-mon[34404]: pgmap v401: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:06:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:22 smithi136 ceph-mon[34754]: pgmap v402: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:06:22.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:22 smithi092 ceph-mon[31190]: pgmap v402: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:06:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:22 smithi099 ceph-mon[34404]: pgmap v402: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:06:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:24 smithi136 ceph-mon[34754]: pgmap v403: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:06:24.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:24 smithi092 ceph-mon[31190]: pgmap v403: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:06:24.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:24 smithi099 ceph-mon[34404]: pgmap v403: 97 pgs: 97 active+clean; 1.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:06:26.633 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:26 smithi099 ceph-mon[34404]: pgmap v404: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:06:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:26 smithi136 ceph-mon[34754]: pgmap v404: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:06:26.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:26 smithi092 ceph-mon[31190]: pgmap v404: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:06:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 -- 172.21.15.92:0/4106109944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 msgr2=0x7f4c6410b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 --2- 172.21.15.92:0/4106109944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6410b0c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009900 tx=0x7f4c5802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 -- 172.21.15.92:0/4106109944 shutdown_connections 2024-09-21T02:06:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 --2- 172.21.15.92:0/4106109944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c6410b600 0x7f4c6410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 --2- 172.21.15.92:0/4106109944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6410b0c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 --2- 172.21.15.92:0/4106109944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c6406c2a0 0x7f4c64108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 -- 172.21.15.92:0/4106109944 >> 172.21.15.92:0/4106109944 conn(0x7f4c640fdf00 msgr2=0x7f4c641003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 -- 172.21.15.92:0/4106109944 shutdown_connections 2024-09-21T02:06:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.146+0000 7f4c69907640 1 -- 172.21.15.92:0/4106109944 wait complete. 2024-09-21T02:06:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 Processor -- start 2024-09-21T02:06:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 -- start start 2024-09-21T02:06:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c6406c2a0 0x7f4c6419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c6410b600 0x7f4c6419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641114f0 con 0x7f4c64108ca0 2024-09-21T02:06:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64111370 con 0x7f4c6410b600 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c69907640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64111670 con 0x7f4c6406c2a0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.147+0000 7f4c627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46442/0 (socket says 172.21.15.92:46442) 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 -- 172.21.15.92:0/3644819328 learned_addr learned my addr 172.21.15.92:0/3644819328 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c6406c2a0 0x7f4c6419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c6410b600 0x7f4c6419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 -- 172.21.15.92:0/3644819328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c6406c2a0 msgr2=0x7f4c6419c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c6406c2a0 0x7f4c6419c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 -- 172.21.15.92:0/3644819328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c6410b600 msgr2=0x7f4c6419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c6410b600 0x7f4c6419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 -- 172.21.15.92:0/3644819328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6419db90 con 0x7f4c64108ca0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c627fc640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6419c7c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006010 tx=0x7f4c58004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c68905640 1 -- 172.21.15.92:0/3644819328 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5803f030 con 0x7f4c64108ca0 2024-09-21T02:06:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6419de20 con 0x7f4c64108ca0 2024-09-21T02:06:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c68905640 1 -- 172.21.15.92:0/3644819328 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c58003cb0 con 0x7f4c64108ca0 2024-09-21T02:06:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c68905640 1 -- 172.21.15.92:0/3644819328 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58035070 con 0x7f4c64108ca0 2024-09-21T02:06:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.148+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a4570 con 0x7f4c64108ca0 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.151+0000 7f4c70872740 1 Processor -- start 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.151+0000 7f4c70872740 1 -- start start 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c70872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 0x5582e054ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c70872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e054d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c70872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e054e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c70872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5582e04bf6f0 con 0x5582e01844d0 2024-09-21T02:06:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c70872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5582e04aeb20 con 0x5582e01a51b0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c70872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5582e04b2930 con 0x5582e017c2f0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 0x5582e054ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 0x5582e054ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:52202/0 (socket says 172.21.15.92:52202) 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 -- 172.21.15.92:0/3667482063 learned_addr learned my addr 172.21.15.92:0/3667482063 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c68905640 1 -- 172.21.15.92:0/3644819328 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4c58042030 con 0x7f4c64108ca0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e054d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c627fc640 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e054e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 -- 172.21.15.92:0/3667482063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 msgr2=0x5582e054d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e054d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 -- 172.21.15.92:0/3667482063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 msgr2=0x5582e054e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e054e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.152+0000 7f4c637fe640 1 -- 172.21.15.92:0/3667482063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e0474530 con 0x5582e01a51b0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 0x5582e054ce50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00b700 tx=0x7f4c5c00bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9019da64e27972f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c68905640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c4c0b6510 0x7f4c4c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c49ffb640 1 -- 172.21.15.92:0/3667482063 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c003dd0 con 0x5582e01a51b0 2024-09-21T02:06:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c4c0b6510 0x7f4c4c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c49ffb640 1 -- 172.21.15.92:0/3667482063 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c015070 con 0x5582e01a51b0 2024-09-21T02:06:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c49ffb640 1 -- 172.21.15.92:0/3667482063 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0106a0 con 0x5582e01a51b0 2024-09-21T02:06:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.153+0000 7f4c68905640 1 -- 172.21.15.92:0/3644819328 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4c58107920 con 0x7f4c64108ca0 2024-09-21T02:06:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 -- 172.21.15.92:0/3667482063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 msgr2=0x5582e054ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 0x5582e054ce50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00b700 tx=0x7f4c5c00bbc0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c4c0b6510 0x7f4c4c0b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005fd0 tx=0x7f4c54009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 -- 172.21.15.92:0/3667482063 shutdown_connections 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e054e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e054d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 --2- 172.21.15.92:0/3667482063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01a51b0 0x5582e054ce50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 -- 172.21.15.92:0/3667482063 >> 172.21.15.92:0/3667482063 conn(0x5582e0199300 msgr2=0x5582e05488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 -- 172.21.15.92:0/3667482063 shutdown_connections 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.154+0000 7f4c70872740 1 -- 172.21.15.92:0/3667482063 wait complete. 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 -- start start 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 Processor -- start 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e0556620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01844d0 0x5582e0556ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e0557850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5582e03e6ab0 con 0x5582e01a51b0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5582e0454730 con 0x5582e01844d0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5582e054eb30 con 0x5582e017c2f0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c70872740 10 client.0 _ll_register_callbacks cb 0x5582e04a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e0557850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e0557850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46472/0 (socket says 172.21.15.92:46472) 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c637fe640 1 -- 172.21.15.92:0/3271777920 learned_addr learned my addr 172.21.15.92:0/3271777920 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.155+0000 7f4c627fc640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01844d0 0x5582e0556ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e0556620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c637fe640 1 -- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 msgr2=0x5582e0556620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e0556620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c637fe640 1 -- 172.21.15.92:0/3271777920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01844d0 msgr2=0x5582e0556ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01844d0 0x5582e0556ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c637fe640 1 -- 172.21.15.92:0/3271777920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e0210790 con 0x5582e01a51b0 2024-09-21T02:06:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e0556620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c637fe640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e0557850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c003f40 tx=0x7f4c5c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0c22e63d617a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c42ffd640 1 -- 172.21.15.92:0/3271777920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c028060 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c42ffd640 1 -- 172.21.15.92:0/3271777920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4c5c0116d0 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c015070 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c42ffd640 1 -- 172.21.15.92:0/3271777920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c0118d0 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c70872740 10 client.15957 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.156+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582e0289620 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c42ffd640 1 -- 172.21.15.92:0/3271777920 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c015070 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c42ffd640 1 -- 172.21.15.92:0/3271777920 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c5c02e490 con 0x5582e01a51b0 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c70872740 10 client.15957 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c70872740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c70872740 20 client.15957 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c70872740 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e0163d80 0x5582e0563a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c70872740 4 client.15957 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582e04c1640 con 0x5582e0163d80 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.157+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e0163d80 0x5582e0563a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.158+0000 7f4c42ffd640 10 client.15957 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.158+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e0163d80 0x5582e0563a90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70aa32b3a94e3f6 server_cookie=76ea476ec248354f in_seq=0 out_seq=0 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c42ffd640 1 -- 172.21.15.92:0/3271777920 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582e04c1640 con 0x5582e0163d80 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c42ffd640 10 client.15957 handle_command_reply: tid=0 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 client.15957 shutdown 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 msgr2=0x5582e0557850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e0557850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c003f40 tx=0x7f4c5c003f70 comp rx=0 tx=0).stop 2024-09-21T02:06:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 shutdown_connections 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e0163d80 0x5582e0563a90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e0557850 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e01844d0 0x5582e0556ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 --2- 172.21.15.92:0/3271777920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e017c2f0 0x5582e0556620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c42ffd640 0 client.15957 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.159+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 >> 172.21.15.92:0/3271777920 conn(0x5582e0199300 msgr2=0x5582e0546ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.160+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 shutdown_connections 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.160+0000 7f4c70872740 1 -- 172.21.15.92:0/3271777920 wait complete. 2024-09-21T02:06:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.160+0000 7f4c70872740 20 client.15957 trim_cache size 0 max 16384 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 Processor -- start 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 -- start start 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e054de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e0555050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01a51b0 0x5582e0548220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641ad1f0 con 0x5582e01844d0 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641aed30 con 0x5582e017c2f0 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.174+0000 7f4c70872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641ad8c0 con 0x5582e01a51b0 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e0555050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e0555050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46476/0 (socket says 172.21.15.92:46476) 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 -- 172.21.15.92:0/2368585115 learned_addr learned my addr 172.21.15.92:0/2368585115 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c637fe640 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01a51b0 0x5582e0548220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 -- 172.21.15.92:0/2368585115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01a51b0 msgr2=0x5582e0548220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e054de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01a51b0 0x5582e0548220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 -- 172.21.15.92:0/2368585115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 msgr2=0x5582e054de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e054de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 -- 172.21.15.92:0/2368585115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e0549260 con 0x5582e01844d0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c637fe640 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01a51b0 0x5582e0548220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c627fc640 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e0555050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4c58105400 tx=0x7f4c5802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b14420851d1b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c497fa640 1 -- 172.21.15.92:0/2368585115 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58049070 con 0x5582e01844d0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c497fa640 1 -- 172.21.15.92:0/2368585115 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c580cd650 con 0x5582e01844d0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.175+0000 7f4c497fa640 1 -- 172.21.15.92:0/2368585115 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5810d080 con 0x5582e01844d0 2024-09-21T02:06:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 -- 172.21.15.92:0/2368585115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 msgr2=0x5582e0555050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e0555050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4c58105400 tx=0x7f4c5802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 -- 172.21.15.92:0/2368585115 shutdown_connections 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01a51b0 0x5582e0548220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01844d0 0x5582e0555050 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 --2- 172.21.15.92:0/2368585115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e054de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 -- 172.21.15.92:0/2368585115 >> 172.21.15.92:0/2368585115 conn(0x5582e0199300 msgr2=0x5582e0546ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 -- 172.21.15.92:0/2368585115 shutdown_connections 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 -- 172.21.15.92:0/2368585115 wait complete. 2024-09-21T02:06:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.176+0000 7f4c70872740 1 -- start start 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 Processor -- start 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e0559ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01844d0 0x5582e055a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e055dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641ae660 con 0x5582e01a51b0 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641adf90 con 0x5582e017c2f0 2024-09-21T02:06:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641ab1d0 con 0x5582e01844d0 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c70872740 10 client.0 _ll_register_callbacks cb 0x7f4c5c004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e055dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e055dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46490/0 (socket says 172.21.15.92:46490) 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 -- 172.21.15.92:0/1453196982 learned_addr learned my addr 172.21.15.92:0/1453196982 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e0559ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c627fc640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01844d0 0x5582e055a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 -- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01844d0 msgr2=0x5582e055a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01844d0 0x5582e055a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 -- 172.21.15.92:0/1453196982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 msgr2=0x5582e0559ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e0559ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 -- 172.21.15.92:0/1453196982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e055e410 con 0x5582e01a51b0 2024-09-21T02:06:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e0559ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c627fc640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01844d0 0x5582e055a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c637fe640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e055dcb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727b27df9d02a2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.177+0000 7f4c42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c42ffd640 1 -- 172.21.15.92:0/1453196982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c028070 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c42ffd640 1 -- 172.21.15.92:0/1453196982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4c5c0116d0 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c580cd650 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c42ffd640 1 -- 172.21.15.92:0/1453196982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c02e400 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 10 client.15963 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582e055b750 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c42ffd640 1 -- 172.21.15.92:0/1453196982 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c580cd650 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c42ffd640 1 -- 172.21.15.92:0/1453196982 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c5c004640 con 0x5582e01a51b0 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 20 client.15963 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e01391c0 0x5582e05d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 4 client.15963 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:06:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.178+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582e04bd0f0 con 0x5582e01391c0 2024-09-21T02:06:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.179+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e01391c0 0x5582e05d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.179+0000 7f4c62ffd640 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e01391c0 0x5582e05d0910 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427893ffea97d122 server_cookie=f093aa4a9ad721f1 in_seq=0 out_seq=0 2024-09-21T02:06:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.179+0000 7f4c42ffd640 10 client.15963 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c42ffd640 1 -- 172.21.15.92:0/1453196982 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582e04bd0f0 con 0x5582e01391c0 2024-09-21T02:06:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c42ffd640 10 client.15963 handle_command_reply: tid=0 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 client.15963 shutdown 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 msgr2=0x5582e055dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e055dcb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c0039c0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 shutdown_connections 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5582e01391c0 0x5582e05d0910 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582e01a51b0 0x5582e055dcb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582e01844d0 0x5582e055a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 --2- 172.21.15.92:0/1453196982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582e017c2f0 0x5582e0559ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c42ffd640 0 client.15963 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.192+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 >> 172.21.15.92:0/1453196982 conn(0x5582e0199300 msgr2=0x5582e0555820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 shutdown_connections 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c70872740 1 -- 172.21.15.92:0/1453196982 wait complete. 2024-09-21T02:06:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c70872740 20 client.15963 trim_cache size 0 max 16384 2024-09-21T02:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:06:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c4c0b6510 msgr2=0x7f4c4c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c4c0b6510 0x7f4c4c0b8a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005fd0 tx=0x7f4c54009680 comp rx=0 tx=0).stop 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 msgr2=0x7f4c6419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6419c7c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006010 tx=0x7f4c58004340 comp rx=0 tx=0).stop 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 shutdown_connections 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4c4c0b6510 0x7f4c4c0b8a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c6410b600 0x7f4c6419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c64108ca0 0x7f4c6419c7c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 --2- 172.21.15.92:0/3644819328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c6406c2a0 0x7f4c6419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 >> 172.21.15.92:0/3644819328 conn(0x7f4c640fdf00 msgr2=0x7f4c64109b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 shutdown_connections 2024-09-21T02:06:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:27.193+0000 7f4c69907640 1 -- 172.21.15.92:0/3644819328 wait complete. 2024-09-21T02:06:28.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:28 smithi136 ceph-mon[34754]: pgmap v405: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:28.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:28 smithi092 ceph-mon[31190]: pgmap v405: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:28 smithi099 ceph-mon[34404]: pgmap v405: 97 pgs: 97 active+clean; 1.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:30.785 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:30 smithi136 ceph-mon[34754]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:06:31.010 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:30 smithi099 ceph-mon[34404]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:06:31.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:30 smithi092 ceph-mon[31190]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:06:32.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:06:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:32 smithi136 ceph-mon[34754]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:06:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:32 smithi092 ceph-mon[31190]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:06:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:32 smithi099 ceph-mon[34404]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 -- 172.21.15.92:0/3245545139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf810de10 msgr2=0x7f5cf8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/3245545139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf810de10 0x7f5cf8110300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009900 tx=0x7f5cf002a410 comp rx=0 tx=0).stop 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 -- 172.21.15.92:0/3245545139 shutdown_connections 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/3245545139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf810de10 0x7f5cf8110300 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/3245545139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 0x7f5cf810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/3245545139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf81049b0 0x7f5cf8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 -- 172.21.15.92:0/3245545139 >> 172.21.15.92:0/3245545139 conn(0x7f5cf8079730 msgr2=0x7f5cf8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.227+0000 7f5cfeb33640 1 -- 172.21.15.92:0/3245545139 shutdown_connections 2024-09-21T02:06:33.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.228+0000 7f5cfeb33640 1 -- 172.21.15.92:0/3245545139 wait complete. 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.228+0000 7f5cfeb33640 1 Processor -- start 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.228+0000 7f5cfeb33640 1 -- start start 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfeb33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 0x7f5cf819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfeb33640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 0x7f5cf819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfeb33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf810de10 0x7f5cf819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfeb33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8113a90 con 0x7f5cf81049b0 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfeb33640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8113910 con 0x7f5cf810de10 2024-09-21T02:06:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfeb33640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8113c10 con 0x7f5cf8105300 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 0x7f5cf819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 0x7f5cf819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43130/0 (socket says 172.21.15.92:43130) 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 -- 172.21.15.92:0/1973509491 learned_addr learned my addr 172.21.15.92:0/1973509491 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 0x7f5cf819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf810de10 0x7f5cf819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 -- 172.21.15.92:0/1973509491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 msgr2=0x7f5cf819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 0x7f5cf819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 -- 172.21.15.92:0/1973509491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf810de10 msgr2=0x7f5cf819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf810de10 0x7f5cf819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 -- 172.21.15.92:0/1973509491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a0510 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 0x7f5cf819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.229+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 0x7f5cf819a5a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800ea80 tx=0x7f5ce800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.230+0000 7f5cf5ffb640 1 -- 172.21.15.92:0/1973509491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800ce40 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.230+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a0800 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.230+0000 7f5cf5ffb640 1 -- 172.21.15.92:0/1973509491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce8011920 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.230+0000 7f5cf5ffb640 1 -- 172.21.15.92:0/1973509491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce801aaa0 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.230+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a6f60 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.232+0000 7f5cf5ffb640 1 -- 172.21.15.92:0/1973509491 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5ce801ac40 con 0x7f5cf81049b0 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.232+0000 7f5d050a7740 1 Processor -- start 2024-09-21T02:06:33.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 -- start start 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765ea1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765ea1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765ea20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c765e17e50 con 0x55c765ad9390 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c765e07280 con 0x55c765afa070 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5d050a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c765e0b090 con 0x55c765ad11b0 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765ea20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765ea20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43138/0 (socket says 172.21.15.92:43138) 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3706646604 learned_addr learned my addr 172.21.15.92:0/3706646604 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765ea1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765ea1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf5ffb640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5ce40b6510 0x7f5ce40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:06:33.273 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:33.274 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:06:33.275 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:33.276 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:06:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3706646604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 msgr2=0x55c765ea1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765ea1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3706646604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 msgr2=0x55c765ea1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765ea1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3706646604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c765b65650 con 0x55c765ad9390 2024-09-21T02:06:33.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.233+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5ce40b6510 0x7f5ce40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.234+0000 7f5cf5ffb640 1 -- 172.21.15.92:0/1973509491 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5ce80e79b0 con 0x7f5cf81049b0 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.234+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765ea20d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000cb90 tx=0x7f5ce000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826887c92fb56c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.234+0000 7f5ceeffd640 1 -- 172.21.15.92:0/3706646604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0005220 con 0x55c765ad9390 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.234+0000 7f5ceeffd640 1 -- 172.21.15.92:0/3706646604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce0004c50 con 0x55c765ad9390 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.234+0000 7f5ceeffd640 1 -- 172.21.15.92:0/3706646604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0010790 con 0x55c765ad9390 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 -- 172.21.15.92:0/3706646604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 msgr2=0x55c765ea20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5ce40b6510 0x7f5ce40b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0031440 tx=0x7f5cf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765ea20d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000cb90 tx=0x7f5ce000e720 comp rx=0 tx=0).stop 2024-09-21T02:06:33.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 -- 172.21.15.92:0/3706646604 shutdown_connections 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765ea20d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765ea1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3706646604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765ea1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 -- 172.21.15.92:0/3706646604 >> 172.21.15.92:0/3706646604 conn(0x55c765aee460 msgr2=0x55c765e9f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 -- 172.21.15.92:0/3706646604 shutdown_connections 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 -- 172.21.15.92:0/3706646604 wait complete. 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 -- start start 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.235+0000 7f5d050a7740 1 Processor -- start 2024-09-21T02:06:33.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c765c68700 con 0x55c765ad9390 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c765da2110 con 0x55c765afa070 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c765eb4190 con 0x55c765ad11b0 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5d050a7740 10 client.0 _ll_register_callbacks cb 0x55c765dbb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43144/0 (socket says 172.21.15.92:43144) 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 -- 172.21.15.92:0/1174476272 learned_addr learned my addr 172.21.15.92:0/1174476272 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 -- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 msgr2=0x55c765eb1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 -- 172.21.15.92:0/1174476272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 msgr2=0x55c765eb1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cf7fff640 1 -- 172.21.15.92:0/1174476272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c765dcfc40 con 0x55c765ad9390 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.236+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb1550 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005080 tx=0x7f5ce00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea71644bcb1cdfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cd7fff640 1 -- 172.21.15.92:0/1174476272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0029060 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cd7fff640 1 -- 172.21.15.92:0/1174476272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5ce0010720 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce0004c50 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cd7fff640 1 -- 172.21.15.92:0/1174476272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ce000ce90 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 10 client.15975 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c765ebfdd0 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cd7fff640 1 -- 172.21.15.92:0/1174476272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce0004c50 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5cd7fff640 1 -- 172.21.15.92:0/1174476272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5ce001a4a0 con 0x55c765ad9390 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 10 client.15975 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 20 client.15975 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:33.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765ab89f0 0x55c765ec31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 4 client.15975 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.237+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c765e19da0 con 0x55c765ab89f0 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.238+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765ab89f0 0x55c765ec31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.238+0000 7f5cd7fff640 10 client.15975 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.238+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765ab89f0 0x55c765ec31e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f30716b5e0af7e08 server_cookie=d891158257f56ddb in_seq=0 out_seq=0 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5cd7fff640 1 -- 172.21.15.92:0/1174476272 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c765e19da0 con 0x55c765ab89f0 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5cd7fff640 10 client.15975 handle_command_reply: tid=0 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 client.15975 shutdown 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 msgr2=0x55c765eb1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb1550 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005080 tx=0x7f5ce00050b0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 shutdown_connections 2024-09-21T02:06:33.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765ab89f0 0x55c765ec31e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb1550 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5cd7fff640 0 client.15975 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 --2- 172.21.15.92:0/1174476272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 >> 172.21.15.92:0/1174476272 conn(0x55c765aee460 msgr2=0x55c765e9db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 shutdown_connections 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 1 -- 172.21.15.92:0/1174476272 wait complete. 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.239+0000 7f5d050a7740 20 client.15975 trim_cache size 0 max 16384 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 Processor -- start 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 -- start start 2024-09-21T02:06:33.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765ad11b0 0x55c765ebd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765afa070 0x55c765eae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ad300 con 0x55c765ad9390 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81aee40 con 0x55c765ad11b0 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.256+0000 7f5d050a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ad9d0 con 0x55c765afa070 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43156/0 (socket says 172.21.15.92:43156) 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3443109800 learned_addr learned my addr 172.21.15.92:0/3443109800 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765ad11b0 0x55c765ebd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765afa070 0x55c765eae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3443109800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765afa070 msgr2=0x55c765eae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765afa070 0x55c765eae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3443109800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765ad11b0 msgr2=0x55c765ebd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765ad11b0 0x55c765ebd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3443109800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c765ec3640 con 0x55c765ad9390 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb2c40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005940 tx=0x7f5ce000c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10537da6f16f19ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cee7fc640 1 -- 172.21.15.92:0/3443109800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0033070 con 0x55c765ad9390 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cee7fc640 1 -- 172.21.15.92:0/3443109800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce001e030 con 0x55c765ad9390 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.257+0000 7f5cee7fc640 1 -- 172.21.15.92:0/3443109800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce001a410 con 0x55c765ad9390 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 -- 172.21.15.92:0/3443109800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 msgr2=0x55c765eb2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb2c40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005940 tx=0x7f5ce000c3b0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 -- 172.21.15.92:0/3443109800 shutdown_connections 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765afa070 0x55c765eae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb2c40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3443109800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765ad11b0 0x55c765ebd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 -- 172.21.15.92:0/3443109800 >> 172.21.15.92:0/3443109800 conn(0x55c765aee460 msgr2=0x55c765eb2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 -- 172.21.15.92:0/3443109800 shutdown_connections 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 -- 172.21.15.92:0/3443109800 wait complete. 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.258+0000 7f5d050a7740 1 -- start start 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 Processor -- start 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ae770 con 0x55c765ad9390 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ae0a0 con 0x55c765afa070 2024-09-21T02:06:33.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ab2e0 con 0x55c765ad11b0 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5d050a7740 10 client.0 _ll_register_callbacks cb 0x7f5ce0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43160/0 (socket says 172.21.15.92:43160) 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3157268847 learned_addr learned my addr 172.21.15.92:0/3157268847 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cfd0d0640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 msgr2=0x55c765eb9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3157268847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 msgr2=0x55c765eb5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cf7fff640 1 -- 172.21.15.92:0/3157268847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c765eb5f90 con 0x55c765ad9390 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.259+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cf7fff640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb5200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000ec20 tx=0x7f5ce000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61baf127697256c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cd7fff640 1 -- 172.21.15.92:0/3157268847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce003c070 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cd7fff640 1 -- 172.21.15.92:0/3157268847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5ce001a410 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce001e030 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cd7fff640 1 -- 172.21.15.92:0/3157268847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ce0046070 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 10 client.15981 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c765f2dab0 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cd7fff640 1 -- 172.21.15.92:0/3157268847 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce001e030 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5cd7fff640 1 -- 172.21.15.92:0/3157268847 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5ce0010720 con 0x55c765ad9390 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 10 client.15981 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 20 client.15981 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765a8dda0 0x55c765f30160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 4 client.15981 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.260+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c765e15850 con 0x55c765a8dda0 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765a8dda0 0x55c765f30160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5cfc8cf640 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765a8dda0 0x55c765f30160 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988a4672da079e41 server_cookie=64f25dde34196bf6 in_seq=0 out_seq=0 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5cd7fff640 10 client.15981 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5cd7fff640 1 -- 172.21.15.92:0/3157268847 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55c765e15850 con 0x55c765a8dda0 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5cd7fff640 10 client.15981 handle_command_reply: tid=0 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5d050a7740 1 client.15981 shutdown 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 msgr2=0x55c765eb5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.261+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb5200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000ec20 tx=0x7f5ce000c760 comp rx=0 tx=0).stop 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 shutdown_connections 2024-09-21T02:06:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c765a8dda0 0x55c765f30160 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c765afa070 0x55c765eb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c765ad9390 0x55c765eb5200 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5cd7fff640 0 client.15981 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 --2- 172.21.15.92:0/3157268847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c765ad11b0 0x55c765eb9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 >> 172.21.15.92:0/3157268847 conn(0x55c765aee460 msgr2=0x55c765eb23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 shutdown_connections 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 1 -- 172.21.15.92:0/3157268847 wait complete. 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5d050a7740 20 client.15981 trim_cache size 0 max 16384 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5ce40b6510 msgr2=0x7f5ce40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5ce40b6510 0x7f5ce40b8a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0031440 tx=0x7f5cf0009260 comp rx=0 tx=0).stop 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 msgr2=0x7f5cf819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.262+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 0x7f5cf819a5a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800ea80 tx=0x7f5ce800ef40 comp rx=0 tx=0).stop 2024-09-21T02:06:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 shutdown_connections 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5ce40b6510 0x7f5ce40b8a20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf810de10 0x7f5cf819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cf8105300 0x7f5cf819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 --2- 172.21.15.92:0/1973509491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf81049b0 0x7f5cf819a5a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 >> 172.21.15.92:0/1973509491 conn(0x7f5cf8079730 msgr2=0x7f5cf80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 shutdown_connections 2024-09-21T02:06:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:33.263+0000 7f5cfeb33640 1 -- 172.21.15.92:0/1973509491 wait complete. 2024-09-21T02:06:34.018 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:06:34.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:06:34.272 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:34.660 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:34 smithi092 ceph-mon[31190]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 -- 172.21.15.92:0/2196159861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8102a60 msgr2=0x7f30c8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 --2- 172.21.15.92:0/2196159861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8102a60 0x7f30c8102e70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 -- 172.21.15.92:0/2196159861 shutdown_connections 2024-09-21T02:06:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 --2- 172.21.15.92:0/2196159861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30c8108cf0 0x7f30c810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 --2- 172.21.15.92:0/2196159861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c81083a0 0x7f30c81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 --2- 172.21.15.92:0/2196159861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8102a60 0x7f30c8102e70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 -- 172.21.15.92:0/2196159861 >> 172.21.15.92:0/2196159861 conn(0x7f30c8079730 msgr2=0x7f30c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 -- 172.21.15.92:0/2196159861 shutdown_connections 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.750+0000 7f30ce923640 1 -- 172.21.15.92:0/2196159861 wait complete. 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 Processor -- start 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 -- start start 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 0x7f30c8105cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30c81083a0 0x7f30c81042b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 0x7f30c8104850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8112960 con 0x7f30c8108cf0 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81127e0 con 0x7f30c8102a60 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.751+0000 7f30ce923640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8112ae0 con 0x7f30c81083a0 2024-09-21T02:06:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 0x7f30c8104850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 0x7f30c8105cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 0x7f30c8105cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:55942/0 (socket says 172.21.15.92:55942) 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 0x7f30c8104850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43180/0 (socket says 172.21.15.92:43180) 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c7fff640 1 -- 172.21.15.92:0/1732270188 learned_addr learned my addr 172.21.15.92:0/1732270188 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c77fe640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30c81083a0 0x7f30c81042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 -- 172.21.15.92:0/1732270188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30c81083a0 msgr2=0x7f30c81042b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30c81083a0 0x7f30c81042b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 -- 172.21.15.92:0/1732270188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 msgr2=0x7f30c8105cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 0x7f30c8105cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 -- 172.21.15.92:0/1732270188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c8105070 con 0x7f30c8108cf0 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c7fff640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 0x7f30c8105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ccec0640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 0x7f30c8104850 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f30c000ca30 tx=0x7f30c000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0007d90 con 0x7f30c8108cf0 2024-09-21T02:06:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c81ace00 con 0x7f30c8108cf0 2024-09-21T02:06:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c0005020 con 0x7f30c8108cf0 2024-09-21T02:06:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.752+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c81ad2b0 con 0x7f30c8108cf0 2024-09-21T02:06:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.753+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c000f690 con 0x7f30c8108cf0 2024-09-21T02:06:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.754+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b40052c0 con 0x7f30c8108cf0 2024-09-21T02:06:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.755+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f30c0004a40 con 0x7f30c8108cf0 2024-09-21T02:06:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.756+0000 7f30c57fa640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30b00b6590 0x7f30b00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.756+0000 7f30c7fff640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30b00b6590 0x7f30b00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.756+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f30c00e7650 con 0x7f30c8108cf0 2024-09-21T02:06:34.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.757+0000 7f30c7fff640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30b00b6590 0x7f30b00b8aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009230 tx=0x7f30b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:34.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.759+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30c0014070 con 0x7f30c8108cf0 2024-09-21T02:06:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:34 smithi136 ceph-mon[34754]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:34 smithi099 ceph-mon[34404]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:34.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.973+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30b40055b0 con 0x7f30c8108cf0 2024-09-21T02:06:34.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.974+0000 7f30c57fa640 1 -- 172.21.15.92:0/1732270188 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f30c00b2320 con 0x7f30c8108cf0 2024-09-21T02:06:34.976 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:06:34.976 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:06:34.976 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30b00b6590 msgr2=0x7f30b00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30b00b6590 0x7f30b00b8aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009230 tx=0x7f30b80058d0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 msgr2=0x7f30c8104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 0x7f30c8104850 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f30c000ca30 tx=0x7f30c000cef0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 shutdown_connections 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f30b00b6590 0x7f30b00b8aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c8108cf0 0x7f30c8104850 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30c81083a0 0x7f30c81042b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 --2- 172.21.15.92:0/1732270188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30c8102a60 0x7f30c8105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 >> 172.21.15.92:0/1732270188 conn(0x7f30c8079730 msgr2=0x7f30c81019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:34.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 shutdown_connections 2024-09-21T02:06:34.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:34.976+0000 7f30ce923640 1 -- 172.21.15.92:0/1732270188 wait complete. 2024-09-21T02:06:35.149 INFO:tasks.workunit.client.0.smithi092.stdout:1/600: open fd = 3 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:2/600: open fd = 3 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:3/600: open fd = 3 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.150 INFO:tasks.workunit.client.0.smithi092.stdout:4/600: open fd = 3 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:5/600: open fd = 3 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:6/600: open fd = 3 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:7/600: open fd = 3 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.151 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:8/600: open fd = 3 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:9/600: open fd = 3 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:10/600: open fd = 3 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:11/600: open fd = 3 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.152 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:12/600: open fd = 3 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:13/600: open fd = 3 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:14/600: open fd = 3 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:15/600: open fd = 3 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.153 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:16/600: open fd = 3 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:17/600: open fd = 3 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:18/600: open fd = 3 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.154 INFO:tasks.workunit.client.0.smithi092.stdout:19/600: open fd = 3 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:20/600: open fd = 3 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:21/600: open fd = 3 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:22/600: open fd = 3 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.155 INFO:tasks.workunit.client.0.smithi092.stdout:23/600: open fd = 3 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:24/600: open fd = 3 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:25/600: open fd = 3 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:26/600: open fd = 3 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.156 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:27/600: open fd = 3 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:28/600: open fd = 3 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:29/600: open fd = 3 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:30/600: open fd = 3 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.157 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:31/600: open fd = 3 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:32/600: open fd = 3 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:33/600: open fd = 3 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:34/600: open fd = 3 2024-09-21T02:06:35.158 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:35/600: open fd = 3 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:36/600: open fd = 3 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:37/600: open fd = 3 2024-09-21T02:06:35.159 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:38/600: open fd = 3 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:39/600: open fd = 3 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:40/600: open fd = 3 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:41/600: open fd = 3 2024-09-21T02:06:35.160 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:42/600: open fd = 3 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:43/600: open fd = 3 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:44/600: open fd = 3 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.161 INFO:tasks.workunit.client.0.smithi092.stdout:45/600: open fd = 3 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:46/600: open fd = 3 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:47/600: open fd = 3 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:48/600: open fd = 3 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.162 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:49/600: open fd = 3 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:50/600: open fd = 3 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:51/600: open fd = 3 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.163 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:52/600: open fd = 3 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:53/600: open fd = 3 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:54/600: open fd = 3 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:55/600: open fd = 3 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.164 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:56/600: open fd = 3 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:57/600: open fd = 3 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:58/600: open fd = 3 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:59/600: open fd = 3 2024-09-21T02:06:35.165 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:60/600: open fd = 3 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:61/600: open fd = 3 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:62/600: open fd = 3 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.166 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.167 INFO:tasks.workunit.client.0.smithi092.stdout:63/600: open fd = 3 2024-09-21T02:06:35.167 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.167 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:06:35.167 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:06:35.630 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:35 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1732270188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:06:35.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:06:35.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:35 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1732270188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:06:35.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:35 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1732270188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:06:35.908 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 -- 172.21.15.92:0/319126883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd828103b80 msgr2=0x7fd828103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 --2- 172.21.15.92:0/319126883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd828103b80 0x7fd828103f90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd818009900 tx=0x7fd81802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:36.393 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 -- 172.21.15.92:0/319126883 shutdown_connections 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 --2- 172.21.15.92:0/319126883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd828109e10 0x7fd828110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 --2- 172.21.15.92:0/319126883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8281094c0 0x7fd8281098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 --2- 172.21.15.92:0/319126883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd828103b80 0x7fd828103f90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 -- 172.21.15.92:0/319126883 >> 172.21.15.92:0/319126883 conn(0x7fd8280ff5d0 msgr2=0x7fd828101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 -- 172.21.15.92:0/319126883 shutdown_connections 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 -- 172.21.15.92:0/319126883 wait complete. 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.384+0000 7fd82de66640 1 Processor -- start 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 -- start start 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd828103b80 0x7fd82819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 0x7fd82819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd828109e10 0x7fd82819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd828113ad0 con 0x7fd8281094c0 2024-09-21T02:06:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd828113950 con 0x7fd828103b80 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd82de66640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd828113c50 con 0x7fd828109e10 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd826ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 0x7fd82819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd826ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 0x7fd82819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43194/0 (socket says 172.21.15.92:43194) 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd826ffd640 1 -- 172.21.15.92:0/3065409397 learned_addr learned my addr 172.21.15.92:0/3065409397 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd828103b80 0x7fd82819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd827fff640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd828109e10 0x7fd82819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd826ffd640 1 -- 172.21.15.92:0/3065409397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd828109e10 msgr2=0x7fd82819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd828109e10 0x7fd82819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.385+0000 7fd826ffd640 1 -- 172.21.15.92:0/3065409397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd828103b80 msgr2=0x7fd82819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd828103b80 0x7fd82819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd826ffd640 1 -- 172.21.15.92:0/3065409397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8281a0430 con 0x7fd8281094c0 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 0x7fd82819aa60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d950 tx=0x7fd81c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd824ff9640 1 -- 172.21.15.92:0/3065409397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c017070 con 0x7fd8281094c0 2024-09-21T02:06:36.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8281a0720 con 0x7fd8281094c0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd824ff9640 1 -- 172.21.15.92:0/3065409397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd81c009c30 con 0x7fd8281094c0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd824ff9640 1 -- 172.21.15.92:0/3065409397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0047a0 con 0x7fd8281094c0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.386+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8281a6e80 con 0x7fd8281094c0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 Processor -- start 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd824ff9640 1 -- 172.21.15.92:0/3065409397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd81c004020 con 0x7fd8281094c0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 -- start start 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290527be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55629015f390 0x556290528120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55629049dea0 con 0x556290180070 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55629048d2d0 con 0x55629015f390 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8343da740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562904910e0 con 0x5562901571b0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905276a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43200/0 (socket says 172.21.15.92:43200) 2024-09-21T02:06:36.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 -- 172.21.15.92:0/1506423217 learned_addr learned my addr 172.21.15.92:0/1506423217 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd826ffd640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55629015f390 0x556290528120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd8277fe640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290527be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 -- 172.21.15.92:0/1506423217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 msgr2=0x556290527be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290527be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 -- 172.21.15.92:0/1506423217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55629015f390 msgr2=0x556290528120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55629015f390 0x556290528120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.389+0000 7fd827fff640 1 -- 172.21.15.92:0/1506423217 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562901eb650 con 0x556290180070 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd826ffd640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55629015f390 0x556290528120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd8277fe640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290527be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd824ff9640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd8100b6510 0x7fd8100b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd8100b6510 0x7fd8100b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd827fff640 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905276a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd82000ae10 tx=0x7fd82000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b10990a711e529a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd824ff9640 1 -- 172.21.15.92:0/3065409397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd81c0e6f30 con 0x7fd8281094c0 2024-09-21T02:06:36.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd80dffb640 1 -- 172.21.15.92:0/1506423217 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820003b70 con 0x556290180070 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd80dffb640 1 -- 172.21.15.92:0/1506423217 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd820003d10 con 0x556290180070 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.390+0000 7fd80dffb640 1 -- 172.21.15.92:0/1506423217 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200109c0 con 0x556290180070 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 -- 172.21.15.92:0/1506423217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 msgr2=0x5562905276a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd8100b6510 0x7fd8100b8a20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd818031440 tx=0x7fd818009640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905276a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd82000ae10 tx=0x7fd82000f790 comp rx=0 tx=0).stop 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 -- 172.21.15.92:0/1506423217 shutdown_connections 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55629015f390 0x556290528120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290527be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 --2- 172.21.15.92:0/1506423217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905276a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 -- 172.21.15.92:0/1506423217 >> 172.21.15.92:0/1506423217 conn(0x556290174460 msgr2=0x556290525590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 -- 172.21.15.92:0/1506423217 shutdown_connections 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.391+0000 7fd8343da740 1 -- 172.21.15.92:0/1506423217 wait complete. 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 -- start start 2024-09-21T02:06:36.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 Processor -- start 2024-09-21T02:06:36.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290532d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 0x5562905332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290180070 0x55629053f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5562902ee700 con 0x55629015f390 2024-09-21T02:06:36.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556290428160 con 0x556290180070 2024-09-21T02:06:36.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556290535e80 con 0x5562901571b0 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd8343da740 10 client.0 _ll_register_callbacks cb 0x5562904419e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd826ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 0x5562905332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd826ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 0x5562905332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43216/0 (socket says 172.21.15.92:43216) 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.392+0000 7fd826ffd640 1 -- 172.21.15.92:0/3671620496 learned_addr learned my addr 172.21.15.92:0/3671620496 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290532d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd826ffd640 1 -- 172.21.15.92:0/3671620496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 msgr2=0x556290532d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290532d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd827fff640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290180070 0x55629053f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd826ffd640 1 -- 172.21.15.92:0/3671620496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290180070 msgr2=0x55629053f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290180070 0x55629053f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd826ffd640 1 -- 172.21.15.92:0/3671620496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556290455c90 con 0x55629015f390 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290532d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 0x5562905332b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0acda0 tx=0x7fd81c0a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b62b095cb1bc07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:36.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd806ffd640 1 -- 172.21.15.92:0/3671620496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0ac9f0 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd806ffd640 1 -- 172.21.15.92:0/3671620496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd81c0a8bf0 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd820003d10 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd806ffd640 1 -- 172.21.15.92:0/3671620496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd81c0ac810 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd8343da740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556290541c00 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.393+0000 7fd806ffd640 1 -- 172.21.15.92:0/3671620496 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd820003d10 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd806ffd640 1 -- 172.21.15.92:0/3671620496 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd81c0981a0 con 0x55629015f390 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8343da740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8343da740 10 client.15999 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8343da740 20 client.15999 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8343da740 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55629013e9f0 0x556290545130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8343da740 4 client.15999 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55629049fdf0 con 0x55629013e9f0 2024-09-21T02:06:36.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55629013e9f0 0x556290545130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd806ffd640 10 client.15999 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.394+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55629013e9f0 0x556290545130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffbf7d8e6a9e0f29 server_cookie=e9ea447dfec8087f in_seq=0 out_seq=0 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd806ffd640 1 -- 172.21.15.92:0/3671620496 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55629049fdf0 con 0x55629013e9f0 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd806ffd640 10 client.15999 handle_command_reply: tid=0 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 client.15999 shutdown 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 msgr2=0x5562905332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 0x5562905332b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0acda0 tx=0x7fd81c0a97b0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 shutdown_connections 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55629013e9f0 0x556290545130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd806ffd640 0 client.15999 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290180070 0x55629053f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55629015f390 0x5562905332b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 --2- 172.21.15.92:0/3671620496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x556290532d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 >> 172.21.15.92:0/3671620496 conn(0x556290174460 msgr2=0x55629052a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:36.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 shutdown_connections 2024-09-21T02:06:36.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.395+0000 7fd8343da740 1 -- 172.21.15.92:0/3671620496 wait complete. 2024-09-21T02:06:36.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.396+0000 7fd8343da740 20 client.15999 trim_cache size 0 max 16384 2024-09-21T02:06:36.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.408+0000 7fd8343da740 1 Processor -- start 2024-09-21T02:06:36.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.408+0000 7fd8343da740 1 -- start start 2024-09-21T02:06:36.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8343da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 0x556290529090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8343da740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556290180070 0x55629053f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8343da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8343da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281af5c0 con 0x5562901571b0 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8343da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281b1100 con 0x556290113da0 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8343da740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281afc90 con 0x556290180070 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 0x556290529090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 0x556290529090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43232/0 (socket says 172.21.15.92:43232) 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 -- 172.21.15.92:0/3320673646 learned_addr learned my addr 172.21.15.92:0/3320673646 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd827fff640 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd826ffd640 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556290180070 0x55629053f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 -- 172.21.15.92:0/3320673646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556290180070 msgr2=0x55629053f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556290180070 0x55629053f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 -- 172.21.15.92:0/3320673646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 msgr2=0x5562905347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 -- 172.21.15.92:0/3320673646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562905209c0 con 0x5562901571b0 2024-09-21T02:06:36.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd827fff640 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:36.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd8277fe640 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 0x556290529090 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd818031750 tx=0x7fd81803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b801b92f12455e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:36.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.409+0000 7fd80d7fa640 1 -- 172.21.15.92:0/3320673646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818035040 con 0x5562901571b0 2024-09-21T02:06:36.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd80d7fa640 1 -- 172.21.15.92:0/3320673646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd818005030 con 0x5562901571b0 2024-09-21T02:06:36.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd80d7fa640 1 -- 172.21.15.92:0/3320673646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818004b20 con 0x5562901571b0 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 -- 172.21.15.92:0/3320673646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 msgr2=0x556290529090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 0x556290529090 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd818031750 tx=0x7fd81803e000 comp rx=0 tx=0).stop 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 -- 172.21.15.92:0/3320673646 shutdown_connections 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556290180070 0x55629053f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 --2- 172.21.15.92:0/3320673646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5562901571b0 0x556290529090 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 -- 172.21.15.92:0/3320673646 >> 172.21.15.92:0/3320673646 conn(0x556290174460 msgr2=0x5562905340d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 -- 172.21.15.92:0/3320673646 shutdown_connections 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.410+0000 7fd8343da740 1 -- 172.21.15.92:0/3320673646 wait complete. 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 -- start start 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 Processor -- start 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x5562905a8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281b0a30 con 0x556290180070 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281b0360 con 0x556290113da0 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.411+0000 7fd8343da740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281ad5a0 con 0x5562901571b0 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd8343da740 10 client.0 _ll_register_callbacks cb 0x7fd81c0056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905a9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43236/0 (socket says 172.21.15.92:43236) 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 -- 172.21.15.92:0/4082685220 learned_addr learned my addr 172.21.15.92:0/4082685220 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd826ffd640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x5562905a8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd8277fe640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 -- 172.21.15.92:0/4082685220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 msgr2=0x5562905a8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x5562905a8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 -- 172.21.15.92:0/4082685220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 msgr2=0x5562905a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:36.418 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 -- 172.21.15.92:0/4082685220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562905aa220 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd8277fe640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd827fff640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905a9bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd82000ade0 tx=0x7fd820004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f290ac40261d775 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd806ffd640 1 -- 172.21.15.92:0/4082685220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820028070 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd806ffd640 1 -- 172.21.15.92:0/4082685220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd820010910 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd818005030 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd806ffd640 1 -- 172.21.15.92:0/4082685220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd820034050 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.412+0000 7fd8343da740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5562905b4830 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd806ffd640 1 -- 172.21.15.92:0/4082685220 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd818005030 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd806ffd640 1 -- 172.21.15.92:0/4082685220 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd820005870 con 0x556290180070 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:36.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 10 client.16005 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 20 client.16005 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5562905b4ca0 0x5562905b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 4 client.16005 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55629049b8a0 con 0x5562905b4ca0 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd8277fe640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5562905b4ca0 0x5562905b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.413+0000 7fd806ffd640 10 client.16005 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8277fe640 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5562905b4ca0 0x5562905b70c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d0ee3bd64d922eb server_cookie=a85c49a3645eada6 in_seq=0 out_seq=0 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd806ffd640 1 -- 172.21.15.92:0/4082685220 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55629049b8a0 con 0x5562905b4ca0 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd806ffd640 10 client.16005 handle_command_reply: tid=0 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 client.16005 shutdown 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 msgr2=0x5562905a9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905a9bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd82000ade0 tx=0x7fd820004df0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 shutdown_connections 2024-09-21T02:06:36.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5562905b4ca0 0x5562905b70c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556290180070 0x5562905a9bb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd806ffd640 0 client.16005 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562901571b0 0x5562905a8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 --2- 172.21.15.92:0/4082685220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556290113da0 0x5562905a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 >> 172.21.15.92:0/4082685220 conn(0x556290174460 msgr2=0x556290528ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 shutdown_connections 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 1 -- 172.21.15.92:0/4082685220 wait complete. 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.414+0000 7fd8343da740 20 client.16005 trim_cache size 0 max 16384 2024-09-21T02:06:36.421 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd8100b6510 msgr2=0x7fd8100b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd8100b6510 0x7fd8100b8a20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd818031440 tx=0x7fd818009640 comp rx=0 tx=0).stop 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 msgr2=0x7fd82819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 0x7fd82819aa60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d950 tx=0x7fd81c00de10 comp rx=0 tx=0).stop 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 shutdown_connections 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd8100b6510 0x7fd8100b8a20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd828109e10 0x7fd82819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8281094c0 0x7fd82819aa60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 --2- 172.21.15.92:0/3065409397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd828103b80 0x7fd82819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 >> 172.21.15.92:0/3065409397 conn(0x7fd8280ff5d0 msgr2=0x7fd8281019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 shutdown_connections 2024-09-21T02:06:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:36.415+0000 7fd82de66640 1 -- 172.21.15.92:0/3065409397 wait complete. 2024-09-21T02:06:36.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:36 smithi092 ceph-mon[31190]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:06:36.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:36 smithi136 ceph-mon[34754]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:06:36.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:36 smithi099 ceph-mon[34404]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:06:37.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:06:37.470 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 -- 172.21.15.92:0/1080574082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 msgr2=0x7f347c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 --2- 172.21.15.92:0/1080574082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c1087c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3470009900 tx=0x7f347002a410 comp rx=0 tx=0).stop 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 -- 172.21.15.92:0/1080574082 shutdown_connections 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 --2- 172.21.15.92:0/1080574082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f347c108d00 0x7f347c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 --2- 172.21.15.92:0/1080574082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c1087c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 --2- 172.21.15.92:0/1080574082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f347c102ac0 0x7f347c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 -- 172.21.15.92:0/1080574082 >> 172.21.15.92:0/1080574082 conn(0x7f347c079720 msgr2=0x7f347c079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 -- 172.21.15.92:0/1080574082 shutdown_connections 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.954+0000 7f348172f640 1 -- 172.21.15.92:0/1080574082 wait complete. 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.955+0000 7f348172f640 1 Processor -- start 2024-09-21T02:06:37.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.955+0000 7f348172f640 1 -- start start 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f347c102ac0 0x7f347c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f347c108d00 0x7f347c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1129c0 con 0x7f347c1083b0 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f347c112840 con 0x7f347c108d00 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f347c112b40 con 0x7f347c102ac0 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43248/0 (socket says 172.21.15.92:43248) 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 -- 172.21.15.92:0/1766293766 learned_addr learned my addr 172.21.15.92:0/1766293766 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347affd640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f347c102ac0 0x7f347c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347b7fe640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f347c108d00 0x7f347c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 -- 172.21.15.92:0/1766293766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f347c102ac0 msgr2=0x7f347c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f347c102ac0 0x7f347c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 -- 172.21.15.92:0/1766293766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f347c108d00 msgr2=0x7f347c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f347c108d00 0x7f347c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 -- 172.21.15.92:0/1766293766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c1a04b0 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c19aae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f347002a930 tx=0x7f3470004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f3463fff640 1 -- 172.21.15.92:0/1766293766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347003f030 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f3463fff640 1 -- 172.21.15.92:0/1766293766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3470003aa0 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c1a0740 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.956+0000 7f3463fff640 1 -- 172.21.15.92:0/1766293766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470035070 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.957+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c1a6f00 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 Processor -- start 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 -- start start 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3463fff640 1 -- 172.21.15.92:0/1766293766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f3470042030 con 0x7f347c1083b0 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580028521b0 0x558002c22800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 0x558002c22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558002b98ea0 con 0x55800285a390 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558002b882d0 con 0x5580028521b0 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f3487ca3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558002b8c0e0 con 0x55800287b070 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f347a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 0x558002c22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f347b7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f347a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 0x558002c22d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43264/0 (socket says 172.21.15.92:43264) 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.959+0000 7f347a7fc640 1 -- 172.21.15.92:0/3891668980 learned_addr learned my addr 172.21.15.92:0/3891668980 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347affd640 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580028521b0 0x558002c22800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347a7fc640 1 -- 172.21.15.92:0/3891668980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 msgr2=0x558002c222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347a7fc640 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347a7fc640 1 -- 172.21.15.92:0/3891668980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580028521b0 msgr2=0x558002c22800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347a7fc640 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580028521b0 0x558002c22800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347a7fc640 1 -- 172.21.15.92:0/3891668980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580028e6650 con 0x55800285a390 2024-09-21T02:06:37.972 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347b7fe640 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347a7fc640 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 0x558002c22d40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f347010dc10 tx=0x7f347010dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1767b1aed42309f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f3463fff640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34640b6590 0x7f34640b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f34617fa640 1 -- 172.21.15.92:0/3891668980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470116070 con 0x55800285a390 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f34617fa640 1 -- 172.21.15.92:0/3891668980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34700cd600 con 0x55800285a390 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f34617fa640 1 -- 172.21.15.92:0/3891668980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700f4050 con 0x55800285a390 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f347b7fe640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34640b6590 0x7f34640b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.960+0000 7f3463fff640 1 -- 172.21.15.92:0/1766293766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f34701078d0 con 0x7f347c1083b0 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 -- 172.21.15.92:0/3891668980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 msgr2=0x558002c22d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f347b7fe640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34640b6590 0x7f34640b8aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f347400b890 tx=0x7f347400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 0x558002c22d40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f347010dc10 tx=0x7f347010dd00 comp rx=0 tx=0).stop 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 -- 172.21.15.92:0/3891668980 shutdown_connections 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55800285a390 0x558002c22d40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580028521b0 0x558002c22800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3891668980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 -- 172.21.15.92:0/3891668980 >> 172.21.15.92:0/3891668980 conn(0x55800286f460 msgr2=0x558002c201b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 -- 172.21.15.92:0/3891668980 shutdown_connections 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.961+0000 7f3487ca3740 1 -- 172.21.15.92:0/3891668980 wait complete. 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 -- start start 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 Processor -- start 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800285a390 0x558002c2dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800287b070 0x558002c2e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5580029e9700 con 0x5580028521b0 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558002b23160 con 0x55800287b070 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558002c30860 con 0x55800285a390 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f3487ca3740 10 client.0 _ll_register_callbacks cb 0x558002b3c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f347affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f347affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43274/0 (socket says 172.21.15.92:43274) 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f347affd640 1 -- 172.21.15.92:0/1434226527 learned_addr learned my addr 172.21.15.92:0/1434226527 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800285a390 0x558002c2dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.962+0000 7f347b7fe640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800287b070 0x558002c2e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347affd640 1 -- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800285a390 msgr2=0x558002c2dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347affd640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800285a390 0x558002c2dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347affd640 1 -- 172.21.15.92:0/1434226527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800287b070 msgr2=0x558002c2e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347affd640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800287b070 0x558002c2e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347affd640 1 -- 172.21.15.92:0/1434226527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558002b50c90 con 0x5580028521b0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800285a390 0x558002c2dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347b7fe640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800287b070 0x558002c2e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f347affd640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2d840 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f346c010b70 tx=0x7f346c010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb2bd2920785342 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f345eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f345eff5640 1 -- 172.21.15.92:0/1434226527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c00fd70 con 0x5580028521b0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f345eff5640 1 -- 172.21.15.92:0/1434226527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f346c004b20 con 0x5580028521b0 2024-09-21T02:06:37.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34700cd600 con 0x5580028521b0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f345eff5640 1 -- 172.21.15.92:0/1434226527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f346c004d20 con 0x5580028521b0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f3487ca3740 10 client.16017 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558002c3c400 con 0x5580028521b0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f345eff5640 1 -- 172.21.15.92:0/1434226527 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34700cd600 con 0x5580028521b0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.963+0000 7f345eff5640 1 -- 172.21.15.92:0/1434226527 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f346c019530 con 0x5580028521b0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f3487ca3740 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f3487ca3740 10 client.16017 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f3487ca3740 20 client.16017 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f3487ca3740 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5580028399f0 0x558002c3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f3487ca3740 4 client.16017 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558002b9adf0 con 0x5580028399f0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5580028399f0 0x558002c3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f345eff5640 10 client.16017 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.964+0000 7f347a7fc640 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5580028399f0 0x558002c3f980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40bfcf2cde77b66 server_cookie=34e927be9abcb223 in_seq=0 out_seq=0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f345eff5640 1 -- 172.21.15.92:0/1434226527 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558002b9adf0 con 0x5580028399f0 2024-09-21T02:06:37.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f345eff5640 10 client.16017 handle_command_reply: tid=0 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 client.16017 shutdown 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 msgr2=0x558002c2d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2d840 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f346c010b70 tx=0x7f346c010f70 comp rx=0 tx=0).stop 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 shutdown_connections 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5580028399f0 0x558002c3f980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800287b070 0x558002c2e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f345eff5640 0 client.16017 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800285a390 0x558002c2dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 --2- 172.21.15.92:0/1434226527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2d840 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 >> 172.21.15.92:0/1434226527 conn(0x55800286f460 msgr2=0x558002c25450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 shutdown_connections 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.965+0000 7f3487ca3740 1 -- 172.21.15.92:0/1434226527 wait complete. 2024-09-21T02:06:37.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.966+0000 7f3487ca3740 20 client.16017 trim_cache size 0 max 16384 2024-09-21T02:06:37.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 Processor -- start 2024-09-21T02:06:37.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 -- start start 2024-09-21T02:06:37.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c2aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c2af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1af4f0 con 0x5580028521b0 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1b1030 con 0x55800285a390 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f3487ca3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1afbc0 con 0x55800287b070 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43290/0 (socket says 172.21.15.92:43290) 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 -- 172.21.15.92:0/3223086472 learned_addr learned my addr 172.21.15.92:0/3223086472 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347b7fe640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c2af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347a7fc640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c2aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 -- 172.21.15.92:0/3223086472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 msgr2=0x558002c2af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c2af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 -- 172.21.15.92:0/3223086472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 msgr2=0x558002c2aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c2aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 -- 172.21.15.92:0/3223086472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558002c245b0 con 0x5580028521b0 2024-09-21T02:06:37.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347b7fe640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:37.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347a7fc640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c2aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:37.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.979+0000 7f347affd640 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2f470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f346c005cc0 tx=0x7f346c010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91df0cfcce21994e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:37.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.980+0000 7f3460ff9640 1 -- 172.21.15.92:0/3223086472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c028040 con 0x5580028521b0 2024-09-21T02:06:37.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.980+0000 7f3460ff9640 1 -- 172.21.15.92:0/3223086472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346c0049e0 con 0x5580028521b0 2024-09-21T02:06:37.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.980+0000 7f3460ff9640 1 -- 172.21.15.92:0/3223086472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c003e00 con 0x5580028521b0 2024-09-21T02:06:37.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.980+0000 7f3487ca3740 1 -- 172.21.15.92:0/3223086472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 msgr2=0x558002c2f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.980+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2f470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f346c005cc0 tx=0x7f346c010760 comp rx=0 tx=0).stop 2024-09-21T02:06:37.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.980+0000 7f3487ca3740 1 -- 172.21.15.92:0/3223086472 shutdown_connections 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002c2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c2aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 --2- 172.21.15.92:0/3223086472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c2f470 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- 172.21.15.92:0/3223086472 >> 172.21.15.92:0/3223086472 conn(0x55800286f460 msgr2=0x558002c2ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- 172.21.15.92:0/3223086472 shutdown_connections 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- 172.21.15.92:0/3223086472 wait complete. 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- start start 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 Processor -- start 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c31d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c32300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002ca9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1b0960 con 0x5580028521b0 2024-09-21T02:06:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1b0290 con 0x55800285a390 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1ad4d0 con 0x55800287b070 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f3487ca3740 10 client.0 _ll_register_callbacks cb 0x7f346c00aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f347affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f347affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c31d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43298/0 (socket says 172.21.15.92:43298) 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.981+0000 7f347affd640 1 -- 172.21.15.92:0/103783138 learned_addr learned my addr 172.21.15.92:0/103783138 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347a7fc640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c32300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347b7fe640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002ca9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347affd640 1 -- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 msgr2=0x558002ca9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347affd640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002ca9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347affd640 1 -- 172.21.15.92:0/103783138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 msgr2=0x558002c32300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347affd640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c32300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347affd640 1 -- 172.21.15.92:0/103783138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558002ca98c0 con 0x5580028521b0 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347a7fc640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c32300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347b7fe640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002ca9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f347affd640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c31d80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f346c01b040 tx=0x7f346c004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3482ffda370530c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f345eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f345eff5640 1 -- 172.21.15.92:0/103783138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c0193e0 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f345eff5640 1 -- 172.21.15.92:0/103783138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f346c019580 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f346c0049e0 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f345eff5640 1 -- 172.21.15.92:0/103783138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f346c02d480 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f3487ca3740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.982+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558002caa8f0 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f345eff5640 1 -- 172.21.15.92:0/103783138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346c0049e0 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f345eff5640 1 -- 172.21.15.92:0/103783138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f346c034020 con 0x5580028521b0 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f3487ca3740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f3487ca3740 10 client.16023 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f3487ca3740 20 client.16023 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f3487ca3740 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55800280eda0 0x558002cacf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f3487ca3740 4 client.16023 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558002b968a0 con 0x55800280eda0 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f347a7fc640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55800280eda0 0x558002cacf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f347a7fc640 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55800280eda0 0x558002cacf90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0e07a447f782e84 server_cookie=a236a5e85ab460a4 in_seq=0 out_seq=0 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.983+0000 7f345eff5640 10 client.16023 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f345eff5640 1 -- 172.21.15.92:0/103783138 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558002b968a0 con 0x55800280eda0 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f345eff5640 10 client.16023 handle_command_reply: tid=0 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 client.16023 shutdown 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 msgr2=0x558002c31d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c31d80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f346c01b040 tx=0x7f346c004c20 comp rx=0 tx=0).stop 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 shutdown_connections 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55800287b070 0x558002ca9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55800280eda0 0x558002cacf90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55800285a390 0x558002c32300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 --2- 172.21.15.92:0/103783138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5580028521b0 0x558002c31d80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 >> 172.21.15.92:0/103783138 conn(0x55800286f460 msgr2=0x558002c2ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:37.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f345eff5640 0 client.16023 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:06:37.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 shutdown_connections 2024-09-21T02:06:37.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 1 -- 172.21.15.92:0/103783138 wait complete. 2024-09-21T02:06:37.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.984+0000 7f3487ca3740 20 client.16023 trim_cache size 0 max 16384 2024-09-21T02:06:37.991 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34640b6590 msgr2=0x7f34640b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34640b6590 0x7f34640b8aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f347400b890 tx=0x7f347400b2e0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 msgr2=0x7f347c19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c19aae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f347002a930 tx=0x7f3470004270 comp rx=0 tx=0).stop 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 shutdown_connections 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f34640b6590 0x7f34640b8aa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f347c108d00 0x7f347c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f347c1083b0 0x7f347c19aae0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 --2- 172.21.15.92:0/1766293766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f347c102ac0 0x7f347c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 >> 172.21.15.92:0/1766293766 conn(0x7f347c079720 msgr2=0x7f347c101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 shutdown_connections 2024-09-21T02:06:37.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:37.985+0000 7f348172f640 1 -- 172.21.15.92:0/1766293766 wait complete. 2024-09-21T02:06:38.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:38 smithi136 ceph-mon[34754]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:06:38.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:38 smithi092 ceph-mon[31190]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:06:38.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:38 smithi099 ceph-mon[34404]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:06:38.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:06:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 -- 172.21.15.92:0/1084920732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c076100 msgr2=0x7f523c10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 --2- 172.21.15.92:0/1084920732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c076100 0x7f523c10fba0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5230009900 tx=0x7f523002a3f0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 -- 172.21.15.92:0/1084920732 shutdown_connections 2024-09-21T02:06:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 --2- 172.21.15.92:0/1084920732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f523c1100e0 0x7f523c112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 --2- 172.21.15.92:0/1084920732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c076100 0x7f523c10fba0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 --2- 172.21.15.92:0/1084920732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f523c0757b0 0x7f523c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 -- 172.21.15.92:0/1084920732 >> 172.21.15.92:0/1084920732 conn(0x7f523c0ff610 msgr2=0x7f523c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 -- 172.21.15.92:0/1084920732 shutdown_connections 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.566+0000 7f52437a2640 1 -- 172.21.15.92:0/1084920732 wait complete. 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 Processor -- start 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 -- start start 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f523c0757b0 0x7f523c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f523c076100 0x7f523c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 0x7f523c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f523c115c90 con 0x7f523c1100e0 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f523c115b10 con 0x7f523c076100 2024-09-21T02:06:39.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.567+0000 7f52437a2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f523c115e10 con 0x7f523c0757b0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 0x7f523c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 0x7f523c1a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43312/0 (socket says 172.21.15.92:43312) 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 -- 172.21.15.92:0/3656902828 learned_addr learned my addr 172.21.15.92:0/3656902828 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5240d3d640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f523c076100 0x7f523c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f524153e640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f523c0757b0 0x7f523c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 -- 172.21.15.92:0/3656902828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f523c0757b0 msgr2=0x7f523c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f523c0757b0 0x7f523c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 -- 172.21.15.92:0/3656902828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f523c076100 msgr2=0x7f523c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f523c076100 0x7f523c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 -- 172.21.15.92:0/3656902828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c1a24d0 con 0x7f523c1100e0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f5241d3f640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 0x7f523c1a1e60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f522c00cce0 tx=0x7f522c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f523a7fc640 1 -- 172.21.15.92:0/3656902828 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00f450 con 0x7f523c1100e0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f523a7fc640 1 -- 172.21.15.92:0/3656902828 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c00faa0 con 0x7f523c1100e0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c1a27c0 con 0x7f523c1100e0 2024-09-21T02:06:39.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.568+0000 7f523a7fc640 1 -- 172.21.15.92:0/3656902828 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c018a80 con 0x7f523c1100e0 2024-09-21T02:06:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.569+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a2c80 con 0x7f523c1100e0 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.571+0000 7f523a7fc640 1 -- 172.21.15.92:0/3656902828 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f522c00fc50 con 0x7f523c1100e0 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.571+0000 7f5249d16740 1 Processor -- start 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.571+0000 7f5249d16740 1 -- start start 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5249d16740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5249d16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f0c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5249d16740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5249d16740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556c92e82ea0 con 0x556c92b44390 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5249d16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c92e722d0 con 0x556c92b3c1b0 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5249d16740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c92e760e0 con 0x556c92b65070 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f0cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f0cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43326/0 (socket says 172.21.15.92:43326) 2024-09-21T02:06:39.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 -- 172.21.15.92:0/3626192803 learned_addr learned my addr 172.21.15.92:0/3626192803 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f524153e640 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f0c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f523a7fc640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f52240b6540 0x7f52240b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5241d3f640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f52240b6540 0x7f52240b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 -- 172.21.15.92:0/3626192803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 msgr2=0x556c92f0c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f0c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 -- 172.21.15.92:0/3626192803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 msgr2=0x556c92f0c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f0c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f5240d3d640 1 -- 172.21.15.92:0/3626192803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c92bd0650 con 0x556c92b44390 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.572+0000 7f523a7fc640 1 -- 172.21.15.92:0/3656902828 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f522c0e7670 con 0x7f523c1100e0 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.573+0000 7f5240d3d640 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f0cd40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f523c06c970 tx=0x7f5230009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2611063bb90a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.573+0000 7f52237fe640 1 -- 172.21.15.92:0/3626192803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230035070 con 0x556c92b44390 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.573+0000 7f5241d3f640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f52240b6540 0x7f52240b8a50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f522c063250 tx=0x7f522c0a2090 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.573+0000 7f52237fe640 1 -- 172.21.15.92:0/3626192803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5230031900 con 0x556c92b44390 2024-09-21T02:06:39.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.573+0000 7f52237fe640 1 -- 172.21.15.92:0/3626192803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230005590 con 0x556c92b44390 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 -- 172.21.15.92:0/3626192803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 msgr2=0x556c92f0cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f0cd40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f523c06c970 tx=0x7f5230009560 comp rx=0 tx=0).stop 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 -- 172.21.15.92:0/3626192803 shutdown_connections 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f0cd40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f0c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 --2- 172.21.15.92:0/3626192803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f0c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 -- 172.21.15.92:0/3626192803 >> 172.21.15.92:0/3626192803 conn(0x556c92b59460 msgr2=0x556c92f0a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 -- 172.21.15.92:0/3626192803 shutdown_connections 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 -- 172.21.15.92:0/3626192803 wait complete. 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 -- start start 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.574+0000 7f5249d16740 1 Processor -- start 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f17e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556c92cd3700 con 0x556c92b44390 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c92e0d160 con 0x556c92b3c1b0 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c92f1a9e0 con 0x556c92b65070 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5249d16740 10 client.0 _ll_register_callbacks cb 0x556c92e269e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f17e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f17e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43340/0 (socket says 172.21.15.92:43340) 2024-09-21T02:06:39.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 -- 172.21.15.92:0/445137015 learned_addr learned my addr 172.21.15.92:0/445137015 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5240d3d640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5240d3d640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f18380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 -- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 msgr2=0x556c92f18380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f18380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 -- 172.21.15.92:0/445137015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 msgr2=0x556c92f178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f178e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 -- 172.21.15.92:0/445137015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c92e3ac90 con 0x556c92b44390 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5240d3d640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f524153e640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f17e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5234010a50 tx=0x7f5234010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11d4dd8d151a0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.575+0000 7f5220ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5220ff9640 1 -- 172.21.15.92:0/445137015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523401e060 con 0x556c92b44390 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c92e84df0 con 0x556c92b44390 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5220ff9640 1 -- 172.21.15.92:0/445137015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f523400fbd0 con 0x556c92b44390 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5220ff9640 1 -- 172.21.15.92:0/445137015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f523400fdd0 con 0x556c92b44390 2024-09-21T02:06:39.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 10 client.16035 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c92f26740 con 0x556c92b44390 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5220ff9640 1 -- 172.21.15.92:0/445137015 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556c92e84df0 con 0x556c92b44390 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5220ff9640 1 -- 172.21.15.92:0/445137015 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5234024520 con 0x556c92b44390 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 10 client.16035 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 10 client.16035 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 20 client.16035 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92b239f0 0x556c92f29c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 4 client.16035 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.576+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c92e808a0 con 0x556c92b239f0 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.577+0000 7f5240d3d640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92b239f0 0x556c92f29c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.577+0000 7f5220ff9640 10 client.16035 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.577+0000 7f5240d3d640 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92b239f0 0x556c92f29c50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8387d814890a827 server_cookie=dddb227dc401e67b in_seq=0 out_seq=0 2024-09-21T02:06:39.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5220ff9640 1 -- 172.21.15.92:0/445137015 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c92e808a0 con 0x556c92b239f0 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5220ff9640 10 client.16035 handle_command_reply: tid=0 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 client.16035 shutdown 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 msgr2=0x556c92f17e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f17e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5234010a50 tx=0x7f5234010f10 comp rx=0 tx=0).stop 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 shutdown_connections 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92b239f0 0x556c92f29c50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f18380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f17e20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 --2- 172.21.15.92:0/445137015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5220ff9640 0 client.16035 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 >> 172.21.15.92:0/445137015 conn(0x556c92b59460 msgr2=0x556c92f0f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 shutdown_connections 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 1 -- 172.21.15.92:0/445137015 wait complete. 2024-09-21T02:06:39.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.578+0000 7f5249d16740 20 client.16035 trim_cache size 0 max 16384 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 Processor -- start 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 -- start start 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f17da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f09ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b1560 con 0x556c92b44390 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b30a0 con 0x556c92b3c1b0 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.591+0000 7f5249d16740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b1c30 con 0x556c92b65070 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f09760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43344/0 (socket says 172.21.15.92:43344) 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 -- 172.21.15.92:0/188147804 learned_addr learned my addr 172.21.15.92:0/188147804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f5240d3d640 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f17da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f5240d3d640 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f09ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 -- 172.21.15.92:0/188147804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 msgr2=0x556c92f09ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f09ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 -- 172.21.15.92:0/188147804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 msgr2=0x556c92f17da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f17da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 -- 172.21.15.92:0/188147804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c92f15de0 con 0x556c92b44390 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f5240d3d640 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f17da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f524153e640 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f09760 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f523400fd60 tx=0x7f523400a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60cc90faa991d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f5222ffd640 1 -- 172.21.15.92:0/188147804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234024bb0 con 0x556c92b44390 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f5222ffd640 1 -- 172.21.15.92:0/188147804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234024d50 con 0x556c92b44390 2024-09-21T02:06:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.592+0000 7f5222ffd640 1 -- 172.21.15.92:0/188147804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340048c0 con 0x556c92b44390 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 -- 172.21.15.92:0/188147804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 msgr2=0x556c92f09760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f09760 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f523400fd60 tx=0x7f523400a8d0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 -- 172.21.15.92:0/188147804 shutdown_connections 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b65070 0x556c92f09ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b44390 0x556c92f09760 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 --2- 172.21.15.92:0/188147804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b3c1b0 0x556c92f17da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 -- 172.21.15.92:0/188147804 >> 172.21.15.92:0/188147804 conn(0x556c92b59460 msgr2=0x556c92f17690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 -- 172.21.15.92:0/188147804 shutdown_connections 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 -- 172.21.15.92:0/188147804 wait complete. 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.593+0000 7f5249d16740 1 -- start start 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 Processor -- start 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b3c1b0 0x556c92f1bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.599 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b44390 0x556c92f1c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 0x556c92f93650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b29d0 con 0x556c92b65070 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b2300 con 0x556c92b44390 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1af540 con 0x556c92b3c1b0 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5249d16740 10 client.0 _ll_register_callbacks cb 0x7f523400a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5240d3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 0x556c92f93650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5240d3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 0x556c92f93650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43348/0 (socket says 172.21.15.92:43348) 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5240d3d640 1 -- 172.21.15.92:0/561051720 learned_addr learned my addr 172.21.15.92:0/561051720 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5240d3d640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b3c1b0 0x556c92f1bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f524153e640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b44390 0x556c92f1c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5240d3d640 1 -- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b3c1b0 msgr2=0x556c92f1bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.594+0000 7f5240d3d640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b3c1b0 0x556c92f1bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5240d3d640 1 -- 172.21.15.92:0/561051720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b44390 msgr2=0x556c92f1c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5240d3d640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b44390 0x556c92f1c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5240d3d640 1 -- 172.21.15.92:0/561051720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c92f1cd20 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f524153e640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b44390 0x556c92f1c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5240d3d640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 0x556c92f93650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5230031ae0 tx=0x7f5230005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0141021ca22523 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5220ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5220ff9640 1 -- 172.21.15.92:0/561051720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523005a070 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5220ff9640 1 -- 172.21.15.92:0/561051720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5230005320 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5234024d50 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5220ff9640 1 -- 172.21.15.92:0/561051720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5230035040 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c92f94be0 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5220ff9640 1 -- 172.21.15.92:0/561051720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234024d50 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5220ff9640 1 -- 172.21.15.92:0/561051720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f523005f070 con 0x556c92b65070 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 10 client.16041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:06:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 20 client.16041 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92af8da0 0x556c92f96ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 4 client.16041 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.595+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c92e7dcd0 con 0x556c92af8da0 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.596+0000 7f524153e640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92af8da0 0x556c92f96ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.596+0000 7f5220ff9640 10 client.16041 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.596+0000 7f524153e640 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92af8da0 0x556c92f96ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=703a0e91a63e65eb server_cookie=fd37f9c8d703e4fa in_seq=0 out_seq=0 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.596+0000 7f5220ff9640 1 -- 172.21.15.92:0/561051720 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c92e7dcd0 con 0x556c92af8da0 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.596+0000 7f5220ff9640 10 client.16041 handle_command_reply: tid=0 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.596+0000 7f5249d16740 1 client.16041 shutdown 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 msgr2=0x556c92f93650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 0x556c92f93650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5230031ae0 tx=0x7f5230005130 comp rx=0 tx=0).stop 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 shutdown_connections 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x556c92af8da0 0x556c92f96ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556c92b65070 0x556c92f93650 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c92b44390 0x556c92f1c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 --2- 172.21.15.92:0/561051720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c92b3c1b0 0x556c92f1bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5220ff9640 0 client.16041 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 >> 172.21.15.92:0/561051720 conn(0x556c92b59460 msgr2=0x556c92f22bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 shutdown_connections 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 1 -- 172.21.15.92:0/561051720 wait complete. 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f5249d16740 20 client.16041 trim_cache size 0 max 16384 2024-09-21T02:06:39.603 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f52240b6540 msgr2=0x7f52240b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f52240b6540 0x7f52240b8a50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f522c063250 tx=0x7f522c0a2090 comp rx=0 tx=0).stop 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 msgr2=0x7f523c1a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 0x7f523c1a1e60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f522c00cce0 tx=0x7f522c00a620 comp rx=0 tx=0).stop 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 shutdown_connections 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f52240b6540 0x7f52240b8a50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f523c1100e0 0x7f523c1a1e60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f523c076100 0x7f523c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 --2- 172.21.15.92:0/3656902828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f523c0757b0 0x7f523c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:39.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.597+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 >> 172.21.15.92:0/3656902828 conn(0x7f523c0ff610 msgr2=0x7f523c109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:39.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.598+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 shutdown_connections 2024-09-21T02:06:39.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:39.598+0000 7f52437a2640 1 -- 172.21.15.92:0/3656902828 wait complete. 2024-09-21T02:06:40.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 -- 172.21.15.92:0/2428752131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4103640 msgr2=0x7f7ac4103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 --2- 172.21.15.92:0/2428752131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4103640 0x7f7ac4103a50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009900 tx=0x7f7ab802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 -- 172.21.15.92:0/2428752131 shutdown_connections 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 --2- 172.21.15.92:0/2428752131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ac4103fe0 0x7f7ac410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 --2- 172.21.15.92:0/2428752131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac4103640 0x7f7ac4103a50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 --2- 172.21.15.92:0/2428752131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac41025c0 0x7f7ac4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 -- 172.21.15.92:0/2428752131 >> 172.21.15.92:0/2428752131 conn(0x7f7ac40fdf00 msgr2=0x7f7ac41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 -- 172.21.15.92:0/2428752131 shutdown_connections 2024-09-21T02:06:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 -- 172.21.15.92:0/2428752131 wait complete. 2024-09-21T02:06:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.292+0000 7f7acaa15640 1 Processor -- start 2024-09-21T02:06:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 -- start start 2024-09-21T02:06:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 0x7f7ac406c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac4103640 0x7f7ac419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ac4103fe0 0x7f7ac41a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac410e210 con 0x7f7ac41025c0 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac410e090 con 0x7f7ac4103640 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7acaa15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac410e390 con 0x7f7ac4103fe0 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 0x7f7ac406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 0x7f7ac406c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43374/0 (socket says 172.21.15.92:43374) 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 -- 172.21.15.92:0/3421987735 learned_addr learned my addr 172.21.15.92:0/3421987735 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac4103640 0x7f7ac419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ac4103fe0 0x7f7ac41a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 -- 172.21.15.92:0/3421987735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ac4103fe0 msgr2=0x7f7ac41a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ac4103fe0 0x7f7ac41a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 -- 172.21.15.92:0/3421987735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac4103640 msgr2=0x7f7ac419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac4103640 0x7f7ac419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.293+0000 7f7ac3fff640 1 -- 172.21.15.92:0/3421987735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac41a1a00 con 0x7f7ac41025c0 2024-09-21T02:06:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.294+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 0x7f7ac406c210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400d950 tx=0x7f7ab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.294+0000 7f7ac17fa640 1 -- 172.21.15.92:0/3421987735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4017070 con 0x7f7ac41025c0 2024-09-21T02:06:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.294+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac41a1cf0 con 0x7f7ac41025c0 2024-09-21T02:06:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.294+0000 7f7ac17fa640 1 -- 172.21.15.92:0/3421987735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab4009c30 con 0x7f7ac41025c0 2024-09-21T02:06:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.295+0000 7f7ac17fa640 1 -- 172.21.15.92:0/3421987735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400f620 con 0x7f7ac41025c0 2024-09-21T02:06:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.295+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac41a21d0 con 0x7f7ac41025c0 2024-09-21T02:06:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ac17fa640 1 -- 172.21.15.92:0/3421987735 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f7ab4010460 con 0x7f7ac41025c0 2024-09-21T02:06:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 Processor -- start 2024-09-21T02:06:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 -- start start 2024-09-21T02:06:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c39260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603948642f0 0x560394c398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c3a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560394ba76f0 con 0x56039488d1b0 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560394b96b20 con 0x56039486c4d0 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560394b9a930 con 0x5603948642f0 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ac8fb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c39260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ac8fb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c39260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43380/0 (socket says 172.21.15.92:43380) 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.297+0000 7f7ac8fb2640 1 -- 172.21.15.92:0/355909938 learned_addr learned my addr 172.21.15.92:0/355909938 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603948642f0 0x560394c398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c3a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac8fb2640 1 -- 172.21.15.92:0/355909938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603948642f0 msgr2=0x560394c398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603948642f0 0x560394c398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac8fb2640 1 -- 172.21.15.92:0/355909938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 msgr2=0x560394c3a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c3a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac8fb2640 1 -- 172.21.15.92:0/355909938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560394b5c530 con 0x56039488d1b0 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603948642f0 0x560394c398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c3a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac17fa640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7aac0b6510 0x7f7aac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c39260 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009fd0 tx=0x7f7abc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea188ed93c3617fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7aac0b6510 0x7f7aac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7aaa7fc640 1 -- 172.21.15.92:0/355909938 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc004d20 con 0x56039488d1b0 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7ac17fa640 1 -- 172.21.15.92:0/3421987735 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7ab40e6ec0 con 0x7f7ac41025c0 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.298+0000 7f7aaa7fc640 1 -- 172.21.15.92:0/355909938 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc004050 con 0x56039488d1b0 2024-09-21T02:06:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7aaa7fc640 1 -- 172.21.15.92:0/355909938 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc005280 con 0x56039488d1b0 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/355909938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 msgr2=0x560394c39260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c39260 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009fd0 tx=0x7f7abc0118d0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ac3fff640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7aac0b6510 0x7f7aac0b8a20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4000c00 tx=0x7f7ab4005880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/355909938 shutdown_connections 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c3a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603948642f0 0x560394c398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/355909938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c39260 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.299+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/355909938 >> 172.21.15.92:0/355909938 conn(0x560394881300 msgr2=0x560394c34cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.300+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/355909938 shutdown_connections 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.300+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/355909938 wait complete. 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.300+0000 7f7ad1a5f740 1 -- start start 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.300+0000 7f7ad1a5f740 1 Processor -- start 2024-09-21T02:06:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.300+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603948642f0 0x560394c42a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.300+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c43c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560394aceab0 con 0x56039488d1b0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560394b3c730 con 0x5603948642f0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560394c3af90 con 0x56039486c4d0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ad1a5f740 10 client.0 _ll_register_callbacks cb 0x560394b906b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c430f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50096/0 (socket says 172.21.15.92:50096) 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 -- 172.21.15.92:0/1758469310 learned_addr learned my addr 172.21.15.92:0/1758469310 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c43c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603948642f0 0x560394c42a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 -- 172.21.15.92:0/1758469310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603948642f0 msgr2=0x560394c42a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603948642f0 0x560394c42a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 -- 172.21.15.92:0/1758469310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 msgr2=0x560394c43c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c43c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 -- 172.21.15.92:0/1758469310 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603948f8790 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603948642f0 0x560394c42a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c430f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009900 tx=0x7f7ab80317d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fffe03e47b6ee645 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.301+0000 7f7aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7aa37fe640 1 -- 172.21.15.92:0/1758469310 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8035070 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7abc004050 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7aa37fe640 1 -- 172.21.15.92:0/1758469310 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7ab8004240 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7aa37fe640 1 -- 172.21.15.92:0/1758469310 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ab804b070 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 10 client.25498 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560394971620 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7aa37fe640 1 -- 172.21.15.92:0/1758469310 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc004050 con 0x56039486c4d0 2024-09-21T02:06:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7aa37fe640 1 -- 172.21.15.92:0/1758469310 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7ab8031ce0 con 0x56039486c4d0 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 10 client.25498 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 10 client.25498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 20 client.25498 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039484bd80 0x560394c501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 4 client.25498 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.302+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560394ba9640 con 0x56039484bd80 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.303+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039484bd80 0x560394c501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.303+0000 7f7aa37fe640 10 client.25498 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.303+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039484bd80 0x560394c501a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af51ff56235f91e2 server_cookie=468a882d47e53f in_seq=0 out_seq=0 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7aa37fe640 1 -- 172.21.15.92:0/1758469310 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560394ba9640 con 0x56039484bd80 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7aa37fe640 10 client.25498 handle_command_reply: tid=0 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 client.25498 shutdown 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 msgr2=0x560394c430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c430f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009900 tx=0x7f7ab80317d0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 shutdown_connections 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039484bd80 0x560394c501a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56039488d1b0 0x560394c43c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c430f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7aa37fe640 0 client.25498 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/1758469310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603948642f0 0x560394c42a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 >> 172.21.15.92:0/1758469310 conn(0x560394881300 msgr2=0x560394c2eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 shutdown_connections 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.304+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/1758469310 wait complete. 2024-09-21T02:06:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.305+0000 7f7ad1a5f740 20 client.25498 trim_cache size 0 max 16384 2024-09-21T02:06:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 Processor -- start 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 -- start start 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039484bd80 0x560394c40180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c39870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c41c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41a8e60 con 0x5603948642f0 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41aa9a0 con 0x56039484bd80 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41a9530 con 0x56039486c4d0 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ac37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c39870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ac37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c39870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43398/0 (socket says 172.21.15.92:43398) 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ac37fe640 1 -- 172.21.15.92:0/517477461 learned_addr learned my addr 172.21.15.92:0/517477461 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.319+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039484bd80 0x560394c40180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c41c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac37fe640 1 -- 172.21.15.92:0/517477461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 msgr2=0x560394c41c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c41c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac37fe640 1 -- 172.21.15.92:0/517477461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039484bd80 msgr2=0x560394c40180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039484bd80 0x560394c40180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac37fe640 1 -- 172.21.15.92:0/517477461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560394c4ac00 con 0x5603948642f0 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039484bd80 0x560394c40180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c39870 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8031500 tx=0x7f7ab8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3551991eaaf1d9f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7aa9ffb640 1 -- 172.21.15.92:0/517477461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8035040 con 0x5603948642f0 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7aa9ffb640 1 -- 172.21.15.92:0/517477461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab803ba90 con 0x5603948642f0 2024-09-21T02:06:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.320+0000 7f7aa9ffb640 1 -- 172.21.15.92:0/517477461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8003d70 con 0x5603948642f0 2024-09-21T02:06:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/517477461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 msgr2=0x560394c39870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c39870 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8031500 tx=0x7f7ab8001e70 comp rx=0 tx=0).stop 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/517477461 shutdown_connections 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039486c4d0 0x560394c41c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c39870 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/517477461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039484bd80 0x560394c40180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/517477461 >> 172.21.15.92:0/517477461 conn(0x560394881300 msgr2=0x560394c3fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/517477461 shutdown_connections 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.321+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/517477461 wait complete. 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 -- start start 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 Processor -- start 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 0x560394c4a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41aa2d0 con 0x5603948642f0 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41a9c00 con 0x56039486c4d0 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41a6e40 con 0x56039484bd80 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ad1a5f740 10 client.0 _ll_register_callbacks cb 0x7f7ab8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ac8fb2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 0x560394c4a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ac37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ac8fb2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 0x560394c4a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50120/0 (socket says 172.21.15.92:50120) 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ac37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c46090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43414/0 (socket says 172.21.15.92:43414) 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ac8fb2640 1 -- 172.21.15.92:0/4062969104 learned_addr learned my addr 172.21.15.92:0/4062969104 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.322+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac37fe640 1 -- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 msgr2=0x560394c4a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 0x560394c4a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac37fe640 1 -- 172.21.15.92:0/4062969104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 msgr2=0x560394c465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac37fe640 1 -- 172.21.15.92:0/4062969104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560394c47200 con 0x5603948642f0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 0x560394c4a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ac37fe640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c46090 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8031500 tx=0x7f7ab802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa45bd4b880a2d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7aa37fe640 1 -- 172.21.15.92:0/4062969104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8035040 con 0x5603948642f0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7aa37fe640 1 -- 172.21.15.92:0/4062969104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7ab8050680 con 0x5603948642f0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab803ba90 con 0x5603948642f0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7aa37fe640 1 -- 172.21.15.92:0/4062969104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ab8050880 con 0x5603948642f0 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ad1a5f740 10 client.16059 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.323+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560394cbace0 con 0x5603948642f0 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7aa37fe640 1 -- 172.21.15.92:0/4062969104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab803ba90 con 0x5603948642f0 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7aa37fe640 1 -- 172.21.15.92:0/4062969104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7ab8031ce0 con 0x5603948642f0 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ad1a5f740 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ad1a5f740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ad1a5f740 20 client.16059 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039488d1b0 0x560394cbd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ad1a5f740 4 client.16059 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560394ba50f0 con 0x56039488d1b0 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.324+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039488d1b0 0x560394cbd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.325+0000 7f7aa37fe640 10 client.16059 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.325+0000 7f7ac8fb2640 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039488d1b0 0x560394cbd650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77456446fc61141c server_cookie=58a7f0d190c83298 in_seq=0 out_seq=0 2024-09-21T02:06:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7aa37fe640 1 -- 172.21.15.92:0/4062969104 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560394ba50f0 con 0x56039488d1b0 2024-09-21T02:06:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7aa37fe640 10 client.16059 handle_command_reply: tid=0 2024-09-21T02:06:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7ad1a5f740 1 client.16059 shutdown 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 msgr2=0x560394c46090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c46090 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8031500 tx=0x7f7ab802a410 comp rx=0 tx=0).stop 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 shutdown_connections 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.335+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56039488d1b0 0x560394cbd650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039486c4d0 0x560394c465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603948642f0 0x560394c46090 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 1 --2- 172.21.15.92:0/4062969104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56039484bd80 0x560394c4a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 >> 172.21.15.92:0/4062969104 conn(0x560394881300 msgr2=0x560394c41c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7aa37fe640 0 client.16059 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 shutdown_connections 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 1 -- 172.21.15.92:0/4062969104 wait complete. 2024-09-21T02:06:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7ad1a5f740 20 client.16059 trim_cache size 0 max 16384 2024-09-21T02:06:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:06:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:06:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:06:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7aac0b6510 msgr2=0x7f7aac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7acaa15640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7aac0b6510 0x7f7aac0b8a20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4000c00 tx=0x7f7ab4005880 comp rx=0 tx=0).stop 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 msgr2=0x7f7ac406c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.336+0000 7f7acaa15640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 0x7f7ac406c210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400d950 tx=0x7f7ab400de10 comp rx=0 tx=0).stop 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 shutdown_connections 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7aac0b6510 0x7f7aac0b8a20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ac4103fe0 0x7f7ac41a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac4103640 0x7f7ac419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 --2- 172.21.15.92:0/3421987735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ac41025c0 0x7f7ac406c210 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 >> 172.21.15.92:0/3421987735 conn(0x7f7ac40fdf00 msgr2=0x7f7ac4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 shutdown_connections 2024-09-21T02:06:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:40.337+0000 7f7acaa15640 1 -- 172.21.15.92:0/3421987735 wait complete. 2024-09-21T02:06:40.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:40 smithi092 ceph-mon[31190]: pgmap v411: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:06:40.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:06:40.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:40 smithi136 ceph-mon[34754]: pgmap v411: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:06:40.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:06:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:40 smithi099 ceph-mon[34404]: pgmap v411: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:06:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:06:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 -- 172.21.15.92:0/2380134790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350108d20 msgr2=0x7fc35010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 --2- 172.21.15.92:0/2380134790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350108d20 0x7fc35010f550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc348009ca0 tx=0x7fc34802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 -- 172.21.15.92:0/2380134790 shutdown_connections 2024-09-21T02:06:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 --2- 172.21.15.92:0/2380134790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350108d20 0x7fc35010f550 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 --2- 172.21.15.92:0/2380134790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3501083d0 0x7fc3501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 --2- 172.21.15.92:0/2380134790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc350102ae0 0x7fc350102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 -- 172.21.15.92:0/2380134790 >> 172.21.15.92:0/2380134790 conn(0x7fc350079790 msgr2=0x7fc350079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 -- 172.21.15.92:0/2380134790 shutdown_connections 2024-09-21T02:06:40.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.916+0000 7fc354d8f640 1 -- 172.21.15.92:0/2380134790 wait complete. 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 Processor -- start 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 -- start start 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 0x7fc350110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3501083d0 0x7fc350110570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc350108d20 0x7fc350110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc350113b00 con 0x7fc350102ae0 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc350113980 con 0x7fc350108d20 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.917+0000 7fc354d8f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc350113c80 con 0x7fc3501083d0 2024-09-21T02:06:40.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 0x7fc350110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 0x7fc350110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43440/0 (socket says 172.21.15.92:43440) 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 -- 172.21.15.92:0/760954623 learned_addr learned my addr 172.21.15.92:0/760954623 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3501083d0 0x7fc350110570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc350108d20 0x7fc350110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 -- 172.21.15.92:0/760954623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3501083d0 msgr2=0x7fc350110570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3501083d0 0x7fc350110570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 -- 172.21.15.92:0/760954623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc350108d20 msgr2=0x7fc350110ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc350108d20 0x7fc350110ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 -- 172.21.15.92:0/760954623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3501a87d0 con 0x7fc350102ae0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc34e59c640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 0x7fc350110030 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc34000d950 tx=0x7fc34000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc3477fe640 1 -- 172.21.15.92:0/760954623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340017070 con 0x7fc350102ae0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3501a8ac0 con 0x7fc350102ae0 2024-09-21T02:06:40.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc3477fe640 1 -- 172.21.15.92:0/760954623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340009c30 con 0x7fc350102ae0 2024-09-21T02:06:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc3477fe640 1 -- 172.21.15.92:0/760954623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34000f620 con 0x7fc350102ae0 2024-09-21T02:06:40.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.918+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501a9050 con 0x7fc350102ae0 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 Processor -- start 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 -- start start 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc3477fe640 1 -- 172.21.15.92:0/760954623 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc340010460 con 0x7fc350102ae0 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 0x55d7c0e846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e84be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0abc390 0x55d7c0e85120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0dfaea0 con 0x55d7c0abc390 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0dea2d0 con 0x55d7c0ab41b0 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.921+0000 7fc35b303740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0dee0e0 con 0x55d7c0add070 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0abc390 0x55d7c0e85120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 0x55d7c0e846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0abc390 0x55d7c0e85120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43454/0 (socket says 172.21.15.92:43454) 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34dd9b640 1 -- 172.21.15.92:0/2003454396 learned_addr learned my addr 172.21.15.92:0/2003454396 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 0x55d7c0e846a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50152/0 (socket says 172.21.15.92:50152) 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34e59c640 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e84be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 -- 172.21.15.92:0/2003454396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 msgr2=0x55d7c0e84be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e84be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 -- 172.21.15.92:0/2003454396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0abc390 msgr2=0x55d7c0e85120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0abc390 0x55d7c0e85120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 -- 172.21.15.92:0/2003454396 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c0b48650 con 0x55d7c0add070 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc3477fe640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc33c0b6510 0x7fc33c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34e59c640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc33c0b6510 0x7fc33c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 0x55d7c0e846a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc348005e30 tx=0x7fc3480098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff61f45af263eac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.922+0000 7fc3477fe640 1 -- 172.21.15.92:0/760954623 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc3400e6eb0 con 0x7fc350102ae0 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.923+0000 7fc344ff9640 1 -- 172.21.15.92:0/2003454396 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348031db0 con 0x55d7c0add070 2024-09-21T02:06:40.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.923+0000 7fc344ff9640 1 -- 172.21.15.92:0/2003454396 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc348036030 con 0x55d7c0add070 2024-09-21T02:06:40.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.923+0000 7fc344ff9640 1 -- 172.21.15.92:0/2003454396 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348004240 con 0x55d7c0add070 2024-09-21T02:06:40.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 -- 172.21.15.92:0/2003454396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 msgr2=0x55d7c0e846a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc34e59c640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc33c0b6510 0x7fc33c0b8a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55d7c0e83320 tx=0x7fc340094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 0x55d7c0e846a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc348005e30 tx=0x7fc3480098b0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 -- 172.21.15.92:0/2003454396 shutdown_connections 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0abc390 0x55d7c0e85120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e84be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 --2- 172.21.15.92:0/2003454396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0add070 0x55d7c0e846a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 -- 172.21.15.92:0/2003454396 >> 172.21.15.92:0/2003454396 conn(0x55d7c0ad1460 msgr2=0x55d7c0e82590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 -- 172.21.15.92:0/2003454396 shutdown_connections 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 -- 172.21.15.92:0/2003454396 wait complete. 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 -- start start 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.924+0000 7fc35b303740 1 Processor -- start 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0c4b700 con 0x55d7c0add070 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0d85160 con 0x55d7c0ab41b0 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0e92ca0 con 0x55d7c0abc390 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc35b303740 10 client.0 _ll_register_callbacks cb 0x55d7c0d9e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e90790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43460/0 (socket says 172.21.15.92:43460) 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34dd9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 -- 172.21.15.92:0/3806740361 learned_addr learned my addr 172.21.15.92:0/3806740361 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 -- 172.21.15.92:0/3806740361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 msgr2=0x55d7c0e90210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e90210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 -- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 msgr2=0x55d7c0e8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 -- 172.21.15.92:0/3806740361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c0db2c90 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e90210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e90790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc348003e40 tx=0x7fc348003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c791050f6585c22d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc335ffb640 1 -- 172.21.15.92:0/3806740361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348058060 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.925+0000 7fc335ffb640 1 -- 172.21.15.92:0/3806740361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc348031450 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc348036030 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc335ffb640 1 -- 172.21.15.92:0/3806740361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc348031690 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 10 client.16071 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7c0e9e970 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc335ffb640 1 -- 172.21.15.92:0/3806740361 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc348036030 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc335ffb640 1 -- 172.21.15.92:0/3806740361 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc348050070 con 0x55d7c0add070 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:40.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 10 client.16071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 20 client.16071 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a9b9f0 0x55d7c0ea1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 4 client.16071 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c0dfcdf0 con 0x55d7c0a9b9f0 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.926+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a9b9f0 0x55d7c0ea1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.927+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a9b9f0 0x55d7c0ea1cd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74511db721ac50f server_cookie=f75c425f36a5ddf9 in_seq=0 out_seq=0 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.927+0000 7fc335ffb640 10 client.16071 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.927+0000 7fc335ffb640 1 -- 172.21.15.92:0/3806740361 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7c0dfcdf0 con 0x55d7c0a9b9f0 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.927+0000 7fc335ffb640 10 client.16071 handle_command_reply: tid=0 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.927+0000 7fc35b303740 1 client.16071 shutdown 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 msgr2=0x55d7c0e90790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e90790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc348003e40 tx=0x7fc348003d40 comp rx=0 tx=0).stop 2024-09-21T02:06:40.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 shutdown_connections 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a9b9f0 0x55d7c0ea1cd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e90790 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e90210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 --2- 172.21.15.92:0/3806740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc335ffb640 0 client.16071 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 >> 172.21.15.92:0/3806740361 conn(0x55d7c0ad1460 msgr2=0x55d7c0e87830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 shutdown_connections 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 1 -- 172.21.15.92:0/3806740361 wait complete. 2024-09-21T02:06:40.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.928+0000 7fc35b303740 20 client.16071 trim_cache size 0 max 16384 2024-09-21T02:06:40.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 Processor -- start 2024-09-21T02:06:40.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 -- start start 2024-09-21T02:06:40.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e85de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e8cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0add070 0x55d7c0e8d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b1680 con 0x55d7c0ab41b0 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b31c0 con 0x55d7c0add070 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc35b303740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b1d50 con 0x55d7c0abc390 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e85de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e85de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43472/0 (socket says 172.21.15.92:43472) 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc34ed9d640 1 -- 172.21.15.92:0/887417277 learned_addr learned my addr 172.21.15.92:0/887417277 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e8cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.941+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0add070 0x55d7c0e8d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc34ed9d640 1 -- 172.21.15.92:0/887417277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 msgr2=0x55d7c0e8cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e8cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc34ed9d640 1 -- 172.21.15.92:0/887417277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0add070 msgr2=0x55d7c0e8d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0add070 0x55d7c0e8d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc34ed9d640 1 -- 172.21.15.92:0/887417277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c0e8e1d0 con 0x55d7c0ab41b0 2024-09-21T02:06:40.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e85de0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc34802fae0 tx=0x7fc348003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fccd604a2bbe5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc337fff640 1 -- 172.21.15.92:0/887417277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348058070 con 0x55d7c0ab41b0 2024-09-21T02:06:40.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc337fff640 1 -- 172.21.15.92:0/887417277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc348036040 con 0x55d7c0ab41b0 2024-09-21T02:06:40.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.942+0000 7fc337fff640 1 -- 172.21.15.92:0/887417277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34802fcd0 con 0x55d7c0ab41b0 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- 172.21.15.92:0/887417277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 msgr2=0x55d7c0e85de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e85de0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc34802fae0 tx=0x7fc348003db0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- 172.21.15.92:0/887417277 shutdown_connections 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0add070 0x55d7c0e8d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0abc390 0x55d7c0e8cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- 172.21.15.92:0/887417277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e85de0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- 172.21.15.92:0/887417277 >> 172.21.15.92:0/887417277 conn(0x55d7c0ad1460 msgr2=0x55d7c0e856d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- 172.21.15.92:0/887417277 shutdown_connections 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- 172.21.15.92:0/887417277 wait complete. 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- start start 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 Processor -- start 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0abc390 0x55d7c0e93cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b2af0 con 0x55d7c0add070 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501b2420 con 0x55d7c0abc390 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.943+0000 7fc35b303740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501af660 con 0x55d7c0ab41b0 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc35b303740 10 client.0 _ll_register_callbacks cb 0x7fc338004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e942c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43482/0 (socket says 172.21.15.92:43482) 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 -- 172.21.15.92:0/1375202180 learned_addr learned my addr 172.21.15.92:0/1375202180 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0abc390 0x55d7c0e93cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 -- 172.21.15.92:0/1375202180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0ab41b0 msgr2=0x55d7c0e987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 -- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0abc390 msgr2=0x55d7c0e93cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0abc390 0x55d7c0e93cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 -- 172.21.15.92:0/1375202180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c0e94aa0 con 0x55d7c0add070 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc34ed9d640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e942c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55d7c0e92d40 tx=0x7fc348009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26edf6fa062bfd7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc335ffb640 1 -- 172.21.15.92:0/1375202180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348058040 con 0x55d7c0add070 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc335ffb640 1 -- 172.21.15.92:0/1375202180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc348031dc0 con 0x55d7c0add070 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc348036040 con 0x55d7c0add070 2024-09-21T02:06:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc335ffb640 1 -- 172.21.15.92:0/1375202180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc34802fcd0 con 0x55d7c0add070 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc35b303740 10 client.16077 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.944+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7c0f0c460 con 0x55d7c0add070 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc335ffb640 1 -- 172.21.15.92:0/1375202180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc348036040 con 0x55d7c0add070 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc335ffb640 1 -- 172.21.15.92:0/1375202180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc348065070 con 0x55d7c0add070 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc35b303740 10 client.16077 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc35b303740 10 client.16077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc35b303740 20 client.16077 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc35b303740 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a70da0 0x55d7c0f0eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc35b303740 4 client.16077 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7c0df88a0 con 0x55d7c0a70da0 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.945+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a70da0 0x55d7c0f0eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc34dd9b640 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a70da0 0x55d7c0f0eb60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=361f4861a7158b7f server_cookie=dd6e017c4c92971c in_seq=0 out_seq=0 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc335ffb640 10 client.16077 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:06:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc335ffb640 1 -- 172.21.15.92:0/1375202180 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7c0df88a0 con 0x55d7c0a70da0 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc335ffb640 10 client.16077 handle_command_reply: tid=0 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 client.16077 shutdown 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 msgr2=0x55d7c0e942c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e942c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55d7c0e92d40 tx=0x7fc348009460 comp rx=0 tx=0).stop 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 shutdown_connections 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55d7c0a70da0 0x55d7c0f0eb60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7c0add070 0x55d7c0e942c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc335ffb640 0 client.16077 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c0abc390 0x55d7c0e93cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 --2- 172.21.15.92:0/1375202180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d7c0ab41b0 0x55d7c0e987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 >> 172.21.15.92:0/1375202180 conn(0x55d7c0ad1460 msgr2=0x55d7c0e85e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 shutdown_connections 2024-09-21T02:06:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 1 -- 172.21.15.92:0/1375202180 wait complete. 2024-09-21T02:06:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.946+0000 7fc35b303740 20 client.16077 trim_cache size 0 max 16384 2024-09-21T02:06:40.953 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc33c0b6510 msgr2=0x7fc33c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc33c0b6510 0x7fc33c0b8a20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55d7c0e83320 tx=0x7fc340094040 comp rx=0 tx=0).stop 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 msgr2=0x7fc350110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 0x7fc350110030 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc34000d950 tx=0x7fc34000de10 comp rx=0 tx=0).stop 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 shutdown_connections 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc33c0b6510 0x7fc33c0b8a20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc350108d20 0x7fc350110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3501083d0 0x7fc350110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 --2- 172.21.15.92:0/760954623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc350102ae0 0x7fc350110030 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 >> 172.21.15.92:0/760954623 conn(0x7fc350079790 msgr2=0x7fc350106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 shutdown_connections 2024-09-21T02:06:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:40.947+0000 7fc354d8f640 1 -- 172.21.15.92:0/760954623 wait complete. 2024-09-21T02:06:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:06:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:06:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:06:41.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:06:41.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:06:41.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:06:41.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:06:41.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:06:41.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:06:41.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:06:42.058 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:42.452 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:42 smithi092 ceph-mon[31190]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:06:42.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 -- 172.21.15.92:0/127394775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b4103b60 msgr2=0x7f47b4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/127394775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b4103b60 0x7f47b4103f70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009900 tx=0x7f47a402a410 comp rx=0 tx=0).stop 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 -- 172.21.15.92:0/127394775 shutdown_connections 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/127394775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b4109df0 0x7f47b4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/127394775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b41094a0 0x7f47b41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/127394775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b4103b60 0x7f47b4103f70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 -- 172.21.15.92:0/127394775 >> 172.21.15.92:0/127394775 conn(0x7f47b407b850 msgr2=0x7f47b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 -- 172.21.15.92:0/127394775 shutdown_connections 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.544+0000 7f47b9d5c640 1 -- 172.21.15.92:0/127394775 wait complete. 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.545+0000 7f47b9d5c640 1 Processor -- start 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.545+0000 7f47b9d5c640 1 -- start start 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.545+0000 7f47b9d5c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b4103b60 0x7f47b419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b9d5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 0x7f47b419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b9d5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b4109df0 0x7f47b419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b9d5c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113b30 con 0x7f47b41094a0 2024-09-21T02:06:42.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b9d5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41139b0 con 0x7f47b4109df0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b9d5c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113cb0 con 0x7f47b4103b60 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 0x7f47b419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 0x7f47b419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43494/0 (socket says 172.21.15.92:43494) 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2597147120 learned_addr learned my addr 172.21.15.92:0/2597147120 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b4103b60 0x7f47b419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b3fff640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b4109df0 0x7f47b419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2597147120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b4103b60 msgr2=0x7f47b419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b4103b60 0x7f47b419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2597147120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b4109df0 msgr2=0x7f47b419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b4109df0 0x7f47b419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2597147120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b41a0490 con 0x7f47b41094a0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 0x7f47b419aac0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f47a800d750 tx=0x7f47a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b0ff9640 1 -- 172.21.15.92:0/2597147120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8017070 con 0x7f47b41094a0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.546+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b41a0780 con 0x7f47b41094a0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.547+0000 7f47b0ff9640 1 -- 172.21.15.92:0/2597147120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a80054d0 con 0x7f47b41094a0 2024-09-21T02:06:42.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.547+0000 7f47b0ff9640 1 -- 172.21.15.92:0/2597147120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8004c30 con 0x7f47b41094a0 2024-09-21T02:06:42.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.547+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b41a6ee0 con 0x7f47b41094a0 2024-09-21T02:06:42.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 Processor -- start 2024-09-21T02:06:42.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 -- start start 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea71070 0x5624fee182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 0x5624fee18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b0ff9640 1 -- 172.21.15.92:0/2597147120 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f47a8004dd0 con 0x7f47b41094a0 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5624fed8eea0 con 0x5624fea481b0 2024-09-21T02:06:42.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624fed7e2d0 con 0x5624fea71070 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47c02d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5624fed820e0 con 0x5624fea50390 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 0x5624fee18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 0x5624fee18800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43496/0 (socket says 172.21.15.92:43496) 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 -- 172.21.15.92:0/2518011788 learned_addr learned my addr 172.21.15.92:0/2518011788 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b3fff640 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea71070 0x5624fee182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 -- 172.21.15.92:0/2518011788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 msgr2=0x5624fee18d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee18d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 -- 172.21.15.92:0/2518011788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea71070 msgr2=0x5624fee182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea71070 0x5624fee182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.550+0000 7f47b37fe640 1 -- 172.21.15.92:0/2518011788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624feadc650 con 0x5624fea481b0 2024-09-21T02:06:42.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 0x5624fee18800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009900 tx=0x7f47a4032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc56adb6cbf83ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:42.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f47b0ff9640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f479c0b6510 0x7f479c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f4799ffb640 1 -- 172.21.15.92:0/2518011788 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4040070 con 0x5624fea481b0 2024-09-21T02:06:42.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f4799ffb640 1 -- 172.21.15.92:0/2518011788 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a40316a0 con 0x5624fea481b0 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f4799ffb640 1 -- 172.21.15.92:0/2518011788 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4035040 con 0x5624fea481b0 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f47b3fff640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f479c0b6510 0x7f479c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.551+0000 7f47b0ff9640 1 -- 172.21.15.92:0/2597147120 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47a80e6c90 con 0x7f47b41094a0 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 -- 172.21.15.92:0/2518011788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 msgr2=0x5624fee18800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47b3fff640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f479c0b6510 0x7f479c0b8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00a400 tx=0x7f47ac00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 0x5624fee18800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009900 tx=0x7f47a4032e60 comp rx=0 tx=0).stop 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 -- 172.21.15.92:0/2518011788 shutdown_connections 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee18d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea481b0 0x5624fee18800 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2518011788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea71070 0x5624fee182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 -- 172.21.15.92:0/2518011788 >> 172.21.15.92:0/2518011788 conn(0x5624fea65460 msgr2=0x5624fee161b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 -- 172.21.15.92:0/2518011788 shutdown_connections 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 -- 172.21.15.92:0/2518011788 wait complete. 2024-09-21T02:06:42.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.552+0000 7f47c02d0740 1 -- start start 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 Processor -- start 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 0x5624fee23f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea71070 0x5624fee2fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5624febdf700 con 0x5624fea50390 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624fed19160 con 0x5624fea481b0 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5624fee26a40 con 0x5624fea71070 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47c02d0740 10 client.0 _ll_register_callbacks cb 0x5624fed329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 0x5624fee23f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 0x5624fee23f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43508/0 (socket says 172.21.15.92:43508) 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2772540108 learned_addr learned my addr 172.21.15.92:0/2772540108 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b3fff640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea71070 0x5624fee2fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea71070 msgr2=0x5624fee2fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea71070 0x5624fee2fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2772540108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 msgr2=0x5624fee23a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee23a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b2ffd640 1 -- 172.21.15.92:0/2772540108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624fed46c90 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.553+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee23a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 0x5624fee23f60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f47a80ace40 tx=0x7f47a80acd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a751a911a7218c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f4792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f4792ffd640 1 -- 172.21.15.92:0/2772540108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8020070 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f4792ffd640 1 -- 172.21.15.92:0/2772540108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47a80940f0 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a40316a0 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f4792ffd640 1 -- 172.21.15.92:0/2772540108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47a80b18e0 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624fee327b0 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f4792ffd640 1 -- 172.21.15.92:0/2772540108 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a40316a0 con 0x5624fea50390 2024-09-21T02:06:42.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f4792ffd640 1 -- 172.21.15.92:0/2772540108 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47a80ec070 con 0x5624fea50390 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 20 client.16089 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea2f9f0 0x5624fee35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 4 client.16089 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.554+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624fed90df0 con 0x5624fea2f9f0 2024-09-21T02:06:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.555+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea2f9f0 0x5624fee35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.555+0000 7f4792ffd640 10 client.16089 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.555+0000 7f47b37fe640 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea2f9f0 0x5624fee35d30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4c58c08b7e828c server_cookie=7656e9b52eb93099 in_seq=0 out_seq=0 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f4792ffd640 1 -- 172.21.15.92:0/2772540108 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624fed90df0 con 0x5624fea2f9f0 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f4792ffd640 10 client.16089 handle_command_reply: tid=0 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 client.16089 shutdown 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 msgr2=0x5624fee23f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 0x5624fee23f60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f47a80ace40 tx=0x7f47a80acd40 comp rx=0 tx=0).stop 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 shutdown_connections 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea2f9f0 0x5624fee35d30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea71070 0x5624fee2fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea50390 0x5624fee23f60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f4792ffd640 0 client.16089 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 --2- 172.21.15.92:0/2772540108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee23a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 >> 172.21.15.92:0/2772540108 conn(0x5624fea65460 msgr2=0x5624fee1b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 shutdown_connections 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 1 -- 172.21.15.92:0/2772540108 wait complete. 2024-09-21T02:06:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.556+0000 7f47c02d0740 20 client.16089 trim_cache size 0 max 16384 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.569+0000 7f47c02d0740 1 Processor -- start 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.569+0000 7f47c02d0740 1 -- start start 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee22910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee1bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47c02d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41af550 con 0x5624fea71070 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47c02d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b1090 con 0x5624fea481b0 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47c02d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41afc20 con 0x5624fea50390 2024-09-21T02:06:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee20c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43516/0 (socket says 172.21.15.92:43516) 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 -- 172.21.15.92:0/1218516702 learned_addr learned my addr 172.21.15.92:0/1218516702 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b37fe640 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee22910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee1bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 -- 172.21.15.92:0/1218516702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 msgr2=0x5624fee1bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee1bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 -- 172.21.15.92:0/1218516702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 msgr2=0x5624fee22910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee22910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 -- 172.21.15.92:0/1218516702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624fee24ce0 con 0x5624fea71070 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee1bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47b3fff640 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee20c10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0188c0 tx=0x7f47ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344468799d3d1888 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.570+0000 7f47997fa640 1 -- 172.21.15.92:0/1218516702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac01e050 con 0x5624fea71070 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47997fa640 1 -- 172.21.15.92:0/1218516702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac028070 con 0x5624fea71070 2024-09-21T02:06:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47997fa640 1 -- 172.21.15.92:0/1218516702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac017620 con 0x5624fea71070 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 -- 172.21.15.92:0/1218516702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 msgr2=0x5624fee20c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee20c10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0188c0 tx=0x7f47ac014000 comp rx=0 tx=0).stop 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 -- 172.21.15.92:0/1218516702 shutdown_connections 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee20c10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee1bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 --2- 172.21.15.92:0/1218516702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee22910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 -- 172.21.15.92:0/1218516702 >> 172.21.15.92:0/1218516702 conn(0x5624fea65460 msgr2=0x5624fee1b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 -- 172.21.15.92:0/1218516702 shutdown_connections 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.571+0000 7f47c02d0740 1 -- 172.21.15.92:0/1218516702 wait complete. 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 -- start start 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 Processor -- start 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee2c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee27e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b09c0 con 0x5624fea71070 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b02f0 con 0x5624fea481b0 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ad530 con 0x5624fea50390 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47c02d0740 10 client.0 _ll_register_callbacks cb 0x7f47a80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee27e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.572+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee27e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43528/0 (socket says 172.21.15.92:43528) 2024-09-21T02:06:42.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 -- 172.21.15.92:0/3658299912 learned_addr learned my addr 172.21.15.92:0/3658299912 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b37fe640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee2c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 -- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 msgr2=0x5624fee278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 -- 172.21.15.92:0/3658299912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 msgr2=0x5624fee2c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee2c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 -- 172.21.15.92:0/3658299912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624fee28560 con 0x5624fea71070 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b2ffd640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b37fe640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee2c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47b3fff640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee27e20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009e60 tx=0x7f47ac023670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaf676cc0a431d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f4792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f4792ffd640 1 -- 172.21.15.92:0/3658299912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac032070 con 0x5624fea71070 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f4792ffd640 1 -- 172.21.15.92:0/3658299912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47ac02c3f0 con 0x5624fea71070 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ac028070 con 0x5624fea71070 2024-09-21T02:06:42.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f4792ffd640 1 -- 172.21.15.92:0/3658299912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47ac01e070 con 0x5624fea71070 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47c02d0740 10 client.16095 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.573+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624feea0550 con 0x5624fea71070 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f4792ffd640 1 -- 172.21.15.92:0/3658299912 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac028070 con 0x5624fea71070 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f4792ffd640 1 -- 172.21.15.92:0/3658299912 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47ac0173c0 con 0x5624fea71070 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47c02d0740 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47c02d0740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47c02d0740 20 client.16095 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47c02d0740 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea04da0 0x5624feea2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47c02d0740 4 client.16095 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624fed8c8a0 con 0x5624fea04da0 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.574+0000 7f47b37fe640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea04da0 0x5624feea2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f4792ffd640 10 client.16095 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47b37fe640 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea04da0 0x5624feea2ca0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f6a49ed189587a server_cookie=20a6bc5142908734 in_seq=0 out_seq=0 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f4792ffd640 1 -- 172.21.15.92:0/3658299912 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624fed8c8a0 con 0x5624fea04da0 2024-09-21T02:06:42.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f4792ffd640 10 client.16095 handle_command_reply: tid=0 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 client.16095 shutdown 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 msgr2=0x5624fee27e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee27e20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009e60 tx=0x7f47ac023670 comp rx=0 tx=0).stop 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 shutdown_connections 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5624fea04da0 0x5624feea2ca0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624fea71070 0x5624fee27e20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5624fea50390 0x5624fee278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 --2- 172.21.15.92:0/3658299912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624fea481b0 0x5624fee2c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 >> 172.21.15.92:0/3658299912 conn(0x5624fea65460 msgr2=0x5624fee24eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f4792ffd640 0 client.16095 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 shutdown_connections 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 1 -- 172.21.15.92:0/3658299912 wait complete. 2024-09-21T02:06:42.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.575+0000 7f47c02d0740 20 client.16095 trim_cache size 0 max 16384 2024-09-21T02:06:42.581 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:06:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f479c0b6510 msgr2=0x7f479c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f479c0b6510 0x7f479c0b8a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00a400 tx=0x7f47ac00d040 comp rx=0 tx=0).stop 2024-09-21T02:06:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 msgr2=0x7f47b419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 0x7f47b419aac0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f47a800d750 tx=0x7f47a800dc10 comp rx=0 tx=0).stop 2024-09-21T02:06:42.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 shutdown_connections 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f479c0b6510 0x7f479c0b8a20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b4109df0 0x7f47b419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b41094a0 0x7f47b419aac0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 --2- 172.21.15.92:0/2597147120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b4103b60 0x7f47b419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 >> 172.21.15.92:0/2597147120 conn(0x7f47b407b850 msgr2=0x7f47b4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 shutdown_connections 2024-09-21T02:06:42.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:42.576+0000 7f47b9d5c640 1 -- 172.21.15.92:0/2597147120 wait complete. 2024-09-21T02:06:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:42 smithi136 ceph-mon[34754]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:06:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:42 smithi099 ceph-mon[34404]: pgmap v412: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:06:44.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:06:44.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:44.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:44 smithi136 ceph-mon[34754]: pgmap v413: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:06:44.791 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:44 smithi092 ceph-mon[31190]: pgmap v413: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:06:44.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:44 smithi099 ceph-mon[34404]: pgmap v413: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:06:45.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 -- 172.21.15.92:0/765024856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb950102a60 msgr2=0x7fb950102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 --2- 172.21.15.92:0/765024856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb950102a60 0x7fb950102e70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009900 tx=0x7fb94c02a410 comp rx=0 tx=0).stop 2024-09-21T02:06:45.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 -- 172.21.15.92:0/765024856 shutdown_connections 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 --2- 172.21.15.92:0/765024856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb950108cf0 0x7fb95010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 --2- 172.21.15.92:0/765024856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9501083a0 0x7fb9501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 --2- 172.21.15.92:0/765024856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb950102a60 0x7fb950102e70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 -- 172.21.15.92:0/765024856 >> 172.21.15.92:0/765024856 conn(0x7fb950079730 msgr2=0x7fb950079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 -- 172.21.15.92:0/765024856 shutdown_connections 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.215+0000 7fb958465640 1 -- 172.21.15.92:0/765024856 wait complete. 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 Processor -- start 2024-09-21T02:06:45.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 -- start start 2024-09-21T02:06:45.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb950102a60 0x7fb95019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 0x7fb95019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb950108cf0 0x7fb95019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb950112960 con 0x7fb9501083a0 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501127e0 con 0x7fb950102a60 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb958465640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb950112ae0 con 0x7fb950108cf0 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb955a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 0x7fb95019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb955a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 0x7fb95019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58854/0 (socket says 172.21.15.92:58854) 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb955a00640 1 -- 172.21.15.92:0/1011484268 learned_addr learned my addr 172.21.15.92:0/1011484268 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb956201640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb950102a60 0x7fb95019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.216+0000 7fb956a02640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb950108cf0 0x7fb95019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb955a00640 1 -- 172.21.15.92:0/1011484268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb950108cf0 msgr2=0x7fb95019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb955a00640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb950108cf0 0x7fb95019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb955a00640 1 -- 172.21.15.92:0/1011484268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb950102a60 msgr2=0x7fb95019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb955a00640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb950102a60 0x7fb95019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb955a00640 1 -- 172.21.15.92:0/1011484268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9501a6560 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb955a00640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 0x7fb95019b810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d770 tx=0x7fb94000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb9477fe640 1 -- 172.21.15.92:0/1011484268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940017070 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb9477fe640 1 -- 172.21.15.92:0/1011484268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb940009c30 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9501a6850 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb9477fe640 1 -- 172.21.15.92:0/1011484268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94000f620 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.217+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501a6de0 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.220+0000 7fb9477fe640 1 -- 172.21.15.92:0/1011484268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb94001a070 con 0x7fb9501083a0 2024-09-21T02:06:45.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.220+0000 7fb95e9d9740 1 Processor -- start 2024-09-21T02:06:45.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.220+0000 7fb95e9d9740 1 -- start start 2024-09-21T02:06:45.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c85e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c48e490 0x55793c85e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c85eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55793c7d4e60 con 0x55793c496670 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55793c7c4290 con 0x55793c4b7350 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55793c7c80a0 con 0x55793c48e490 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c85eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb9477fe640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb93c0b65c0 0x7fb93c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c85eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58856/0 (socket says 172.21.15.92:58856) 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 -- 172.21.15.92:0/2622841841 learned_addr learned my addr 172.21.15.92:0/2622841841 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb956201640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c48e490 0x55793c85e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb956a02640 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c85e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb956a02640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb93c0b65c0 0x7fb93c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb9477fe640 1 -- 172.21.15.92:0/1011484268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb9400e6d10 con 0x7fb9501083a0 2024-09-21T02:06:45.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 -- 172.21.15.92:0/2622841841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c48e490 msgr2=0x55793c85e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c48e490 0x55793c85e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 -- 172.21.15.92:0/2622841841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 msgr2=0x55793c85e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c85e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 -- 172.21.15.92:0/2622841841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793c522930 con 0x55793c496670 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.221+0000 7fb955a00640 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c85eeb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb9400e47f0 tx=0x7fb940010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb489962e992eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.222+0000 7fb956a02640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb93c0b65c0 0x7fb93c0b8ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb948009fd0 tx=0x7fb948009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.222+0000 7fb944d9a640 1 -- 172.21.15.92:0/2622841841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940021070 con 0x55793c496670 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.222+0000 7fb944d9a640 1 -- 172.21.15.92:0/2622841841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9400b1960 con 0x55793c496670 2024-09-21T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.222+0000 7fb944d9a640 1 -- 172.21.15.92:0/2622841841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9400a8ae0 con 0x55793c496670 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2622841841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 msgr2=0x55793c85eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c85eeb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb9400e47f0 tx=0x7fb940010c00 comp rx=0 tx=0).stop 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2622841841 shutdown_connections 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c85eeb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c48e490 0x55793c85e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2622841841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c85e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2622841841 >> 172.21.15.92:0/2622841841 conn(0x55793c4ab740 msgr2=0x55793c85c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2622841841 shutdown_connections 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2622841841 wait complete. 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 -- start start 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 Processor -- start 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c86a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.223+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 0x55793c875e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55793c625890 con 0x55793c4b7350 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55793c75ed40 con 0x55793c48e490 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55793c86cb50 con 0x55793c496670 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb95e9d9740 10 client.0 _ll_register_callbacks cb 0x55793c7c2c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 0x55793c875e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb955a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c86a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 0x55793c875e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58862/0 (socket says 172.21.15.92:58862) 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb955a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c86a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:53972/0 (socket says 172.21.15.92:53972) 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 -- 172.21.15.92:0/3917457866 learned_addr learned my addr 172.21.15.92:0/3917457866 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956201640 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 -- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 msgr2=0x55793c86a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c86a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 -- 172.21.15.92:0/3917457866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 msgr2=0x55793c86a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 -- 172.21.15.92:0/3917457866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793c876430 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb956a02640 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 0x55793c875e40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb94800ad50 tx=0x7fb948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb7569c5c52771b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb939ffb640 1 -- 172.21.15.92:0/3917457866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94800ad80 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb939ffb640 1 -- 172.21.15.92:0/3917457866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb948004b20 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.224+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9400b1960 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb939ffb640 1 -- 172.21.15.92:0/3917457866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb948004d20 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 10 client.16107 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55793c878d60 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb939ffb640 1 -- 172.21.15.92:0/3917457866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9400b1960 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb939ffb640 1 -- 172.21.15.92:0/3917457866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb948020070 con 0x55793c4b7350 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:45.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 20 client.16107 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c475cd0 0x55793c87c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 4 client.16107 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55793c7d6db0 con 0x55793c475cd0 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.225+0000 7fb956201640 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c475cd0 0x55793c87c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.226+0000 7fb939ffb640 10 client.16107 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.226+0000 7fb956201640 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c475cd0 0x55793c87c330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146f960b82099870 server_cookie=6b26a1353daa6e79 in_seq=0 out_seq=0 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb939ffb640 1 -- 172.21.15.92:0/3917457866 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55793c7d6db0 con 0x55793c475cd0 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb939ffb640 10 client.16107 handle_command_reply: tid=0 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 client.16107 shutdown 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 msgr2=0x55793c875e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 0x55793c875e40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb94800ad50 tx=0x7fb948004a40 comp rx=0 tx=0).stop 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 shutdown_connections 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c475cd0 0x55793c87c330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c4b7350 0x55793c875e40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c86a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3917457866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 >> 172.21.15.92:0/3917457866 conn(0x55793c4ab740 msgr2=0x55793c861610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb939ffb640 0 client.16107 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 shutdown_connections 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3917457866 wait complete. 2024-09-21T02:06:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.227+0000 7fb95e9d9740 20 client.16107 trim_cache size 0 max 16384 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.244+0000 7fb95e9d9740 1 Processor -- start 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.244+0000 7fb95e9d9740 1 -- start start 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 0x55793c875d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c869570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c86ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ad2d0 con 0x55793c48e490 2024-09-21T02:06:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501aee10 con 0x55793c4b7350 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ad9a0 con 0x55793c496670 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 0x55793c875d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 0x55793c875d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58864/0 (socket says 172.21.15.92:58864) 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 -- 172.21.15.92:0/3056118337 learned_addr learned my addr 172.21.15.92:0/3056118337 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956a02640 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c86ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb955a00640 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c869570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 -- 172.21.15.92:0/3056118337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 msgr2=0x55793c869570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c869570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 -- 172.21.15.92:0/3056118337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 msgr2=0x55793c86ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c86ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 -- 172.21.15.92:0/3056118337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793c856520 con 0x55793c48e490 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb956201640 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 0x55793c875d40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb94c02a930 tx=0x7fb94c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8198d5eeaba932eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.245+0000 7fb93bfff640 1 -- 172.21.15.92:0/3056118337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c035040 con 0x55793c48e490 2024-09-21T02:06:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb93bfff640 1 -- 172.21.15.92:0/3056118337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb94c004400 con 0x55793c48e490 2024-09-21T02:06:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb93bfff640 1 -- 172.21.15.92:0/3056118337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0035f0 con 0x55793c48e490 2024-09-21T02:06:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3056118337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 msgr2=0x55793c875d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 0x55793c875d40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb94c02a930 tx=0x7fb94c031400 comp rx=0 tx=0).stop 2024-09-21T02:06:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3056118337 shutdown_connections 2024-09-21T02:06:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c4b7350 0x55793c86ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c496670 0x55793c869570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/3056118337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c48e490 0x55793c875d40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3056118337 >> 172.21.15.92:0/3056118337 conn(0x55793c4ab740 msgr2=0x55793c86b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3056118337 shutdown_connections 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.246+0000 7fb95e9d9740 1 -- 172.21.15.92:0/3056118337 wait complete. 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 -- start start 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 Processor -- start 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c86e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c4b7350 0x55793c86e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ae740 con 0x55793c496670 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ae070 con 0x55793c48e490 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ab2b0 con 0x55793c4b7350 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb95e9d9740 10 client.0 _ll_register_callbacks cb 0x7fb948014b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb955a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c86e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb955a00640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c86e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58880/0 (socket says 172.21.15.92:58880) 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb955a00640 1 -- 172.21.15.92:0/2542241596 learned_addr learned my addr 172.21.15.92:0/2542241596 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb956201640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.247+0000 7fb956a02640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c4b7350 0x55793c86e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb955a00640 1 -- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c4b7350 msgr2=0x55793c86e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb955a00640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c4b7350 0x55793c86e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb955a00640 1 -- 172.21.15.92:0/2542241596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 msgr2=0x55793c86ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb955a00640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb955a00640 1 -- 172.21.15.92:0/2542241596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793c8e5d40 con 0x55793c496670 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb955a00640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c86e310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb9400aa1a0 tx=0x7fb94000df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3defceafa10f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb939ffb640 1 -- 172.21.15.92:0/2542241596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9400d4070 con 0x55793c496670 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb939ffb640 1 -- 172.21.15.92:0/2542241596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb94005c170 con 0x55793c496670 2024-09-21T02:06:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb94c004400 con 0x55793c496670 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb939ffb640 1 -- 172.21.15.92:0/2542241596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb940021040 con 0x55793c496670 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb95e9d9740 10 client.16113 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55793c8e7000 con 0x55793c496670 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb939ffb640 1 -- 172.21.15.92:0/2542241596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb94c004400 con 0x55793c496670 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.248+0000 7fb939ffb640 1 -- 172.21.15.92:0/2542241596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb9400b4ce0 con 0x55793c496670 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb95e9d9740 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb95e9d9740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb95e9d9740 20 client.16113 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c44b080 0x55793c8e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb95e9d9740 4 client.16113 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55793c7d2860 con 0x55793c44b080 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb956201640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c44b080 0x55793c8e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb939ffb640 10 client.16113 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.249+0000 7fb956201640 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c44b080 0x55793c8e93a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da398868858b37f server_cookie=e7b32cdaa9c28676 in_seq=0 out_seq=0 2024-09-21T02:06:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb939ffb640 1 -- 172.21.15.92:0/2542241596 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55793c7d2860 con 0x55793c44b080 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb939ffb640 10 client.16113 handle_command_reply: tid=0 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 client.16113 shutdown 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 msgr2=0x55793c86e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c86e310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb9400aa1a0 tx=0x7fb94000df10 comp rx=0 tx=0).stop 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 shutdown_connections 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55793c44b080 0x55793c8e93a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55793c4b7350 0x55793c86e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55793c496670 0x55793c86e310 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 --2- 172.21.15.92:0/2542241596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55793c48e490 0x55793c86ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 >> 172.21.15.92:0/2542241596 conn(0x55793c4ab740 msgr2=0x55793c86ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb939ffb640 0 client.16113 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 shutdown_connections 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 1 -- 172.21.15.92:0/2542241596 wait complete. 2024-09-21T02:06:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.251+0000 7fb95e9d9740 20 client.16113 trim_cache size 0 max 16384 2024-09-21T02:06:45.256 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:06:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:06:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "c97f0c49-9c5d-499d-abe3-b0479f78dccd", 2024-09-21T02:06:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:06:45.256 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb93c0b65c0 msgr2=0x7fb93c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb93c0b65c0 0x7fb93c0b8ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb948009fd0 tx=0x7fb948009600 comp rx=0 tx=0).stop 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 msgr2=0x7fb95019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 0x7fb95019b810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d770 tx=0x7fb94000dc30 comp rx=0 tx=0).stop 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 shutdown_connections 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb93c0b65c0 0x7fb93c0b8ad0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb950108cf0 0x7fb95019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9501083a0 0x7fb95019b810 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 --2- 172.21.15.92:0/1011484268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb950102a60 0x7fb95019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 >> 172.21.15.92:0/1011484268 conn(0x7fb950079730 msgr2=0x7fb950101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 shutdown_connections 2024-09-21T02:06:45.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:45.252+0000 7fb958465640 1 -- 172.21.15.92:0/1011484268 wait complete. 2024-09-21T02:06:46.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:06:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:06:46.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:46 smithi092 ceph-mon[31190]: pgmap v414: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:06:46.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:46 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:06:46.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:46 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:46.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:46 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:06:46.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:46 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:46 smithi136 ceph-mon[34754]: pgmap v414: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:06:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:46 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:06:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:46 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:46 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:06:46.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:46 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:46 smithi099 ceph-mon[34404]: pgmap v414: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:06:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:46 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:06:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:46 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:46 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:06:46.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:46 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 -- 172.21.15.92:0/3989889897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 msgr2=0x7f47cc104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 --2- 172.21.15.92:0/3989889897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc104c90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009900 tx=0x7f47bc02a410 comp rx=0 tx=0).stop 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 -- 172.21.15.92:0/3989889897 shutdown_connections 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 --2- 172.21.15.92:0/3989889897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc10de30 0x7f47cc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 --2- 172.21.15.92:0/3989889897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc1052b0 0x7f47cc10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 --2- 172.21.15.92:0/3989889897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc104c90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 -- 172.21.15.92:0/3989889897 >> 172.21.15.92:0/3989889897 conn(0x7f47cc079740 msgr2=0x7f47cc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 -- 172.21.15.92:0/3989889897 shutdown_connections 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 -- 172.21.15.92:0/3989889897 wait complete. 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.886+0000 7f47d1291640 1 Processor -- start 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 -- start start 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc1052b0 0x7f47cc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc10de30 0x7f47cc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc113aa0 con 0x7f47cc104880 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc113920 con 0x7f47cc10de30 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47d1291640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc113c20 con 0x7f47cc1052b0 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58888/0 (socket says 172.21.15.92:58888) 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 -- 172.21.15.92:0/3937914625 learned_addr learned my addr 172.21.15.92:0/3937914625 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc1052b0 0x7f47cc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc10de30 0x7f47cc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 -- 172.21.15.92:0/3937914625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc1052b0 msgr2=0x7f47cc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc1052b0 0x7f47cc19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 -- 172.21.15.92:0/3937914625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc10de30 msgr2=0x7f47cc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc10de30 0x7f47cc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 -- 172.21.15.92:0/3937914625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc1a03c0 con 0x7f47cc104880 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.887+0000 7f47caffd640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc19a450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005940 tx=0x7f47bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.888+0000 7f47b3fff640 1 -- 172.21.15.92:0/3937914625 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc03f030 con 0x7f47cc104880 2024-09-21T02:06:46.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.888+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc1a0650 con 0x7f47cc104880 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.888+0000 7f47b3fff640 1 -- 172.21.15.92:0/3937914625 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc003cb0 con 0x7f47cc104880 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.888+0000 7f47b3fff640 1 -- 172.21.15.92:0/3937914625 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc035070 con 0x7f47cc104880 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.888+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1a6e10 con 0x7f47cc104880 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.890+0000 7f47d7805740 1 Processor -- start 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.890+0000 7f47d7805740 1 -- start start 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.890+0000 7f47b3fff640 1 -- 172.21.15.92:0/3937914625 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f47bc042030 con 0x7f47cc104880 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.890+0000 7f47d7805740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff3681650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.890+0000 7f47d7805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b11b0 0x558ff3681b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47d7805740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff36820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47d7805740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558ff35f7e50 con 0x558ff32b9390 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47d7805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff35e7280 con 0x558ff32b11b0 2024-09-21T02:06:46.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47d7805740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ff35eb090 con 0x558ff32da070 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff36820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47cb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff3681650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff36820d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58894/0 (socket says 172.21.15.92:58894) 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47cb7fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff3681650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:54016/0 (socket says 172.21.15.92:54016) 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 -- 172.21.15.92:0/2017210792 learned_addr learned my addr 172.21.15.92:0/2017210792 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47caffd640 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b11b0 0x558ff3681b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 -- 172.21.15.92:0/2017210792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 msgr2=0x558ff3681650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff3681650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 -- 172.21.15.92:0/2017210792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b11b0 msgr2=0x558ff3681b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b11b0 0x558ff3681b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 -- 172.21.15.92:0/2017210792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff3345650 con 0x558ff32b9390 2024-09-21T02:06:46.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47b3fff640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b40b6510 0x7f47b40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b40b6510 0x7f47b40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff36820d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f47c000bca0 tx=0x7f47c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12819a562b70f37e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47b3fff640 1 -- 172.21.15.92:0/3937914625 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47bc107bc0 con 0x7f47cc104880 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.891+0000 7f47b17fa640 1 -- 172.21.15.92:0/2017210792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c001c480 con 0x558ff32b9390 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47b17fa640 1 -- 172.21.15.92:0/2017210792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c001cad0 con 0x558ff32b9390 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47b17fa640 1 -- 172.21.15.92:0/2017210792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0011b20 con 0x558ff32b9390 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 -- 172.21.15.92:0/2017210792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 msgr2=0x558ff36820d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b40b6510 0x7f47b40b8a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005fd0 tx=0x7f47c40096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff36820d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f47c000bca0 tx=0x7f47c00107a0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 -- 172.21.15.92:0/2017210792 shutdown_connections 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff36820d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b11b0 0x558ff3681b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 --2- 172.21.15.92:0/2017210792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff3681650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.892+0000 7f47d7805740 1 -- 172.21.15.92:0/2017210792 >> 172.21.15.92:0/2017210792 conn(0x558ff32ce460 msgr2=0x558ff367f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 -- 172.21.15.92:0/2017210792 shutdown_connections 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 -- 172.21.15.92:0/2017210792 wait complete. 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 -- start start 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 Processor -- start 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32b11b0 0x558ff36910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff3691610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32da070 0x558ff3691b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558ff3448700 con 0x558ff32b9390 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff3582110 con 0x558ff32da070 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ff3694190 con 0x558ff32b11b0 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47d7805740 10 client.0 _ll_register_callbacks cb 0x558ff359b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:46.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.893+0000 7f47ca7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff3691610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff3691610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58896/0 (socket says 172.21.15.92:58896) 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 -- 172.21.15.92:0/1816736043 learned_addr learned my addr 172.21.15.92:0/1816736043 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32da070 0x558ff3691b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47caffd640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32b11b0 0x558ff36910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 -- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32b11b0 msgr2=0x558ff36910d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32b11b0 0x558ff36910d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 -- 172.21.15.92:0/1816736043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32da070 msgr2=0x558ff3691b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32da070 0x558ff3691b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 -- 172.21.15.92:0/1816736043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff35afc40 con 0x558ff32b9390 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff3691610 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f47c0011650 tx=0x7f47c000b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f4dd54a60a9019 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:46.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47aeff5640 1 -- 172.21.15.92:0/1816736043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c002b060 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47aeff5640 1 -- 172.21.15.92:0/1816736043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47c001c480 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c001cad0 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47aeff5640 1 -- 172.21.15.92:0/1816736043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47c001c680 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.894+0000 7f47d7805740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.895+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ff369fea0 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.895+0000 7f47aeff5640 1 -- 172.21.15.92:0/1816736043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c001cad0 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.895+0000 7f47aeff5640 1 -- 172.21.15.92:0/1816736043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47c0031520 con 0x558ff32b9390 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47d7805740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47d7805740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47d7805740 20 client.16125 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47d7805740 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff32989f0 0x558ff36a3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47d7805740 4 client.16125 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff35f9da0 con 0x558ff32989f0 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.896+0000 7f47caffd640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff32989f0 0x558ff36a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.897+0000 7f47aeff5640 10 client.16125 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.897+0000 7f47caffd640 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff32989f0 0x558ff36a3200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25657537ba860e1 server_cookie=4c3a66926b5ff2a3 in_seq=0 out_seq=0 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.897+0000 7f47aeff5640 1 -- 172.21.15.92:0/1816736043 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ff35f9da0 con 0x558ff32989f0 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.897+0000 7f47aeff5640 10 client.16125 handle_command_reply: tid=0 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 client.16125 shutdown 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 msgr2=0x558ff3691610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff3691610 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f47c0011650 tx=0x7f47c000b970 comp rx=0 tx=0).stop 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 shutdown_connections 2024-09-21T02:06:46.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff32989f0 0x558ff36a3200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32da070 0x558ff3691b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b9390 0x558ff3691610 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 --2- 172.21.15.92:0/1816736043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32b11b0 0x558ff36910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47aeff5640 0 client.16125 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 >> 172.21.15.92:0/1816736043 conn(0x558ff32ce460 msgr2=0x558ff367db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 shutdown_connections 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 1 -- 172.21.15.92:0/1816736043 wait complete. 2024-09-21T02:06:46.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.898+0000 7f47d7805740 20 client.16125 trim_cache size 0 max 16384 2024-09-21T02:06:46.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.915+0000 7f47d7805740 1 Processor -- start 2024-09-21T02:06:46.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.915+0000 7f47d7805740 1 -- start start 2024-09-21T02:06:46.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47d7805740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff3686e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47d7805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff368e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47d7805740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff368e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47d7805740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ad220 con 0x558ff32b11b0 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47d7805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1aed60 con 0x558ff32b9390 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47d7805740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ad8f0 con 0x558ff32da070 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff3686e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff3686e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58904/0 (socket says 172.21.15.92:58904) 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 -- 172.21.15.92:0/1295220616 learned_addr learned my addr 172.21.15.92:0/1295220616 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff368e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff368e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 -- 172.21.15.92:0/1295220616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 msgr2=0x558ff368e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff368e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 -- 172.21.15.92:0/1295220616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 msgr2=0x558ff368e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff368e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 -- 172.21.15.92:0/1295220616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff367fa80 con 0x558ff32b11b0 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff368e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47caffd640 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff3686e40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0ca300 tx=0x7f47bc10b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a217281eb456519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47b0ff9640 1 -- 172.21.15.92:0/1295220616 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc049040 con 0x558ff32b11b0 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47b0ff9640 1 -- 172.21.15.92:0/1295220616 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0cd840 con 0x558ff32b11b0 2024-09-21T02:06:46.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.916+0000 7f47b0ff9640 1 -- 172.21.15.92:0/1295220616 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc005d50 con 0x558ff32b11b0 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 -- 172.21.15.92:0/1295220616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 msgr2=0x558ff3686e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff3686e40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0ca300 tx=0x7f47bc10b6a0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 -- 172.21.15.92:0/1295220616 shutdown_connections 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff368e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff368e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 --2- 172.21.15.92:0/1295220616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff3686e40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 -- 172.21.15.92:0/1295220616 >> 172.21.15.92:0/1295220616 conn(0x558ff32ce460 msgr2=0x558ff3686730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 -- 172.21.15.92:0/1295220616 shutdown_connections 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.917+0000 7f47d7805740 1 -- 172.21.15.92:0/1295220616 wait complete. 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 -- start start 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 Processor -- start 2024-09-21T02:06:46.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff36a4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff36a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff36a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ae690 con 0x558ff32b11b0 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1adfc0 con 0x558ff32b9390 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ab200 con 0x558ff32da070 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47d7805740 10 client.0 _ll_register_callbacks cb 0x7f47bc10b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff36a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff36a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58906/0 (socket says 172.21.15.92:58906) 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 -- 172.21.15.92:0/2813796687 learned_addr learned my addr 172.21.15.92:0/2813796687 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff36a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff36a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 -- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 msgr2=0x558ff36a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff36a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 -- 172.21.15.92:0/2813796687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 msgr2=0x558ff36a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff36a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47caffd640 1 -- 172.21.15.92:0/2813796687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff370c980 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47cb7fe640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff36a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.918+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff36a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47caffd640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff36a4230 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f47bc1056f0 tx=0x7f47bc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7f27e1382a5c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47aeff5640 1 -- 172.21.15.92:0/2813796687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc049070 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47aeff5640 1 -- 172.21.15.92:0/2813796687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47bc005d50 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc0cd840 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47aeff5640 1 -- 172.21.15.92:0/2813796687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47bc0d2810 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 10 client.16131 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ff370db80 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47aeff5640 1 -- 172.21.15.92:0/2813796687 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0cd840 con 0x558ff32b11b0 2024-09-21T02:06:46.925 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47aeff5640 1 -- 172.21.15.92:0/2813796687 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47bc0c9fa0 con 0x558ff32b11b0 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 10 client.16131 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 20 client.16131 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff326dda0 0x558ff3710240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 4 client.16131 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.919+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ff35f5850 con 0x558ff326dda0 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff326dda0 0x558ff3710240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47aeff5640 10 client.16131 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47ca7fc640 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff326dda0 0x558ff3710240 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad87c4f12308ffda server_cookie=582d693bb9fb8064 in_seq=0 out_seq=0 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47aeff5640 1 -- 172.21.15.92:0/2813796687 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x558ff35f5850 con 0x558ff326dda0 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47aeff5640 10 client.16131 handle_command_reply: tid=0 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47d7805740 1 client.16131 shutdown 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 msgr2=0x558ff36a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.920+0000 7f47d7805740 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff36a4230 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f47bc1056f0 tx=0x7f47bc009650 comp rx=0 tx=0).stop 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 shutdown_connections 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ff326dda0 0x558ff3710240 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ff32da070 0x558ff36a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff32b9390 0x558ff36a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 --2- 172.21.15.92:0/2813796687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ff32b11b0 0x558ff36a4230 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47aeff5640 0 client.16131 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 >> 172.21.15.92:0/2813796687 conn(0x558ff32ce460 msgr2=0x558ff3682680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 shutdown_connections 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 1 -- 172.21.15.92:0/2813796687 wait complete. 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d7805740 20 client.16131 trim_cache size 0 max 16384 2024-09-21T02:06:46.927 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "c97f0c49-9c5d-499d-abe3-b0479f78dccd", 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:06:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:06:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:06:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:06:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:06:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:06:46.931 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:06:46.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b40b6510 msgr2=0x7f47b40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b40b6510 0x7f47b40b8a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005fd0 tx=0x7f47c40096e0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 msgr2=0x7f47cc19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc19a450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005940 tx=0x7f47bc0042f0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 shutdown_connections 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b40b6510 0x7f47b40b8a20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc10de30 0x7f47cc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc1052b0 0x7f47cc19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 --2- 172.21.15.92:0/3937914625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc104880 0x7f47cc19a450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 >> 172.21.15.92:0/3937914625 conn(0x7f47cc079740 msgr2=0x7f47cc100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 shutdown_connections 2024-09-21T02:06:46.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:06:46.921+0000 7f47d1291640 1 -- 172.21.15.92:0/3937914625 wait complete. 2024-09-21T02:06:47.770 INFO:tasks.cephfs.filesystem:scrub status for tag:c97f0c49-9c5d-499d-abe3-b0479f78dccd - {'path': '/', 'tag': 'c97f0c49-9c5d-499d-abe3-b0479f78dccd', 'options': 'recursive,force'} 2024-09-21T02:06:48.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:48 smithi136 ceph-mon[34754]: pgmap v415: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:48.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:48 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:06:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:48 smithi092 ceph-mon[31190]: pgmap v415: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:48 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:06:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:48 smithi099 ceph-mon[34404]: pgmap v415: 97 pgs: 97 active+clean; 1.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:48 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:06:50.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:50 smithi136 ceph-mon[34754]: pgmap v416: 97 pgs: 97 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:06:50.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:50 smithi092 ceph-mon[31190]: pgmap v416: 97 pgs: 97 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:06:50.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:50 smithi099 ceph-mon[34404]: pgmap v416: 97 pgs: 97 active+clean; 1.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:06:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:52 smithi136 ceph-mon[34754]: pgmap v417: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:52 smithi092 ceph-mon[31190]: pgmap v417: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:52.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:52 smithi099 ceph-mon[34404]: pgmap v417: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:06:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 -- 172.21.15.92:0/398881466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 msgr2=0x7f19601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 --2- 172.21.15.92:0/398881466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f19601021c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1950009900 tx=0x7f195002a410 comp rx=0 tx=0).stop 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 -- 172.21.15.92:0/398881466 shutdown_connections 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 --2- 172.21.15.92:0/398881466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1960103770 0x7f196010a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 --2- 172.21.15.92:0/398881466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960102dd0 0x7f19601031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 --2- 172.21.15.92:0/398881466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f19601021c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 -- 172.21.15.92:0/398881466 >> 172.21.15.92:0/398881466 conn(0x7f19600fd690 msgr2=0x7f19600ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 -- 172.21.15.92:0/398881466 shutdown_connections 2024-09-21T02:06:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.445+0000 7f1964cec640 1 -- 172.21.15.92:0/398881466 wait complete. 2024-09-21T02:06:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 Processor -- start 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 -- start start 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f1960198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1960102dd0 0x7f1960199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960103770 0x7f196019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e210 con 0x7f1960101d50 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e090 con 0x7f1960103770 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.446+0000 7f1964cec640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e390 con 0x7f1960102dd0 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f1960198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f1960198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48044/0 (socket says 172.21.15.92:48044) 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 -- 172.21.15.92:0/1934519780 learned_addr learned my addr 172.21.15.92:0/1934519780 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195dd9b640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1960102dd0 0x7f1960199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960103770 0x7f196019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 -- 172.21.15.92:0/1934519780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1960102dd0 msgr2=0x7f1960199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1960102dd0 0x7f1960199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 -- 172.21.15.92:0/1934519780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960103770 msgr2=0x7f196019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960103770 0x7f196019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 -- 172.21.15.92:0/1934519780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196019a8d0 con 0x7f1960101d50 2024-09-21T02:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960103770 0x7f196019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f195e59c640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f1960198e90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1950006010 tx=0x7f1950032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f19577fe640 1 -- 172.21.15.92:0/1934519780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195003f030 con 0x7f1960101d50 2024-09-21T02:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f19577fe640 1 -- 172.21.15.92:0/1934519780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950003aa0 con 0x7f1960101d50 2024-09-21T02:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196019ab60 con 0x7f1960101d50 2024-09-21T02:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f19577fe640 1 -- 172.21.15.92:0/1934519780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950035070 con 0x7f1960101d50 2024-09-21T02:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.447+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196019ae70 con 0x7f1960101d50 2024-09-21T02:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 Processor -- start 2024-09-21T02:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 -- start start 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f19577fe640 1 -- 172.21.15.92:0/1934519780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1950042030 con 0x7f1960101d50 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335ddd1b0 0x55b336184e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b3361854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b336186010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3360f76f0 con 0x55b335db42f0 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b3360e6b20 con 0x55b335ddd1b0 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f196bc58740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b3360ea930 con 0x55b335dbc4d0 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b3361854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b3361854c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48056/0 (socket says 172.21.15.92:48056) 2024-09-21T02:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.450+0000 7f195e59c640 1 -- 172.21.15.92:0/1742250739 learned_addr learned my addr 172.21.15.92:0/1742250739 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195dd9b640 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b336186010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335ddd1b0 0x55b336184e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195e59c640 1 -- 172.21.15.92:0/1742250739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 msgr2=0x55b336186010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195e59c640 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b336186010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195e59c640 1 -- 172.21.15.92:0/1742250739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335ddd1b0 msgr2=0x55b336184e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195e59c640 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335ddd1b0 0x55b336184e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195e59c640 1 -- 172.21.15.92:0/1742250739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3360ac530 con 0x55b335db42f0 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335ddd1b0 0x55b336184e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f19577fe640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f194c0b6510 0x7f194c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195e59c640 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b3361854c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1950105400 tx=0x7f195010fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8cda913490b608 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f194c0b6510 0x7f194c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f19577fe640 1 -- 172.21.15.92:0/1934519780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f19501079e0 con 0x7f1960101d50 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f1954ff9640 1 -- 172.21.15.92:0/1742250739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500ca1a0 con 0x55b335db42f0 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f1954ff9640 1 -- 172.21.15.92:0/1742250739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19500c92b0 con 0x55b335db42f0 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.451+0000 7f1954ff9640 1 -- 172.21.15.92:0/1742250739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500cd2c0 con 0x55b335db42f0 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 -- 172.21.15.92:0/1742250739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 msgr2=0x55b3361854c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f194c0b6510 0x7f194c0b8a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b3361854c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1950105400 tx=0x7f195010fe60 comp rx=0 tx=0).stop 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 -- 172.21.15.92:0/1742250739 shutdown_connections 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b336186010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b3361854c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 --2- 172.21.15.92:0/1742250739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335ddd1b0 0x55b336184e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 -- 172.21.15.92:0/1742250739 >> 172.21.15.92:0/1742250739 conn(0x55b335dd1300 msgr2=0x55b3361808e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 -- 172.21.15.92:0/1742250739 shutdown_connections 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.452+0000 7f196bc58740 1 -- 172.21.15.92:0/1742250739 wait complete. 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 -- start start 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 Processor -- start 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335db42f0 0x55b33618e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b33618e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 0x55b33618f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b33601eab0 con 0x55b335ddd1b0 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b33608c730 con 0x55b335db42f0 2024-09-21T02:06:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b3361868f0 con 0x55b335dbc4d0 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f196bc58740 10 client.0 _ll_register_callbacks cb 0x55b3360e06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f195ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 0x55b33618f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f195ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 0x55b33618f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48060/0 (socket says 172.21.15.92:48060) 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f195ed9d640 1 -- 172.21.15.92:0/972805398 learned_addr learned my addr 172.21.15.92:0/972805398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f195dd9b640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b33618e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.453+0000 7f195e59c640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335db42f0 0x55b33618e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f195ed9d640 1 -- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 msgr2=0x55b33618e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f195ed9d640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b33618e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f195ed9d640 1 -- 172.21.15.92:0/972805398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335db42f0 msgr2=0x55b33618e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f195ed9d640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335db42f0 0x55b33618e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f195ed9d640 1 -- 172.21.15.92:0/972805398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b335e48790 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f195ed9d640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 0x55b33618f480 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1958012a00 tx=0x7f1958004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e3de167a8564b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f1945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f1945ffb640 1 -- 172.21.15.92:0/972805398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195800fd60 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f1945ffb640 1 -- 172.21.15.92:0/972805398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f195801f420 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19500c92b0 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f1945ffb640 1 -- 172.21.15.92:0/972805398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f195802d070 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f196bc58740 10 client.16143 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b335ec1620 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f1945ffb640 1 -- 172.21.15.92:0/972805398 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19500c92b0 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.454+0000 7f1945ffb640 1 -- 172.21.15.92:0/972805398 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19580276a0 con 0x55b335ddd1b0 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f196bc58740 10 client.16143 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f196bc58740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f196bc58740 20 client.16143 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f196bc58740 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d9bd80 0x55b33619ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f196bc58740 4 client.16143 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3360f9640 con 0x55b335d9bd80 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f195e59c640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d9bd80 0x55b33619ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f195e59c640 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d9bd80 0x55b33619ba40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8460da15d39a5e0b server_cookie=ff2674746eb91f08 in_seq=0 out_seq=0 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.455+0000 7f1945ffb640 10 client.16143 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f1945ffb640 1 -- 172.21.15.92:0/972805398 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3360f9640 con 0x55b335d9bd80 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f1945ffb640 10 client.16143 handle_command_reply: tid=0 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 client.16143 shutdown 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 msgr2=0x55b33618f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 0x55b33618f480 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1958012a00 tx=0x7f1958004a40 comp rx=0 tx=0).stop 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 shutdown_connections 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d9bd80 0x55b33619ba40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335ddd1b0 0x55b33618f480 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f1945ffb640 0 client.16143 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335dbc4d0 0x55b33618e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 --2- 172.21.15.92:0/972805398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335db42f0 0x55b33618e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 >> 172.21.15.92:0/972805398 conn(0x55b335dd1300 msgr2=0x55b336185ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 shutdown_connections 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.456+0000 7f196bc58740 1 -- 172.21.15.92:0/972805398 wait complete. 2024-09-21T02:06:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.457+0000 7f196bc58740 20 client.16143 trim_cache size 0 max 16384 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 Processor -- start 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 -- start start 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b33617f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336187d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b336188d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19601a8f20 con 0x55b335db42f0 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19601aaa60 con 0x55b335dbc4d0 2024-09-21T02:06:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f196bc58740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19601a95f0 con 0x55b335ddd1b0 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b33617f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b33617f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48064/0 (socket says 172.21.15.92:48064) 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.471+0000 7f195e59c640 1 -- 172.21.15.92:0/1883770488 learned_addr learned my addr 172.21.15.92:0/1883770488 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b336188d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195e59c640 1 -- 172.21.15.92:0/1883770488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 msgr2=0x55b336188d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195e59c640 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b336188d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195e59c640 1 -- 172.21.15.92:0/1883770488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 msgr2=0x55b336187d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195e59c640 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336187d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195e59c640 1 -- 172.21.15.92:0/1883770488 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3361858c0 con 0x55b335db42f0 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195ed9d640 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b336188d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f195e59c640 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b33617f860 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1950105830 tx=0x7f195010ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af5f406a6fa8985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f1947fff640 1 -- 172.21.15.92:0/1883770488 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950049040 con 0x55b335db42f0 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f1947fff640 1 -- 172.21.15.92:0/1883770488 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195010f530 con 0x55b335db42f0 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.472+0000 7f1947fff640 1 -- 172.21.15.92:0/1883770488 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500cd4a0 con 0x55b335db42f0 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- 172.21.15.92:0/1883770488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 msgr2=0x55b33617f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b33617f860 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1950105830 tx=0x7f195010ff90 comp rx=0 tx=0).stop 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- 172.21.15.92:0/1883770488 shutdown_connections 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b336188d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336187d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- 172.21.15.92:0/1883770488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b33617f860 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- 172.21.15.92:0/1883770488 >> 172.21.15.92:0/1883770488 conn(0x55b335dd1300 msgr2=0x55b336188630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- 172.21.15.92:0/1883770488 shutdown_connections 2024-09-21T02:06:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- 172.21.15.92:0/1883770488 wait complete. 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- start start 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 Processor -- start 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b336195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336194c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b3361951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19601aa390 con 0x55b335db42f0 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19601a9cc0 con 0x55b335dbc4d0 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19601a6f00 con 0x55b335ddd1b0 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.473+0000 7f196bc58740 10 client.0 _ll_register_callbacks cb 0x7f1958009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b336195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b336195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48074/0 (socket says 172.21.15.92:48074) 2024-09-21T02:06:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 -- 172.21.15.92:0/2132672550 learned_addr learned my addr 172.21.15.92:0/2132672550 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195ed9d640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b3361951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195dd9b640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 -- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 msgr2=0x55b3361951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b3361951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 -- 172.21.15.92:0/2132672550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 msgr2=0x55b336194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 -- 172.21.15.92:0/2132672550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b336195910 con 0x55b335db42f0 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f195e59c640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b336195fd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f195010fd70 tx=0x7f195003b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4224b2f8e6322fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f1945ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f1945ffb640 1 -- 172.21.15.92:0/2132672550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950049040 con 0x55b335db42f0 2024-09-21T02:06:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f1945ffb640 1 -- 172.21.15.92:0/2132672550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f195010f940 con 0x55b335db42f0 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195010f530 con 0x55b335db42f0 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f1945ffb640 1 -- 172.21.15.92:0/2132672550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19500cd920 con 0x55b335db42f0 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f196bc58740 10 client.16149 fetch_fsmap learned FSMap version 29 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.474+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b336194760 con 0x55b335db42f0 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f1945ffb640 1 -- 172.21.15.92:0/2132672550 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195010f530 con 0x55b335db42f0 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f1945ffb640 1 -- 172.21.15.92:0/2132672550 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19500d21a0 con 0x55b335db42f0 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f196bc58740 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f196bc58740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f196bc58740 20 client.16149 populate_metadata read hostname 'smithi092' 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f196bc58740 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d711c0 0x55b336208a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f196bc58740 4 client.16149 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:06:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3360f50f0 con 0x55b335d711c0 2024-09-21T02:06:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.475+0000 7f195dd9b640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d711c0 0x55b336208a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.476+0000 7f1945ffb640 10 client.16149 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.476+0000 7f195dd9b640 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d711c0 0x55b336208a20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5650c4ba482f794 server_cookie=21dbac4cb2ab99a4 in_seq=0 out_seq=0 2024-09-21T02:06:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f1945ffb640 1 -- 172.21.15.92:0/2132672550 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3360f50f0 con 0x55b335d711c0 2024-09-21T02:06:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f1945ffb640 10 client.16149 handle_command_reply: tid=0 2024-09-21T02:06:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 client.16149 shutdown 2024-09-21T02:06:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 msgr2=0x55b336195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b336195fd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f195010fd70 tx=0x7f195003b640 comp rx=0 tx=0).stop 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 shutdown_connections 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b335d711c0 0x55b336208a20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b335ddd1b0 0x55b3361951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b335dbc4d0 0x55b336194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f1945ffb640 0 client.16149 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 --2- 172.21.15.92:0/2132672550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b335db42f0 0x55b336195fd0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 >> 172.21.15.92:0/2132672550 conn(0x55b335dd1300 msgr2=0x55b33617f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 shutdown_connections 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 1 -- 172.21.15.92:0/2132672550 wait complete. 2024-09-21T02:06:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.488+0000 7f196bc58740 20 client.16149 trim_cache size 0 max 16384 2024-09-21T02:06:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:06:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:06:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:06:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f194c0b6510 msgr2=0x7f194c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f194c0b6510 0x7f194c0b8a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958009500 comp rx=0 tx=0).stop 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 msgr2=0x7f1960198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f1960198e90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1950006010 tx=0x7f1950032f90 comp rx=0 tx=0).stop 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 shutdown_connections 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f194c0b6510 0x7f194c0b8a20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1960103770 0x7f196019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1960102dd0 0x7f1960199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 --2- 172.21.15.92:0/1934519780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1960101d50 0x7f1960198e90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 >> 172.21.15.92:0/1934519780 conn(0x7f19600fd690 msgr2=0x7f19601060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 shutdown_connections 2024-09-21T02:06:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:06:53.489+0000 7f1964cec640 1 -- 172.21.15.92:0/1934519780 wait complete. 2024-09-21T02:06:54.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:54 smithi136 ceph-mon[34754]: pgmap v418: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:06:54.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:54 smithi092 ceph-mon[31190]: pgmap v418: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:06:54.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:54 smithi099 ceph-mon[34404]: pgmap v418: 97 pgs: 97 active+clean; 1.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:06:56.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:56 smithi136 ceph-mon[34754]: pgmap v419: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:06:56.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:56 smithi092 ceph-mon[31190]: pgmap v419: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:06:56.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:56 smithi099 ceph-mon[34404]: pgmap v419: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:06:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:06:58 smithi136 ceph-mon[34754]: pgmap v420: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:58.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:06:58 smithi092 ceph-mon[31190]: pgmap v420: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:06:58.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:06:58 smithi099 ceph-mon[34404]: pgmap v420: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:00 smithi136 ceph-mon[34754]: pgmap v421: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:00.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:00 smithi092 ceph-mon[31190]: pgmap v421: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:00 smithi099 ceph-mon[34404]: pgmap v421: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:02.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:02 smithi136 ceph-mon[34754]: pgmap v422: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:02.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:02 smithi092 ceph-mon[31190]: pgmap v422: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:02.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:02 smithi099 ceph-mon[34404]: pgmap v422: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:04 smithi136 ceph-mon[34754]: pgmap v423: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:07:04.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:04 smithi092 ceph-mon[31190]: pgmap v423: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:07:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:04 smithi099 ceph-mon[34404]: pgmap v423: 97 pgs: 97 active+clean; 1.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:07:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 -- 172.21.15.92:0/3834218646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 msgr2=0x7ff96010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 --2- 172.21.15.92:0/3834218646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96010b920 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff958009ca0 tx=0x7ff95802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 -- 172.21.15.92:0/3834218646 shutdown_connections 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 --2- 172.21.15.92:0/3834218646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96010b920 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 --2- 172.21.15.92:0/3834218646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff960106b20 0x7ff960108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 --2- 172.21.15.92:0/3834218646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff960101a90 0x7ff960101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 -- 172.21.15.92:0/3834218646 >> 172.21.15.92:0/3834218646 conn(0x7ff9600fd6d0 msgr2=0x7ff9600ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 -- 172.21.15.92:0/3834218646 shutdown_connections 2024-09-21T02:07:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 -- 172.21.15.92:0/3834218646 wait complete. 2024-09-21T02:07:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 Processor -- start 2024-09-21T02:07:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.596+0000 7ff968190640 1 -- start start 2024-09-21T02:07:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff968190640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff960101a90 0x7ff960199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff968190640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff960106b20 0x7ff96019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff968190640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff968190640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff96010f320 con 0x7ff960109480 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff968190640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff96010f1a0 con 0x7ff960101a90 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff968190640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff96010f4a0 con 0x7ff960106b20 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96572b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff960106b20 0x7ff96019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45242/0 (socket says 172.21.15.92:45242) 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 -- 172.21.15.92:0/3590892135 learned_addr learned my addr 172.21.15.92:0/3590892135 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff965f2c640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff960101a90 0x7ff960199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 -- 172.21.15.92:0/3590892135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff960106b20 msgr2=0x7ff96019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff960106b20 0x7ff96019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 -- 172.21.15.92:0/3590892135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff960101a90 msgr2=0x7ff960199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff960101a90 0x7ff960199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 -- 172.21.15.92:0/3590892135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96019b950 con 0x7ff960109480 2024-09-21T02:07:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.597+0000 7ff96672d640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96019b130 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff9580097b0 tx=0x7ff958031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.598+0000 7ff956ffd640 1 -- 172.21.15.92:0/3590892135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958003bf0 con 0x7ff960109480 2024-09-21T02:07:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.598+0000 7ff956ffd640 1 -- 172.21.15.92:0/3590892135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff958003d90 con 0x7ff960109480 2024-09-21T02:07:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.598+0000 7ff956ffd640 1 -- 172.21.15.92:0/3590892135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95802fc30 con 0x7ff960109480 2024-09-21T02:07:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.598+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96019bbe0 con 0x7ff960109480 2024-09-21T02:07:06.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.598+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9601a2330 con 0x7ff960109480 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.600+0000 7ff96f1df740 1 Processor -- start 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff956ffd640 1 -- 172.21.15.92:0/3590892135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff958004040 con 0x7ff960109480 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 -- start start 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 0x56322a339e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a33a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a33b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56322a2ac6f0 con 0x563229f921b0 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56322a29bb20 con 0x563229f692f0 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96f1df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56322a29f930 con 0x563229f714d0 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96572b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 0x56322a339e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96572b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 0x56322a339e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45248/0 (socket says 172.21.15.92:45248) 2024-09-21T02:07:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff96572b640 1 -- 172.21.15.92:0/652473543 learned_addr learned my addr 172.21.15.92:0/652473543 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.601+0000 7ff965f2c640 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a33a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a33b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff956ffd640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff94c0b6510 0x7ff94c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 -- 172.21.15.92:0/652473543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 msgr2=0x56322a33b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a33b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 -- 172.21.15.92:0/652473543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 msgr2=0x56322a33a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a33a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 -- 172.21.15.92:0/652473543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322a261530 con 0x563229f921b0 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96672d640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff94c0b6510 0x7ff94c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff96572b640 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 0x56322a339e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff95000bbe0 tx=0x7ff9500042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b52090bc059da46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff956ffd640 1 -- 172.21.15.92:0/3590892135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff958108700 con 0x7ff960109480 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff94bfff640 1 -- 172.21.15.92:0/652473543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950005020 con 0x563229f921b0 2024-09-21T02:07:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff94bfff640 1 -- 172.21.15.92:0/652473543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff950005670 con 0x563229f921b0 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.602+0000 7ff94bfff640 1 -- 172.21.15.92:0/652473543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500106c0 con 0x563229f921b0 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 -- 172.21.15.92:0/652473543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 msgr2=0x56322a339e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96672d640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff94c0b6510 0x7ff94c0b8a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff9580ce800 tx=0x7ff9580f2040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 0x56322a339e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff95000bbe0 tx=0x7ff9500042b0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 -- 172.21.15.92:0/652473543 shutdown_connections 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a33b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a33a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 --2- 172.21.15.92:0/652473543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f921b0 0x56322a339e50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 -- 172.21.15.92:0/652473543 >> 172.21.15.92:0/652473543 conn(0x563229f86300 msgr2=0x56322a3358e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 -- 172.21.15.92:0/652473543 shutdown_connections 2024-09-21T02:07:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 -- 172.21.15.92:0/652473543 wait complete. 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.603+0000 7ff96f1df740 1 -- start start 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 Processor -- start 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 0x56322a343bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f921b0 0x56322a344720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56322a1d3ab0 con 0x563229f714d0 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56322a241730 con 0x563229f921b0 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56322a33bad0 con 0x563229f692f0 2024-09-21T02:07:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96f1df740 10 client.0 _ll_register_callbacks cb 0x56322a2956b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 0x56322a343bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 0x56322a343bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45256/0 (socket says 172.21.15.92:45256) 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 -- 172.21.15.92:0/4137437657 learned_addr learned my addr 172.21.15.92:0/4137437657 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96572b640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f921b0 0x56322a344720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff96572b640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 -- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f692f0 msgr2=0x56322a3434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 -- 172.21.15.92:0/4137437657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f921b0 msgr2=0x56322a344720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f921b0 0x56322a344720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.604+0000 7ff965f2c640 1 -- 172.21.15.92:0/4137437657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563229ffd790 con 0x563229f714d0 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96572b640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff965f2c640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 0x56322a343bb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff95c012840 tx=0x7ff95c012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94b1069b91c6c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff9497fa640 1 -- 172.21.15.92:0/4137437657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00f8a0 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff9497fa640 1 -- 172.21.15.92:0/4137437657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff95c00fa40 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff950005670 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff9497fa640 1 -- 172.21.15.92:0/4137437657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff95c00fc40 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 10 client.16161 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56322a076620 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff9497fa640 1 -- 172.21.15.92:0/4137437657 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff950005670 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff9497fa640 1 -- 172.21.15.92:0/4137437657 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff95c026420 con 0x563229f714d0 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 10 client.16161 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 20 client.16161 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f50d80 0x56322a350d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 4 client.16161 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.605+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56322a2ae640 con 0x563229f50d80 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.606+0000 7ff96572b640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f50d80 0x56322a350d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.606+0000 7ff9497fa640 10 client.16161 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.606+0000 7ff96572b640 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f50d80 0x56322a350d20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca941a80588b0dc server_cookie=1ff4f6b3970dce77 in_seq=0 out_seq=0 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff9497fa640 1 -- 172.21.15.92:0/4137437657 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56322a2ae640 con 0x563229f50d80 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff9497fa640 10 client.16161 handle_command_reply: tid=0 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 client.16161 shutdown 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 msgr2=0x56322a343bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 0x56322a343bb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff95c012840 tx=0x7ff95c012d00 comp rx=0 tx=0).stop 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 shutdown_connections 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f50d80 0x56322a350d20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f921b0 0x56322a344720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f714d0 0x56322a343bb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff9497fa640 0 client.16161 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4137437657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.607+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 >> 172.21.15.92:0/4137437657 conn(0x563229f86300 msgr2=0x56322a33a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.608+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 shutdown_connections 2024-09-21T02:07:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.608+0000 7ff96f1df740 1 -- 172.21.15.92:0/4137437657 wait complete. 2024-09-21T02:07:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.608+0000 7ff96f1df740 20 client.16161 trim_cache size 0 max 16384 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 Processor -- start 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 -- start start 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 0x56322a33ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a343a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601aaff0 con 0x563229f50d80 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601acb30 con 0x563229f692f0 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96f1df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601ab6c0 con 0x563229f714d0 2024-09-21T02:07:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96572b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 0x56322a33ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96572b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 0x56322a33ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45270/0 (socket says 172.21.15.92:45270) 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.622+0000 7ff96572b640 1 -- 172.21.15.92:0/1840236910 learned_addr learned my addr 172.21.15.92:0/1840236910 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff965f2c640 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff96572b640 1 -- 172.21.15.92:0/1840236910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 msgr2=0x56322a343a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff96572b640 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a343a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff96572b640 1 -- 172.21.15.92:0/1840236910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 msgr2=0x56322a3434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff96572b640 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff96572b640 1 -- 172.21.15.92:0/1840236910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322a34b9c0 con 0x563229f50d80 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff965f2c640 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff96572b640 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 0x56322a33ac90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff950005fd0 tx=0x7ff9500043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe0bcabb784fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff94b7fe640 1 -- 172.21.15.92:0/1840236910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950005040 con 0x563229f50d80 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff94b7fe640 1 -- 172.21.15.92:0/1840236910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff950005690 con 0x563229f50d80 2024-09-21T02:07:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.623+0000 7ff94b7fe640 1 -- 172.21.15.92:0/1840236910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9500106f0 con 0x563229f50d80 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 -- 172.21.15.92:0/1840236910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 msgr2=0x56322a33ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 0x56322a33ac90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff950005fd0 tx=0x7ff9500043e0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 -- 172.21.15.92:0/1840236910 shutdown_connections 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a343a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f692f0 0x56322a3434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 --2- 172.21.15.92:0/1840236910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f50d80 0x56322a33ac90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 -- 172.21.15.92:0/1840236910 >> 172.21.15.92:0/1840236910 conn(0x563229f86300 msgr2=0x56322a341a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 -- 172.21.15.92:0/1840236910 shutdown_connections 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 -- 172.21.15.92:0/1840236910 wait complete. 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 -- start start 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.624+0000 7ff96f1df740 1 Processor -- start 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f50d80 0x56322a34aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 0x56322a34b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a349a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601ac460 con 0x563229f692f0 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601abd90 con 0x563229f50d80 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff9601a8fd0 con 0x563229f714d0 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 10 client.0 _ll_register_callbacks cb 0x7ff95c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 0x56322a34b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 0x56322a34b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45286/0 (socket says 172.21.15.92:45286) 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 -- 172.21.15.92:0/4260787567 learned_addr learned my addr 172.21.15.92:0/4260787567 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96572b640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a349a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96572b640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f50d80 0x56322a34aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 -- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 msgr2=0x56322a349a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a349a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 -- 172.21.15.92:0/4260787567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f50d80 msgr2=0x56322a34aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f50d80 0x56322a34aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 -- 172.21.15.92:0/4260787567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56322a34a200 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96572b640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a349a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff965f2c640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 0x56322a34b360 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff95c012810 tx=0x7ff95c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d16b39ce691ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff9497fa640 1 -- 172.21.15.92:0/4260787567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c026a80 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff9497fa640 1 -- 172.21.15.92:0/4260787567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff95c026c20 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.625+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56322a2aa0f0 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff9497fa640 1 -- 172.21.15.92:0/4260787567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff95c037050 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 10 client.16167 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56322a349060 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff9497fa640 1 -- 172.21.15.92:0/4260787567 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56322a2aa0f0 con 0x563229f692f0 2024-09-21T02:07:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff9497fa640 1 -- 172.21.15.92:0/4260787567 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff95c003860 con 0x563229f692f0 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 20 client.16167 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f921b0 0x56322a3bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 4 client.16167 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56322a2a7520 con 0x563229f921b0 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.626+0000 7ff96572b640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f921b0 0x56322a3bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.627+0000 7ff9497fa640 10 client.16167 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.627+0000 7ff96572b640 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f921b0 0x56322a3bdc70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3623b8706db59cda server_cookie=8f93796377c16014 in_seq=0 out_seq=0 2024-09-21T02:07:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff9497fa640 1 -- 172.21.15.92:0/4260787567 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56322a2a7520 con 0x563229f921b0 2024-09-21T02:07:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff9497fa640 10 client.16167 handle_command_reply: tid=0 2024-09-21T02:07:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 client.16167 shutdown 2024-09-21T02:07:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 msgr2=0x56322a34b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 0x56322a34b360 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff95c012810 tx=0x7ff95c009f30 comp rx=0 tx=0).stop 2024-09-21T02:07:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 shutdown_connections 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563229f921b0 0x56322a3bdc70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563229f714d0 0x56322a349a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563229f692f0 0x56322a34b360 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff9497fa640 0 client.16167 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 --2- 172.21.15.92:0/4260787567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563229f50d80 0x56322a34aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 >> 172.21.15.92:0/4260787567 conn(0x563229f86300 msgr2=0x56322a33ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 shutdown_connections 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 1 -- 172.21.15.92:0/4260787567 wait complete. 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.639+0000 7ff96f1df740 20 client.16167 trim_cache size 0 max 16384 2024-09-21T02:07:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:07:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:07:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:07:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:07:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff94c0b6510 msgr2=0x7ff94c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff94c0b6510 0x7ff94c0b8a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff9580ce800 tx=0x7ff9580f2040 comp rx=0 tx=0).stop 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 msgr2=0x7ff96019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96019b130 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff9580097b0 tx=0x7ff958031840 comp rx=0 tx=0).stop 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 shutdown_connections 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff94c0b6510 0x7ff94c0b8a20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff960109480 0x7ff96019b130 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff960106b20 0x7ff96019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 --2- 172.21.15.92:0/3590892135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff960101a90 0x7ff960199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 >> 172.21.15.92:0/3590892135 conn(0x7ff9600fd6d0 msgr2=0x7ff9601078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 shutdown_connections 2024-09-21T02:07:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:06.640+0000 7ff968190640 1 -- 172.21.15.92:0/3590892135 wait complete. 2024-09-21T02:07:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:06 smithi136 ceph-mon[34754]: pgmap v424: 97 pgs: 97 active+clean; 1.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:06 smithi092 ceph-mon[31190]: pgmap v424: 97 pgs: 97 active+clean; 1.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:06.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:06 smithi099 ceph-mon[34404]: pgmap v424: 97 pgs: 97 active+clean; 1.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:08.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:08 smithi136 ceph-mon[34754]: pgmap v425: 97 pgs: 97 active+clean; 1.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:07:08.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:08 smithi092 ceph-mon[31190]: pgmap v425: 97 pgs: 97 active+clean; 1.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:07:08.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:08 smithi099 ceph-mon[34404]: pgmap v425: 97 pgs: 97 active+clean; 1.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:07:10.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:10 smithi136 ceph-mon[34754]: pgmap v426: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:10.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:10 smithi092 ceph-mon[31190]: pgmap v426: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:10.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:10 smithi099 ceph-mon[34404]: pgmap v426: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:12.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:12 smithi136 ceph-mon[34754]: pgmap v427: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:12.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:12 smithi092 ceph-mon[31190]: pgmap v427: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:12.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:12 smithi099 ceph-mon[34404]: pgmap v427: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:14.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:14 smithi136 ceph-mon[34754]: pgmap v428: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:14.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:14 smithi092 ceph-mon[31190]: pgmap v428: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:14.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:14 smithi099 ceph-mon[34404]: pgmap v428: 97 pgs: 97 active+clean; 1.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:16 smithi136 ceph-mon[34754]: pgmap v429: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:07:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:16 smithi092 ceph-mon[31190]: pgmap v429: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:07:16.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:16 smithi099 ceph-mon[34404]: pgmap v429: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:07:17.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:07:18.028 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:18.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 -- 172.21.15.92:0/3598228756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 msgr2=0x7f69b8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3598228756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8110620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009ca0 tx=0x7f69a802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 -- 172.21.15.92:0/3598228756 shutdown_connections 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3598228756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8110620 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3598228756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b81094a0 0x7f69b81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3598228756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69b8103b60 0x7f69b8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 -- 172.21.15.92:0/3598228756 >> 172.21.15.92:0/3598228756 conn(0x7f69b807b850 msgr2=0x7f69b807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 -- 172.21.15.92:0/3598228756 shutdown_connections 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.537+0000 7f69bfdca640 1 -- 172.21.15.92:0/3598228756 wait complete. 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 Processor -- start 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 -- start start 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69b8103b60 0x7f69b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b81094a0 0x7f69b8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8113b30 con 0x7f69b8109df0 2024-09-21T02:07:18.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81139b0 con 0x7f69b8103b60 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69bfdca640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8113cb0 con 0x7f69b81094a0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.538+0000 7f69be367640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47104/0 (socket says 172.21.15.92:47104) 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 -- 172.21.15.92:0/3259750190 learned_addr learned my addr 172.21.15.92:0/3259750190 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69bd365640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b81094a0 0x7f69b8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69bdb66640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69b8103b60 0x7f69b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 -- 172.21.15.92:0/3259750190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b81094a0 msgr2=0x7f69b8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b81094a0 0x7f69b8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 -- 172.21.15.92:0/3259750190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69b8103b60 msgr2=0x7f69b8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69b8103b60 0x7f69b8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 -- 172.21.15.92:0/3259750190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b81a87a0 con 0x7f69b8109df0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69be367640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8076190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005cc0 tx=0x7f69a802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69b6ffd640 1 -- 172.21.15.92:0/3259750190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8003bf0 con 0x7f69b8109df0 2024-09-21T02:07:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b81a8a30 con 0x7f69b8109df0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69b6ffd640 1 -- 172.21.15.92:0/3259750190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a8003d90 con 0x7f69b8109df0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69b6ffd640 1 -- 172.21.15.92:0/3259750190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a802fbc0 con 0x7f69b8109df0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.539+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b81a9040 con 0x7f69b8109df0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 Processor -- start 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69b6ffd640 1 -- 172.21.15.92:0/3259750190 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f69a802fd60 con 0x7f69b8109df0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 -- start start 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078982070 0x55e078d29650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d29b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078d2a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e078c9fe50 con 0x55e0789591b0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e078c8f280 con 0x55e078982070 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.542+0000 7f69c633e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e078c93090 con 0x55e078961390 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d29b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d29b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47114/0 (socket says 172.21.15.92:47114) 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 -- 172.21.15.92:0/1485409281 learned_addr learned my addr 172.21.15.92:0/1485409281 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bd365640 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078982070 0x55e078d29650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bd365640 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078d2a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69b6ffd640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f69a00b6510 0x7f69a00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 -- 172.21.15.92:0/1485409281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 msgr2=0x55e078d2a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078d2a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 -- 172.21.15.92:0/1485409281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078982070 msgr2=0x55e078d29650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078982070 0x55e078d29650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 -- 172.21.15.92:0/1485409281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0789ed650 con 0x55e0789591b0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69bdb66640 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d29b90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f69b000a8e0 tx=0x7f69b000ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4005f81e0fb25af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69b6ffd640 1 -- 172.21.15.92:0/3259750190 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f69a8108960 con 0x7f69b8109df0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.543+0000 7f69be367640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f69a00b6510 0x7f69a00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.544+0000 7f699ffff640 1 -- 172.21.15.92:0/1485409281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0015070 con 0x55e0789591b0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.544+0000 7f699ffff640 1 -- 172.21.15.92:0/1485409281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b00113f0 con 0x55e0789591b0 2024-09-21T02:07:18.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.544+0000 7f699ffff640 1 -- 172.21.15.92:0/1485409281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0018560 con 0x55e0789591b0 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.544+0000 7f69c633e740 1 -- 172.21.15.92:0/1485409281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 msgr2=0x55e078d29b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.544+0000 7f69be367640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f69a00b6510 0x7f69a00b8a20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f69a80cb0a0 tx=0x7f69a802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.544+0000 7f69c633e740 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d29b90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f69b000a8e0 tx=0x7f69b000ada0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 -- 172.21.15.92:0/1485409281 shutdown_connections 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078d2a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d29b90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 --2- 172.21.15.92:0/1485409281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078982070 0x55e078d29650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 -- 172.21.15.92:0/1485409281 >> 172.21.15.92:0/1485409281 conn(0x55e078976460 msgr2=0x55e078d27540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 -- 172.21.15.92:0/1485409281 shutdown_connections 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 -- 172.21.15.92:0/1485409281 wait complete. 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 -- start start 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.545+0000 7f69c633e740 1 Processor -- start 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d34ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078982070 0x55e078d35720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e078af0700 con 0x55e0789591b0 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e078c2a110 con 0x55e078961390 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e078d37e30 con 0x55e078982070 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69c633e740 10 client.0 _ll_register_callbacks cb 0x55e078c43990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d34ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d34ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47126/0 (socket says 172.21.15.92:47126) 2024-09-21T02:07:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 -- 172.21.15.92:0/107028673 learned_addr learned my addr 172.21.15.92:0/107028673 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bdb66640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078982070 0x55e078d35720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 -- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078982070 msgr2=0x55e078d35720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078982070 0x55e078d35720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 -- 172.21.15.92:0/107028673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 msgr2=0x55e078d351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 -- 172.21.15.92:0/107028673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e078c57c40 con 0x55e0789591b0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f69bd365640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d34ca0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f69ac004640 tx=0x7f69ac00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14551f99efc74706 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.546+0000 7f699d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f699d7fa640 1 -- 172.21.15.92:0/107028673 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0142d0 con 0x55e0789591b0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f699d7fa640 1 -- 172.21.15.92:0/107028673 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f69ac0047a0 con 0x55e0789591b0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b00113f0 con 0x55e0789591b0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f699d7fa640 1 -- 172.21.15.92:0/107028673 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69ac025070 con 0x55e0789591b0 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 10 client.16179 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:18.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e078d43ab0 con 0x55e0789591b0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f699d7fa640 1 -- 172.21.15.92:0/107028673 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b00113f0 con 0x55e0789591b0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f699d7fa640 1 -- 172.21.15.92:0/107028673 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69ac012610 con 0x55e0789591b0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 20 client.16179 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e0789409f0 0x55e078d46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 4 client.16179 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.547+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e078ca1da0 con 0x55e0789409f0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.548+0000 7f69bdb66640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e0789409f0 0x55e078d46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.548+0000 7f699d7fa640 10 client.16179 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.548+0000 7f69bdb66640 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e0789409f0 0x55e078d46f80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57052c32cd1e3f9a server_cookie=5eb0be13505c53dd in_seq=0 out_seq=0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f699d7fa640 1 -- 172.21.15.92:0/107028673 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e078ca1da0 con 0x55e0789409f0 2024-09-21T02:07:18.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f699d7fa640 10 client.16179 handle_command_reply: tid=0 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 client.16179 shutdown 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 msgr2=0x55e078d34ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d34ca0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f69ac004640 tx=0x7f69ac00cb50 comp rx=0 tx=0).stop 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 shutdown_connections 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e0789409f0 0x55e078d46f80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078982070 0x55e078d35720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 --2- 172.21.15.92:0/107028673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d34ca0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f699d7fa640 0 client.16179 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 >> 172.21.15.92:0/107028673 conn(0x55e078976460 msgr2=0x55e078d2c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 shutdown_connections 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 1 -- 172.21.15.92:0/107028673 wait complete. 2024-09-21T02:07:18.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.549+0000 7f69c633e740 20 client.16179 trim_cache size 0 max 16384 2024-09-21T02:07:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.566+0000 7f69c633e740 1 Processor -- start 2024-09-21T02:07:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.566+0000 7f69c633e740 1 -- start start 2024-09-21T02:07:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69c633e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0789409f0 0x55e078d2b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69c633e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69c633e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69c633e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81b1710 con 0x55e0789591b0 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69c633e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81b3250 con 0x55e078961390 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69c633e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81b1de0 con 0x55e0789409f0 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d368d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47140/0 (socket says 172.21.15.92:47140) 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 -- 172.21.15.92:0/2412488855 learned_addr learned my addr 172.21.15.92:0/2412488855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bd365640 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bd365640 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0789409f0 0x55e078d2b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 -- 172.21.15.92:0/2412488855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0789409f0 msgr2=0x55e078d2b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0789409f0 0x55e078d2b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 -- 172.21.15.92:0/2412488855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 msgr2=0x55e078d31e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d31e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 -- 172.21.15.92:0/2412488855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e078d2c4b0 con 0x55e0789591b0 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bd365640 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:18.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.567+0000 7f69bdb66640 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d368d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55e078d282d0 tx=0x7f69b000cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfb760a24199805 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f699f7fe640 1 -- 172.21.15.92:0/2412488855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0015040 con 0x55e0789591b0 2024-09-21T02:07:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f699f7fe640 1 -- 172.21.15.92:0/2412488855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b0011d30 con 0x55e0789591b0 2024-09-21T02:07:18.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f699f7fe640 1 -- 172.21.15.92:0/2412488855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0027680 con 0x55e0789591b0 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 -- 172.21.15.92:0/2412488855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 msgr2=0x55e078d368d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d368d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55e078d282d0 tx=0x7f69b000cb20 comp rx=0 tx=0).stop 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 -- 172.21.15.92:0/2412488855 shutdown_connections 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e078961390 0x55e078d31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d368d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 --2- 172.21.15.92:0/2412488855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0789409f0 0x55e078d2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 -- 172.21.15.92:0/2412488855 >> 172.21.15.92:0/2412488855 conn(0x55e078976460 msgr2=0x55e078d361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 -- 172.21.15.92:0/2412488855 shutdown_connections 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.568+0000 7f69c633e740 1 -- 172.21.15.92:0/2412488855 wait complete. 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 -- start start 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 Processor -- start 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0789409f0 0x55e078d397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d39d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078db0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81b2b80 con 0x55e0789591b0 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81b24b0 con 0x55e0789409f0 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.569+0000 7f69c633e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81af6f0 con 0x55e078961390 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69c633e740 10 client.0 _ll_register_callbacks cb 0x7f69ac00cd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d39d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d39d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47156/0 (socket says 172.21.15.92:47156) 2024-09-21T02:07:18.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 -- 172.21.15.92:0/2807589697 learned_addr learned my addr 172.21.15.92:0/2807589697 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bd365640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0789409f0 0x55e078d397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bd365640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078db0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 -- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 msgr2=0x55e078db0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078db0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 -- 172.21.15.92:0/2807589697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0789409f0 msgr2=0x55e078d397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0789409f0 0x55e078d397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 -- 172.21.15.92:0/2807589697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e078db0e90 con 0x55e0789591b0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bd365640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078db0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69bdb66640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d39d20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c520 tx=0x7f69b000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf33a690b340244 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f699d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f699d7fa640 1 -- 172.21.15.92:0/2807589697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0015070 con 0x55e0789591b0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f699d7fa640 1 -- 172.21.15.92:0/2807589697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f69b0006a70 con 0x55e0789591b0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b0011d30 con 0x55e0789591b0 2024-09-21T02:07:18.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f699d7fa640 1 -- 172.21.15.92:0/2807589697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69b00113f0 con 0x55e0789591b0 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69c633e740 10 client.16185 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.570+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e078db2120 con 0x55e0789591b0 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f699d7fa640 1 -- 172.21.15.92:0/2807589697 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b0011d30 con 0x55e0789591b0 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f699d7fa640 1 -- 172.21.15.92:0/2807589697 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f69b0024070 con 0x55e0789591b0 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69c633e740 10 client.16185 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69c633e740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69c633e740 20 client.16185 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69c633e740 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e078982070 0x55e078db44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69c633e740 4 client.16185 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e078c9d850 con 0x55e078982070 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.571+0000 7f69bd365640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e078982070 0x55e078db44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f699d7fa640 10 client.16185 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69bd365640 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e078982070 0x55e078db44e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4321cd25b3c10a8 server_cookie=383211cedd96cfc7 in_seq=0 out_seq=0 2024-09-21T02:07:18.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f699d7fa640 1 -- 172.21.15.92:0/2807589697 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x55e078c9d850 con 0x55e078982070 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f699d7fa640 10 client.16185 handle_command_reply: tid=0 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 client.16185 shutdown 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 msgr2=0x55e078d39d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d39d20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c520 tx=0x7f69b000cd20 comp rx=0 tx=0).stop 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 shutdown_connections 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e078982070 0x55e078db44e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e078961390 0x55e078db0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e0789591b0 0x55e078d39d20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 --2- 172.21.15.92:0/2807589697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0789409f0 0x55e078d397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f699d7fa640 0 client.16185 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 >> 172.21.15.92:0/2807589697 conn(0x55e078976460 msgr2=0x55e078d2ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 shutdown_connections 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.572+0000 7f69c633e740 1 -- 172.21.15.92:0/2807589697 wait complete. 2024-09-21T02:07:18.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69c633e740 20 client.16185 trim_cache size 0 max 16384 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:18.578 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:18.579 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:07:18.580 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:07:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f69a00b6510 msgr2=0x7f69a00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f69a00b6510 0x7f69a00b8a20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f69a80cb0a0 tx=0x7f69a802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 msgr2=0x7f69b8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8076190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005cc0 tx=0x7f69a802f380 comp rx=0 tx=0).stop 2024-09-21T02:07:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 shutdown_connections 2024-09-21T02:07:18.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f69a00b6510 0x7f69a00b8a20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69b8109df0 0x7f69b8076190 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b81094a0 0x7f69b8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 --2- 172.21.15.92:0/3259750190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69b8103b60 0x7f69b8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 >> 172.21.15.92:0/3259750190 conn(0x7f69b807b850 msgr2=0x7f69b81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 shutdown_connections 2024-09-21T02:07:18.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:18.573+0000 7f69bfdca640 1 -- 172.21.15.92:0/3259750190 wait complete. 2024-09-21T02:07:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:18 smithi136 ceph-mon[34754]: pgmap v430: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:18 smithi092 ceph-mon[31190]: pgmap v430: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:18.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:18 smithi099 ceph-mon[34404]: pgmap v430: 97 pgs: 97 active+clean; 1.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:19.368 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:07:19.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:07:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 -- 172.21.15.92:0/2061188978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9006c450 msgr2=0x7faf90106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 --2- 172.21.15.92:0/2061188978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9006c450 0x7faf90106310 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faf80009900 tx=0x7faf8002a410 comp rx=0 tx=0).stop 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 -- 172.21.15.92:0/2061188978 shutdown_connections 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 --2- 172.21.15.92:0/2061188978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf90109410 0x7faf9010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 --2- 172.21.15.92:0/2061188978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf90106850 0x7faf90108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 --2- 172.21.15.92:0/2061188978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf9006c450 0x7faf90106310 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 -- 172.21.15.92:0/2061188978 >> 172.21.15.92:0/2061188978 conn(0x7faf900fdf30 msgr2=0x7faf901003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 -- 172.21.15.92:0/2061188978 shutdown_connections 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.737+0000 7faf9724d640 1 -- 172.21.15.92:0/2061188978 wait complete. 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.738+0000 7faf9724d640 1 Processor -- start 2024-09-21T02:07:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.738+0000 7faf9724d640 1 -- start start 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.738+0000 7faf9724d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf9006c450 0x7faf9019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.738+0000 7faf9724d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 0x7faf9019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.738+0000 7faf9724d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf90109410 0x7faf9019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.738+0000 7faf9724d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf9010f340 con 0x7faf90106850 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf9724d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf9010f1c0 con 0x7faf9006c450 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf9724d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf9010f4c0 con 0x7faf90109410 2024-09-21T02:07:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 0x7faf9019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 0x7faf9019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47164/0 (socket says 172.21.15.92:47164) 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 -- 172.21.15.92:0/2824650989 learned_addr learned my addr 172.21.15.92:0/2824650989 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf94fe9640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf9006c450 0x7faf9019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf957ea640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf90109410 0x7faf9019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 -- 172.21.15.92:0/2824650989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf90109410 msgr2=0x7faf9019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf90109410 0x7faf9019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 -- 172.21.15.92:0/2824650989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf9006c450 msgr2=0x7faf9019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf9006c450 0x7faf9019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 -- 172.21.15.92:0/2824650989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9019ba60 con 0x7faf90106850 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8ffff640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 0x7faf9019a690 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9f0 tx=0x7faf8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8dffb640 1 -- 172.21.15.92:0/2824650989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400ccc0 con 0x7faf90106850 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9019bd50 con 0x7faf90106850 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8dffb640 1 -- 172.21.15.92:0/2824650989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840054d0 con 0x7faf90106850 2024-09-21T02:07:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.739+0000 7faf8dffb640 1 -- 172.21.15.92:0/2824650989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84010430 con 0x7faf90106850 2024-09-21T02:07:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.740+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf901a2440 con 0x7faf90106850 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf8dffb640 1 -- 172.21.15.92:0/2824650989 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7faf840046c0 con 0x7faf90106850 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 Processor -- start 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 -- start start 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd8130e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd8132010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561bd80a36f0 con 0x561bd7d684d0 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561bd8092b20 con 0x561bd7d891b0 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561bd8096930 con 0x561bd7d602f0 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd8132010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd8132010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47172/0 (socket says 172.21.15.92:47172) 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.742+0000 7faf8ffff640 1 -- 172.21.15.92:0/2702443236 learned_addr learned my addr 172.21.15.92:0/2702443236 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf94fe9640 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf957ea640 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd8130e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8dffb640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faf780b6510 0x7faf780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8ffff640 1 -- 172.21.15.92:0/2702443236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 msgr2=0x561bd81314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8ffff640 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8ffff640 1 -- 172.21.15.92:0/2702443236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 msgr2=0x561bd8130e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8ffff640 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd8130e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8ffff640 1 -- 172.21.15.92:0/2702443236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd8058530 con 0x561bd7d684d0 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf957ea640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faf780b6510 0x7faf780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8ffff640 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd8132010 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf840c2ad0 tx=0x7faf840ada30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea7b4026d9813ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf8dffb640 1 -- 172.21.15.92:0/2824650989 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faf840e76d0 con 0x7faf90106850 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf76ffd640 1 -- 172.21.15.92:0/2702443236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840ad610 con 0x561bd7d684d0 2024-09-21T02:07:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf76ffd640 1 -- 172.21.15.92:0/2702443236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840ad350 con 0x561bd7d684d0 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.743+0000 7faf76ffd640 1 -- 172.21.15.92:0/2702443236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840acfb0 con 0x561bd7d684d0 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 -- 172.21.15.92:0/2702443236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 msgr2=0x561bd8132010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd8132010 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf840c2ad0 tx=0x7faf840ada30 comp rx=0 tx=0).stop 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf957ea640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faf780b6510 0x7faf780b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf88005fd0 tx=0x7faf88004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 -- 172.21.15.92:0/2702443236 shutdown_connections 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd8132010 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/2702443236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd8130e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 -- 172.21.15.92:0/2702443236 >> 172.21.15.92:0/2702443236 conn(0x561bd7d7d300 msgr2=0x561bd812c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 -- 172.21.15.92:0/2702443236 shutdown_connections 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 -- 172.21.15.92:0/2702443236 wait complete. 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.744+0000 7faf9ddfd740 1 -- start start 2024-09-21T02:07:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 Processor -- start 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd813a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd813acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd813b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561bd7fcaab0 con 0x561bd7d684d0 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561bd8038730 con 0x561bd7d891b0 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561bd8132b30 con 0x561bd7d602f0 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf9ddfd740 10 client.0 _ll_register_callbacks cb 0x561bd808c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd813acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd813acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47178/0 (socket says 172.21.15.92:47178) 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 -- 172.21.15.92:0/1015961005 learned_addr learned my addr 172.21.15.92:0/1015961005 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf94fe9640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd813a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 -- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 msgr2=0x561bd813a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd813a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 -- 172.21.15.92:0/1015961005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 msgr2=0x561bd813b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf957ea640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd813b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd813b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.745+0000 7faf8ffff640 1 -- 172.21.15.92:0/1015961005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd7df4790 con 0x561bd7d684d0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf8ffff640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd813acb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faf840e5200 tx=0x7faf840a9060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48f655b7254db88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf6ffff640 1 -- 172.21.15.92:0/1015961005 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840ad190 con 0x561bd7d684d0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf6ffff640 1 -- 172.21.15.92:0/1015961005 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faf840adca0 con 0x561bd7d684d0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf840ad350 con 0x561bd7d684d0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf6ffff640 1 -- 172.21.15.92:0/1015961005 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf840ad610 con 0x561bd7d684d0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 10 client.16197 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bd7e6d620 con 0x561bd7d684d0 2024-09-21T02:07:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf6ffff640 1 -- 172.21.15.92:0/1015961005 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840ad350 con 0x561bd7d684d0 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf6ffff640 1 -- 172.21.15.92:0/1015961005 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf840b6dd0 con 0x561bd7d684d0 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 10 client.16197 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 20 client.16197 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d47d80 0x561bd8147900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 4 client.16197 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.746+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd80a5640 con 0x561bd7d47d80 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.747+0000 7faf94fe9640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d47d80 0x561bd8147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.747+0000 7faf6ffff640 10 client.16197 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.747+0000 7faf94fe9640 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d47d80 0x561bd8147900 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb144f4378a7e04 server_cookie=ec8f0a1d1389882b in_seq=0 out_seq=0 2024-09-21T02:07:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf6ffff640 1 -- 172.21.15.92:0/1015961005 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bd80a5640 con 0x561bd7d47d80 2024-09-21T02:07:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf6ffff640 10 client.16197 handle_command_reply: tid=0 2024-09-21T02:07:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 client.16197 shutdown 2024-09-21T02:07:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 msgr2=0x561bd813acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd813acb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faf840e5200 tx=0x7faf840a9060 comp rx=0 tx=0).stop 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 shutdown_connections 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d47d80 0x561bd8147900 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d891b0 0x561bd813b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd813acb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/1015961005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd813a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf6ffff640 0 client.16197 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 >> 172.21.15.92:0/1015961005 conn(0x561bd7d7d300 msgr2=0x561bd812aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 shutdown_connections 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 1 -- 172.21.15.92:0/1015961005 wait complete. 2024-09-21T02:07:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.748+0000 7faf9ddfd740 20 client.16197 trim_cache size 0 max 16384 2024-09-21T02:07:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.762+0000 7faf9ddfd740 1 Processor -- start 2024-09-21T02:07:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.762+0000 7faf9ddfd740 1 -- start start 2024-09-21T02:07:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d684d0 0x561bd8138b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 0x561bd8139080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ab160 con 0x561bd7d891b0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf901acca0 con 0x561bd7d684d0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ab830 con 0x561bd7d602f0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 0x561bd8139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 0x561bd8139080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47184/0 (socket says 172.21.15.92:47184) 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 -- 172.21.15.92:0/4267013150 learned_addr learned my addr 172.21.15.92:0/4267013150 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf94fe9640 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf8ffff640 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d684d0 0x561bd8138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 -- 172.21.15.92:0/4267013150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 msgr2=0x561bd81275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 -- 172.21.15.92:0/4267013150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d684d0 msgr2=0x561bd8138b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d684d0 0x561bd8138b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 -- 172.21.15.92:0/4267013150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd8147c70 con 0x561bd7d891b0 2024-09-21T02:07:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf8ffff640 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d684d0 0x561bd8138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf957ea640 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 0x561bd8139080 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faf88013810 tx=0x7faf88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5430c138c8f9626f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf767fc640 1 -- 172.21.15.92:0/4267013150 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88013030 con 0x561bd7d891b0 2024-09-21T02:07:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf767fc640 1 -- 172.21.15.92:0/4267013150 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf8801de30 con 0x561bd7d891b0 2024-09-21T02:07:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.763+0000 7faf767fc640 1 -- 172.21.15.92:0/4267013150 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88012d20 con 0x561bd7d891b0 2024-09-21T02:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 -- 172.21.15.92:0/4267013150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 msgr2=0x561bd8139080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 0x561bd8139080 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faf88013810 tx=0x7faf88014000 comp rx=0 tx=0).stop 2024-09-21T02:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 -- 172.21.15.92:0/4267013150 shutdown_connections 2024-09-21T02:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d891b0 0x561bd8139080 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d684d0 0x561bd8138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/4267013150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d602f0 0x561bd81275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 -- 172.21.15.92:0/4267013150 >> 172.21.15.92:0/4267013150 conn(0x561bd7d7d300 msgr2=0x561bd81397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 -- 172.21.15.92:0/4267013150 shutdown_connections 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 -- 172.21.15.92:0/4267013150 wait complete. 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.764+0000 7faf9ddfd740 1 -- start start 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 Processor -- start 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d602f0 0x561bd81ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd81ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d891b0 0x561bd81aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ac5d0 con 0x561bd7d684d0 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf901abf00 con 0x561bd7d602f0 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf901a9140 con 0x561bd7d891b0 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf9ddfd740 10 client.0 _ll_register_callbacks cb 0x7faf840ed900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd81ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd81ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47200/0 (socket says 172.21.15.92:47200) 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 -- 172.21.15.92:0/683428677 learned_addr learned my addr 172.21.15.92:0/683428677 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf957ea640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d891b0 0x561bd81aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 -- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d891b0 msgr2=0x561bd81aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d891b0 0x561bd81aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 -- 172.21.15.92:0/683428677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d602f0 msgr2=0x561bd81ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf94fe9640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d602f0 0x561bd81ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d602f0 0x561bd81ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf8ffff640 1 -- 172.21.15.92:0/683428677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd81af2f0 con 0x561bd7d684d0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.765+0000 7faf957ea640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d891b0 0x561bd81aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf8ffff640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd81ae020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faf840ad880 tx=0x7faf840a9d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c551b7dbde9152 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf6ffff640 1 -- 172.21.15.92:0/683428677 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840ad190 con 0x561bd7d684d0 2024-09-21T02:07:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf6ffff640 1 -- 172.21.15.92:0/683428677 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faf840e7b30 con 0x561bd7d684d0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf8801de30 con 0x561bd7d684d0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf6ffff640 1 -- 172.21.15.92:0/683428677 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf840b2120 con 0x561bd7d684d0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 10 client.16203 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bd81b6620 con 0x561bd7d684d0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf6ffff640 1 -- 172.21.15.92:0/683428677 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf8801de30 con 0x561bd7d684d0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf6ffff640 1 -- 172.21.15.92:0/683428677 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf840ed150 con 0x561bd7d684d0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 10 client.16203 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 20 client.16203 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d1d1c0 0x561bd81b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 4 client.16203 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.766+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bd80a10f0 con 0x561bd7d1d1c0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.767+0000 7faf94fe9640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d1d1c0 0x561bd81b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.767+0000 7faf6ffff640 10 client.16203 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.767+0000 7faf94fe9640 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d1d1c0 0x561bd81b8d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f5cd52351ff22c server_cookie=817700c751ce5edd in_seq=0 out_seq=0 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf6ffff640 1 -- 172.21.15.92:0/683428677 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bd80a10f0 con 0x561bd7d1d1c0 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf6ffff640 10 client.16203 handle_command_reply: tid=0 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 client.16203 shutdown 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 msgr2=0x561bd81ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd81ae020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faf840ad880 tx=0x7faf840a9d30 comp rx=0 tx=0).stop 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 shutdown_connections 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561bd7d1d1c0 0x561bd81b8d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561bd7d891b0 0x561bd81aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf6ffff640 0 client.16203 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bd7d684d0 0x561bd81ae020 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 --2- 172.21.15.92:0/683428677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561bd7d602f0 0x561bd81ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 >> 172.21.15.92:0/683428677 conn(0x561bd7d7d300 msgr2=0x561bd81275c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 shutdown_connections 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 1 -- 172.21.15.92:0/683428677 wait complete. 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.779+0000 7faf9ddfd740 20 client.16203 trim_cache size 0 max 16384 2024-09-21T02:07:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:07:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:07:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:07:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:07:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faf780b6510 msgr2=0x7faf780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faf780b6510 0x7faf780b8a20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf88005fd0 tx=0x7faf88004a40 comp rx=0 tx=0).stop 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 msgr2=0x7faf9019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 0x7faf9019a690 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9f0 tx=0x7faf8400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 shutdown_connections 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faf780b6510 0x7faf780b8a20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf90109410 0x7faf9019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf90106850 0x7faf9019a690 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 --2- 172.21.15.92:0/2824650989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf9006c450 0x7faf9019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 >> 172.21.15.92:0/2824650989 conn(0x7faf900fdf30 msgr2=0x7faf90104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 shutdown_connections 2024-09-21T02:07:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:19.780+0000 7faf9724d640 1 -- 172.21.15.92:0/2824650989 wait complete. 2024-09-21T02:07:20.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 -- 172.21.15.92:0/74692697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3281094d0 msgr2=0x7ff3281098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/74692697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3281094d0 0x7ff3281098e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff318009900 tx=0x7ff31802a410 comp rx=0 tx=0).stop 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 -- 172.21.15.92:0/74692697 shutdown_connections 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/74692697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff328109e20 0x7ff328110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/74692697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3281094d0 0x7ff3281098e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/74692697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 0x7ff328103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 -- 172.21.15.92:0/74692697 >> 172.21.15.92:0/74692697 conn(0x7ff3280ff600 msgr2=0x7ff328101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 -- 172.21.15.92:0/74692697 shutdown_connections 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.107+0000 7ff32ee9b640 1 -- 172.21.15.92:0/74692697 wait complete. 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 Processor -- start 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 -- start start 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 0x7ff32819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3281094d0 0x7ff32819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 0x7ff32819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff328113ae0 con 0x7ff328109e20 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff328113960 con 0x7ff3281094d0 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.108+0000 7ff32ee9b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff328113c60 con 0x7ff328103b90 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 0x7ff32819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 0x7ff32819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47222/0 (socket says 172.21.15.92:47222) 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 -- 172.21.15.92:0/3493302357 learned_addr learned my addr 172.21.15.92:0/3493302357 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32cc37640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 0x7ff32819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff31ffff640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3281094d0 0x7ff32819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 -- 172.21.15.92:0/3493302357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 msgr2=0x7ff32819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 0x7ff32819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 -- 172.21.15.92:0/3493302357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3281094d0 msgr2=0x7ff32819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3281094d0 0x7ff32819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 -- 172.21.15.92:0/3493302357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3281a0340 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32cc37640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 0x7ff32819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32d438640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 0x7ff32819fcd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff32000cad0 tx=0x7ff32000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000ae20 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3281a0630 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff320005020 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000f6b0 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.109+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3281a0af0 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.110+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff328103fa0 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.113+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff3200051d0 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.114+0000 7ff31dffb640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3100b6540 0x7ff3100b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.114+0000 7ff32cc37640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3100b6540 0x7ff3100b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.114+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff3200e77d0 con 0x7ff328109e20 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.114+0000 7ff32cc37640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3100b6540 0x7ff3100b8a50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff32400abe0 tx=0x7ff32400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.117+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff320014030 con 0x7ff328109e20 2024-09-21T02:07:20.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.332+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff32819b5c0 con 0x7ff328109e20 2024-09-21T02:07:20.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.332+0000 7ff31dffb640 1 -- 172.21.15.92:0/3493302357 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7ff32819b5c0 con 0x7ff328109e20 2024-09-21T02:07:20.334 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:07:20.334 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:07:20.334 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3100b6540 msgr2=0x7ff3100b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3100b6540 0x7ff3100b8a50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff32400abe0 tx=0x7ff32400a5c0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 msgr2=0x7ff32819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 0x7ff32819fcd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff32000cad0 tx=0x7ff32000cf90 comp rx=0 tx=0).stop 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 shutdown_connections 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3100b6540 0x7ff3100b8a50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff328109e20 0x7ff32819fcd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3281094d0 0x7ff32819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 --2- 172.21.15.92:0/3493302357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff328103b90 0x7ff32819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 >> 172.21.15.92:0/3493302357 conn(0x7ff3280ff600 msgr2=0x7ff328101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 shutdown_connections 2024-09-21T02:07:20.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:20.335+0000 7ff32ee9b640 1 -- 172.21.15.92:0/3493302357 wait complete. 2024-09-21T02:07:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:20 smithi136 ceph-mon[34754]: pgmap v431: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:07:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:20 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3493302357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:07:20.786 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:20 smithi092 ceph-mon[31190]: pgmap v431: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:07:20.786 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:20 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3493302357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:07:20.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:20 smithi099 ceph-mon[34404]: pgmap v431: 97 pgs: 97 active+clean; 2.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:07:20.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:20 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3493302357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:07:20.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:07:21.243 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.758+0000 7fe34d43f640 1 -- 172.21.15.92:0/489068109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348102a60 msgr2=0x7fe348102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.758+0000 7fe34d43f640 1 --2- 172.21.15.92:0/489068109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348102a60 0x7fe348102e70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe338009ca0 tx=0x7fe33802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 -- 172.21.15.92:0/489068109 shutdown_connections 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 --2- 172.21.15.92:0/489068109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe348108cf0 0x7fe34810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 --2- 172.21.15.92:0/489068109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3481083a0 0x7fe3481087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 --2- 172.21.15.92:0/489068109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348102a60 0x7fe348102e70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 -- 172.21.15.92:0/489068109 >> 172.21.15.92:0/489068109 conn(0x7fe348079730 msgr2=0x7fe348079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 -- 172.21.15.92:0/489068109 shutdown_connections 2024-09-21T02:07:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 -- 172.21.15.92:0/489068109 wait complete. 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 Processor -- start 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.759+0000 7fe34d43f640 1 -- start start 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe34d43f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe348102a60 0x7fe34819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe34d43f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3481083a0 0x7fe34819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe34d43f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 0x7fe34819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe34d43f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe348113b40 con 0x7fe348108cf0 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe34d43f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481139c0 con 0x7fe348102a60 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe34d43f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe348113cc0 con 0x7fe3481083a0 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 0x7fe34819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 0x7fe34819fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47240/0 (socket says 172.21.15.92:47240) 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 -- 172.21.15.92:0/2089842383 learned_addr learned my addr 172.21.15.92:0/2089842383 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe346ffd640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe348102a60 0x7fe34819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3467fc640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3481083a0 0x7fe34819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 -- 172.21.15.92:0/2089842383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3481083a0 msgr2=0x7fe34819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3481083a0 0x7fe34819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 -- 172.21.15.92:0/2089842383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe348102a60 msgr2=0x7fe34819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe348102a60 0x7fe34819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3477fe640 1 -- 172.21.15.92:0/2089842383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3481a05a0 con 0x7fe348108cf0 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.760+0000 7fe3467fc640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3481083a0 0x7fe34819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.761+0000 7fe3477fe640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 0x7fe34819fe70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe340009fd0 tx=0x7fe34000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.761+0000 7fe32ffff640 1 -- 172.21.15.92:0/2089842383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340011830 con 0x7fe348108cf0 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.761+0000 7fe32ffff640 1 -- 172.21.15.92:0/2089842383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe340016070 con 0x7fe348108cf0 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.761+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3481a0890 con 0x7fe348108cf0 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.761+0000 7fe32ffff640 1 -- 172.21.15.92:0/2089842383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340010de0 con 0x7fe348108cf0 2024-09-21T02:07:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.761+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481a6ff0 con 0x7fe348108cf0 2024-09-21T02:07:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.763+0000 7fe3539b3740 1 Processor -- start 2024-09-21T02:07:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe32ffff640 1 -- 172.21.15.92:0/2089842383 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe3400119d0 con 0x7fe348108cf0 2024-09-21T02:07:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 -- start start 2024-09-21T02:07:21.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df716fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df716fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df716fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55df71670ea0 con 0x55df71332390 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55df716602d0 con 0x55df7132a1b0 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.764+0000 7fe3539b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55df716640e0 con 0x55df71353070 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df716fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df716fb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47252/0 (socket says 172.21.15.92:47252) 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 -- 172.21.15.92:0/3741166539 learned_addr learned my addr 172.21.15.92:0/3741166539 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe346ffd640 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df716fabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df716fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe32ffff640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3300b6510 0x7fe3300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 -- 172.21.15.92:0/3741166539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 msgr2=0x55df716fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df716fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 -- 172.21.15.92:0/3741166539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 msgr2=0x55df716fabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df716fabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 -- 172.21.15.92:0/3741166539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df713be650 con 0x55df71332390 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe346ffd640 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df716fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3477fe640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3300b6510 0x7fe3300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe32ffff640 1 -- 172.21.15.92:0/2089842383 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe3400e68b0 con 0x7fe348108cf0 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe3467fc640 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df716fb120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe33c016550 tx=0x7fe33c00a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e750a9e3a44445ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.765+0000 7fe32d7fa640 1 -- 172.21.15.92:0/3741166539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c018430 con 0x55df71332390 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe32d7fa640 1 -- 172.21.15.92:0/3741166539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c018a80 con 0x55df71332390 2024-09-21T02:07:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe32d7fa640 1 -- 172.21.15.92:0/3741166539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c017870 con 0x55df71332390 2024-09-21T02:07:21.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 -- 172.21.15.92:0/3741166539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 msgr2=0x55df716fb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3477fe640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3300b6510 0x7fe3300b8a20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe34000bde0 tx=0x7fe3400d0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df716fb120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe33c016550 tx=0x7fe33c00a310 comp rx=0 tx=0).stop 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 -- 172.21.15.92:0/3741166539 shutdown_connections 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df716fb120 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df716fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 --2- 172.21.15.92:0/3741166539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df716fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.766+0000 7fe3539b3740 1 -- 172.21.15.92:0/3741166539 >> 172.21.15.92:0/3741166539 conn(0x55df71347460 msgr2=0x55df716f8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 -- 172.21.15.92:0/3741166539 shutdown_connections 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 -- 172.21.15.92:0/3741166539 wait complete. 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 -- start start 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 Processor -- start 2024-09-21T02:07:21.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df7132a1b0 0x55df7170a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df7170a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 0x55df717164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55df714c1700 con 0x55df71353070 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55df715fb160 con 0x55df71332390 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55df7170d150 con 0x55df7132a1b0 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3539b3740 10 client.0 _ll_register_callbacks cb 0x55df716149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 0x55df717164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 0x55df717164e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47268/0 (socket says 172.21.15.92:47268) 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.767+0000 7fe3467fc640 1 -- 172.21.15.92:0/4287853085 learned_addr learned my addr 172.21.15.92:0/4287853085 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df7132a1b0 0x55df7170a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe346ffd640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df7170a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 -- 172.21.15.92:0/4287853085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df7132a1b0 msgr2=0x55df7170a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df7132a1b0 0x55df7170a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 -- 172.21.15.92:0/4287853085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 msgr2=0x55df7170a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df7170a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 -- 172.21.15.92:0/4287853085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df71628c90 con 0x55df71353070 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe346ffd640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df7170a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3467fc640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 0x55df717164e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x55df716f90d0 tx=0x7fe33c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4882f100f72e7e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe32aff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe32aff5640 1 -- 172.21.15.92:0/4287853085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c017c60 con 0x55df71353070 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe32aff5640 1 -- 172.21.15.92:0/4287853085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe33c017e00 con 0x55df71353070 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33c018a80 con 0x55df71353070 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe32aff5640 1 -- 172.21.15.92:0/4287853085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe33c018430 con 0x55df71353070 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3539b3740 10 client.16221 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.768+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df71718f30 con 0x55df71353070 2024-09-21T02:07:21.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe32aff5640 1 -- 172.21.15.92:0/4287853085 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c018a80 con 0x55df71353070 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe32aff5640 1 -- 172.21.15.92:0/4287853085 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe33c012050 con 0x55df71353070 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe3539b3740 10 client.16221 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe3539b3740 10 client.16221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe3539b3740 20 client.16221 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe3539b3740 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df713119f0 0x55df7171c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe3539b3740 4 client.16221 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df71672df0 con 0x55df713119f0 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe346ffd640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df713119f0 0x55df7171c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe32aff5640 10 client.16221 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.769+0000 7fe346ffd640 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df713119f0 0x55df7171c290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b08f11a2bf64446a server_cookie=2daa478355d1a8a7 in_seq=0 out_seq=0 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe32aff5640 1 -- 172.21.15.92:0/4287853085 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df71672df0 con 0x55df713119f0 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe32aff5640 10 client.16221 handle_command_reply: tid=0 2024-09-21T02:07:21.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 client.16221 shutdown 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 msgr2=0x55df717164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 0x55df717164e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x55df716f90d0 tx=0x7fe33c006f90 comp rx=0 tx=0).stop 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 shutdown_connections 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df713119f0 0x55df7171c290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71353070 0x55df717164e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df7170a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 --2- 172.21.15.92:0/4287853085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df7132a1b0 0x55df7170a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 >> 172.21.15.92:0/4287853085 conn(0x55df71347460 msgr2=0x55df716f6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe32aff5640 0 client.16221 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 shutdown_connections 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 1 -- 172.21.15.92:0/4287853085 wait complete. 2024-09-21T02:07:21.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.770+0000 7fe3539b3740 20 client.16221 trim_cache size 0 max 16384 2024-09-21T02:07:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 Processor -- start 2024-09-21T02:07:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 -- start start 2024-09-21T02:07:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 0x55df7170c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df716fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df71716450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ad300 con 0x55df7132a1b0 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481aee40 con 0x55df71332390 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3539b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ad9d0 con 0x55df71353070 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 0x55df7170c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 0x55df7170c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47272/0 (socket says 172.21.15.92:47272) 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 -- 172.21.15.92:0/2992793907 learned_addr learned my addr 172.21.15.92:0/2992793907 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df71716450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe346ffd640 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df716fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 -- 172.21.15.92:0/2992793907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 msgr2=0x55df71716450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df71716450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 -- 172.21.15.92:0/2992793907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 msgr2=0x55df716fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df716fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe3467fc640 1 -- 172.21.15.92:0/2992793907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7170ce30 con 0x55df7132a1b0 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.784+0000 7fe346ffd640 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df716fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.785+0000 7fe3467fc640 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 0x55df7170c080 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55df7170a2e0 tx=0x7fe33c00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317b5f22ef591f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.785+0000 7fe32cff9640 1 -- 172.21.15.92:0/2992793907 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c041070 con 0x55df7132a1b0 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.785+0000 7fe32cff9640 1 -- 172.21.15.92:0/2992793907 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c02b070 con 0x55df7132a1b0 2024-09-21T02:07:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.785+0000 7fe32cff9640 1 -- 172.21.15.92:0/2992793907 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c017190 con 0x55df7132a1b0 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- 172.21.15.92:0/2992793907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 msgr2=0x55df7170c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 0x55df7170c080 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55df7170a2e0 tx=0x7fe33c00f040 comp rx=0 tx=0).stop 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- 172.21.15.92:0/2992793907 shutdown_connections 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df71716450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df71332390 0x55df716fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- 172.21.15.92:0/2992793907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df7132a1b0 0x55df7170c080 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- 172.21.15.92:0/2992793907 >> 172.21.15.92:0/2992793907 conn(0x55df71347460 msgr2=0x55df716fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- 172.21.15.92:0/2992793907 shutdown_connections 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- 172.21.15.92:0/2992793907 wait complete. 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- start start 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 Processor -- start 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df7177a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df7177aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df7177b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ae770 con 0x55df71332390 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.786+0000 7fe3539b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ae0a0 con 0x55df7132a1b0 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe3539b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481ab300 con 0x55df71353070 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe3539b3740 10 client.0 _ll_register_callbacks cb 0x7fe3380093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df7177aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df7177aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47284/0 (socket says 172.21.15.92:47284) 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 -- 172.21.15.92:0/1730902083 learned_addr learned my addr 172.21.15.92:0/1730902083 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe3467fc640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df7177a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe3467fc640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df7177b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 -- 172.21.15.92:0/1730902083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 msgr2=0x55df7177b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df7177b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 -- 172.21.15.92:0/1730902083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 msgr2=0x55df7177a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df7177a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 -- 172.21.15.92:0/1730902083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df71785a50 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe346ffd640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df7177aee0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe338009ca0 tx=0x7fe3380315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1960c0d058a2c974 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe32aff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe32aff5640 1 -- 172.21.15.92:0/1730902083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338003b70 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe32aff5640 1 -- 172.21.15.92:0/1730902083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe338003d10 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.787+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33c02b070 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe32aff5640 1 -- 172.21.15.92:0/1730902083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe338004030 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 10 client.16227 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df71786c70 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe32aff5640 1 -- 172.21.15.92:0/1730902083 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c02b070 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe32aff5640 1 -- 172.21.15.92:0/1730902083 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe33802fa80 con 0x55df71332390 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 10 client.16227 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 10 client.16227 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 20 client.16227 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df712e6da0 0x55df71789500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 4 client.16227 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df7166e8a0 con 0x55df712e6da0 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.788+0000 7fe3467fc640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df712e6da0 0x55df71789500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe32aff5640 10 client.16227 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3467fc640 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df712e6da0 0x55df71789500 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d9f5e0f5d34e815 server_cookie=cf9c620a0962da56 in_seq=0 out_seq=0 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe32aff5640 1 -- 172.21.15.92:0/1730902083 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df7166e8a0 con 0x55df712e6da0 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe32aff5640 10 client.16227 handle_command_reply: tid=0 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 client.16227 shutdown 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 msgr2=0x55df7177aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df7177aee0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe338009ca0 tx=0x7fe3380315b0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 shutdown_connections 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55df712e6da0 0x55df71789500 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55df71353070 0x55df7177b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df71332390 0x55df7177aee0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 --2- 172.21.15.92:0/1730902083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df7132a1b0 0x55df7177a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 >> 172.21.15.92:0/1730902083 conn(0x55df71347460 msgr2=0x55df7170b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe32aff5640 0 client.16227 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 shutdown_connections 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 1 -- 172.21.15.92:0/1730902083 wait complete. 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.789+0000 7fe3539b3740 20 client.16227 trim_cache size 0 max 16384 2024-09-21T02:07:21.795 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:07:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3300b6510 msgr2=0x7fe3300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3300b6510 0x7fe3300b8a20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe34000bde0 tx=0x7fe3400d0040 comp rx=0 tx=0).stop 2024-09-21T02:07:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 msgr2=0x7fe34819fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 0x7fe34819fe70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe340009fd0 tx=0x7fe34000b8e0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 shutdown_connections 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe3300b6510 0x7fe3300b8a20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe348108cf0 0x7fe34819fe70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3481083a0 0x7fe34819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 --2- 172.21.15.92:0/2089842383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe348102a60 0x7fe34819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 >> 172.21.15.92:0/2089842383 conn(0x7fe348079730 msgr2=0x7fe348101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 shutdown_connections 2024-09-21T02:07:21.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:21.790+0000 7fe34d43f640 1 -- 172.21.15.92:0/2089842383 wait complete. 2024-09-21T02:07:22.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:07:22.714 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:22 smithi136 ceph-mon[34754]: pgmap v432: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:07:22.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:22 smithi092 ceph-mon[31190]: pgmap v432: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:07:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:22 smithi099 ceph-mon[34404]: pgmap v432: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:07:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 -- 172.21.15.92:0/879512769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108d70 msgr2=0x7f4dec10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 --2- 172.21.15.92:0/879512769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108d70 0x7f4dec10f5a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009ca0 tx=0x7f4de402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 -- 172.21.15.92:0/879512769 shutdown_connections 2024-09-21T02:07:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 --2- 172.21.15.92:0/879512769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108d70 0x7f4dec10f5a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 --2- 172.21.15.92:0/879512769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dec108420 0x7f4dec108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 --2- 172.21.15.92:0/879512769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dec102ae0 0x7f4dec102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 -- 172.21.15.92:0/879512769 >> 172.21.15.92:0/879512769 conn(0x7f4dec079770 msgr2=0x7f4dec079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 -- 172.21.15.92:0/879512769 shutdown_connections 2024-09-21T02:07:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.217+0000 7f4df3b55640 1 -- 172.21.15.92:0/879512769 wait complete. 2024-09-21T02:07:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.218+0000 7f4df3b55640 1 Processor -- start 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.218+0000 7f4df3b55640 1 -- start start 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df3b55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dec102ae0 0x7f4dec19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df3b55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 0x7f4dec19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df3b55640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dec108d70 0x7f4dec1a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df3b55640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec113b50 con 0x7f4dec108420 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df3b55640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec1139d0 con 0x7f4dec102ae0 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df3b55640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec113cd0 con 0x7f4dec108d70 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 0x7f4dec19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 0x7f4dec19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52324/0 (socket says 172.21.15.92:52324) 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 -- 172.21.15.92:0/1944876248 learned_addr learned my addr 172.21.15.92:0/1944876248 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df20f2640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dec108d70 0x7f4dec1a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df18f1640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dec102ae0 0x7f4dec19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 -- 172.21.15.92:0/1944876248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dec108d70 msgr2=0x7f4dec1a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dec108d70 0x7f4dec1a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 -- 172.21.15.92:0/1944876248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dec102ae0 msgr2=0x7f4dec19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dec102ae0 0x7f4dec19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 -- 172.21.15.92:0/1944876248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec1a2750 con 0x7f4dec108420 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.219+0000 7f4df10f0640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 0x7f4dec19cd80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c970 tx=0x7f4de800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.220+0000 7f4ddeffd640 1 -- 172.21.15.92:0/1944876248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8007bb0 con 0x7f4dec108420 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.220+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec1a2a40 con 0x7f4dec108420 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.220+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dec1a91a0 con 0x7f4dec108420 2024-09-21T02:07:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.220+0000 7f4ddeffd640 1 -- 172.21.15.92:0/1944876248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de8007d50 con 0x7f4dec108420 2024-09-21T02:07:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.221+0000 7f4ddeffd640 1 -- 172.21.15.92:0/1944876248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004e10 con 0x7f4dec108420 2024-09-21T02:07:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 Processor -- start 2024-09-21T02:07:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 -- start start 2024-09-21T02:07:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4ddeffd640 1 -- 172.21.15.92:0/1944876248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4de8005020 con 0x7f4dec108420 2024-09-21T02:07:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 0x5561f36e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5561f3658ea0 con 0x5561f33121b0 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5561f36482d0 con 0x5561f333b070 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561f364c0e0 con 0x5561f331a390 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4df18f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 0x5561f36e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4df18f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 0x5561f36e2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52330/0 (socket says 172.21.15.92:52330) 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4df18f1640 1 -- 172.21.15.92:0/2910038921 learned_addr learned my addr 172.21.15.92:0/2910038921 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.223+0000 7f4df20f2640 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df10f0640 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df18f1640 1 -- 172.21.15.92:0/2910038921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 msgr2=0x5561f36e2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df18f1640 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36e2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df18f1640 1 -- 172.21.15.92:0/2910038921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 msgr2=0x5561f36e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df18f1640 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df18f1640 1 -- 172.21.15.92:0/2910038921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f33a6650 con 0x5561f33121b0 2024-09-21T02:07:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df18f1640 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 0x5561f36e2800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4de000bce0 tx=0x7f4de000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2e6bc9b0af99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4dd7fff640 1 -- 172.21.15.92:0/2910038921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0015070 con 0x5561f33121b0 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4ddeffd640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4dd80b6590 0x7f4dd80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4dd7fff640 1 -- 172.21.15.92:0/2910038921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de001a910 con 0x5561f33121b0 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4dd7fff640 1 -- 172.21.15.92:0/2910038921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011a70 con 0x5561f33121b0 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4df20f2640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4dd80b6590 0x7f4dd80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.224+0000 7f4ddeffd640 1 -- 172.21.15.92:0/1944876248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4de80e6dd0 con 0x7f4dec108420 2024-09-21T02:07:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2910038921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 msgr2=0x5561f36e2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4df20f2640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4dd80b6590 0x7f4dd80b8aa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4de4006010 tx=0x7f4de402f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 0x5561f36e2800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4de000bce0 tx=0x7f4de000f730 comp rx=0 tx=0).stop 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2910038921 shutdown_connections 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36e2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f33121b0 0x5561f36e2800 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2910038921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2910038921 >> 172.21.15.92:0/2910038921 conn(0x5561f332f460 msgr2=0x5561f36e01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2910038921 shutdown_connections 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.225+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2910038921 wait complete. 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 -- start start 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 Processor -- start 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36ed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5561f34a9700 con 0x5561f333b070 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5561f35e3160 con 0x5561f33121b0 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561f36f0aa0 con 0x5561f331a390 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4dfa0c9740 10 client.0 _ll_register_callbacks cb 0x5561f35fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.226+0000 7f4df20f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36ee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52346/0 (socket says 172.21.15.92:52346) 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 -- 172.21.15.92:0/2556865547 learned_addr learned my addr 172.21.15.92:0/2556865547 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df10f0640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df18f1640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36ed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 -- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 msgr2=0x5561f36ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 -- 172.21.15.92:0/2556865547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 msgr2=0x5561f36ed8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36ed8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 -- 172.21.15.92:0/2556865547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f3610c90 con 0x5561f333b070 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4df20f2640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36ee360 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4de4031660 tx=0x7f4de402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b21d4b9a7a40fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dd57fa640 1 -- 172.21.15.92:0/2556865547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4003e00 con 0x5561f333b070 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dd57fa640 1 -- 172.21.15.92:0/2556865547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4de4004040 con 0x5561f333b070 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de001a910 con 0x5561f333b070 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dd57fa640 1 -- 172.21.15.92:0/2556865547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4de4004240 con 0x5561f333b070 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dfa0c9740 10 client.16239 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.227+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561f36fc740 con 0x5561f333b070 2024-09-21T02:07:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dd57fa640 1 -- 172.21.15.92:0/2556865547 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de001a910 con 0x5561f333b070 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dd57fa640 1 -- 172.21.15.92:0/2556865547 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4de4051070 con 0x5561f333b070 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dfa0c9740 10 client.16239 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dfa0c9740 10 client.16239 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dfa0c9740 20 client.16239 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32f99f0 0x5561f36ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dfa0c9740 4 client.16239 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f365adf0 con 0x5561f32f99f0 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.228+0000 7f4df18f1640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32f99f0 0x5561f36ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.229+0000 7f4dd57fa640 10 client.16239 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.229+0000 7f4df18f1640 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32f99f0 0x5561f36ffdc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88f0a380b93c5ca7 server_cookie=8e12f6a745b51e8c in_seq=0 out_seq=0 2024-09-21T02:07:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.229+0000 7f4dd57fa640 1 -- 172.21.15.92:0/2556865547 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561f365adf0 con 0x5561f32f99f0 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.229+0000 7f4dd57fa640 10 client.16239 handle_command_reply: tid=0 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 client.16239 shutdown 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 msgr2=0x5561f36ee360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36ee360 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4de4031660 tx=0x7f4de402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 shutdown_connections 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36ee360 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32f99f0 0x5561f36ffdc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/2556865547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36ed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dd57fa640 0 client.16239 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 >> 172.21.15.92:0/2556865547 conn(0x5561f332f460 msgr2=0x5561f36e5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 shutdown_connections 2024-09-21T02:07:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/2556865547 wait complete. 2024-09-21T02:07:23.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.230+0000 7f4dfa0c9740 20 client.16239 trim_cache size 0 max 16384 2024-09-21T02:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 Processor -- start 2024-09-21T02:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 -- start start 2024-09-21T02:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f33121b0 0x5561f36ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 0x5561f36df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd000e510 con 0x5561f331a390 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0010050 con 0x5561f333b070 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd000ebe0 con 0x5561f33121b0 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4df10f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 0x5561f36df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4df10f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 0x5561f36df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52352/0 (socket says 172.21.15.92:52352) 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.243+0000 7f4df10f0640 1 -- 172.21.15.92:0/1968561634 learned_addr learned my addr 172.21.15.92:0/1968561634 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df10f0640 1 -- 172.21.15.92:0/1968561634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f33121b0 msgr2=0x5561f36ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df18f1640 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f33121b0 0x5561f36ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df10f0640 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f33121b0 0x5561f36ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df10f0640 1 -- 172.21.15.92:0/1968561634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 msgr2=0x5561f36dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df10f0640 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df10f0640 1 -- 172.21.15.92:0/1968561634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f36e45b0 con 0x5561f331a390 2024-09-21T02:07:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4df10f0640 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 0x5561f36df760 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4de80a9510 tx=0x7f4de8004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7354869d1752feed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4dd77fe640 1 -- 172.21.15.92:0/1968561634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de801d070 con 0x5561f331a390 2024-09-21T02:07:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4dd77fe640 1 -- 172.21.15.92:0/1968561634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de80aca50 con 0x5561f331a390 2024-09-21T02:07:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.244+0000 7f4dd77fe640 1 -- 172.21.15.92:0/1968561634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80940c0 con 0x5561f331a390 2024-09-21T02:07:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/1968561634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 msgr2=0x5561f36df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 0x5561f36df760 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4de80a9510 tx=0x7f4de8004fb0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/1968561634 shutdown_connections 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f333b070 0x5561f36dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f331a390 0x5561f36df760 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/1968561634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f33121b0 0x5561f36ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/1968561634 >> 172.21.15.92:0/1968561634 conn(0x5561f332f460 msgr2=0x5561f36e5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/1968561634 shutdown_connections 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/1968561634 wait complete. 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 -- start start 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.245+0000 7f4dfa0c9740 1 Processor -- start 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd000f980 con 0x5561f333b070 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd000f2b0 con 0x5561f33121b0 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd000c4f0 con 0x5561f331a390 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4dfa0c9740 10 client.0 _ll_register_callbacks cb 0x7f4de4009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52358/0 (socket says 172.21.15.92:52358) 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 -- 172.21.15.92:0/875513051 learned_addr learned my addr 172.21.15.92:0/875513051 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df18f1640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df10f0640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 -- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 msgr2=0x5561f36f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 -- 172.21.15.92:0/875513051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 msgr2=0x5561f36f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df20f2640 1 -- 172.21.15.92:0/875513051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f36f33b0 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df18f1640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.246+0000 7f4df10f0640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4df20f2640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36f2dc0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4de4005e30 tx=0x7f4de40095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f292c071385836 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dd57fa640 1 -- 172.21.15.92:0/875513051 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4031450 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de80aca50 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dd57fa640 1 -- 172.21.15.92:0/875513051 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4de40045f0 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dd57fa640 1 -- 172.21.15.92:0/875513051 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4de40047f0 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 10 client.16245 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561f376b530 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dd57fa640 1 -- 172.21.15.92:0/875513051 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de80aca50 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dd57fa640 1 -- 172.21.15.92:0/875513051 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4de4051040 con 0x5561f333b070 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 10 client.16245 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 10 client.16245 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 20 client.16245 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32ceda0 0x5561f376dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 4 client.16245 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.247+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561f36568a0 con 0x5561f32ceda0 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4df18f1640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32ceda0 0x5561f376dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4df18f1640 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32ceda0 0x5561f376dcd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ccac467651de6430 server_cookie=fbd641aeef8f5e0b in_seq=0 out_seq=0 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4dd57fa640 10 client.16245 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4dd57fa640 1 -- 172.21.15.92:0/875513051 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561f36568a0 con 0x5561f32ceda0 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4dd57fa640 10 client.16245 handle_command_reply: tid=0 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4dfa0c9740 1 client.16245 shutdown 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 msgr2=0x5561f36f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.248+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36f2dc0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4de4005e30 tx=0x7f4de40095b0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 shutdown_connections 2024-09-21T02:07:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5561f333b070 0x5561f36f2dc0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561f331a390 0x5561f36f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5561f32ceda0 0x5561f376dcd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 --2- 172.21.15.92:0/875513051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561f33121b0 0x5561f36f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dd57fa640 0 client.16245 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 >> 172.21.15.92:0/875513051 conn(0x5561f332f460 msgr2=0x5561f36e5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 shutdown_connections 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 1 -- 172.21.15.92:0/875513051 wait complete. 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4dfa0c9740 20 client.16245 trim_cache size 0 max 16384 2024-09-21T02:07:23.254 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4dd80b6590 msgr2=0x7f4dd80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4dd80b6590 0x7f4dd80b8aa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4de4006010 tx=0x7f4de402f310 comp rx=0 tx=0).stop 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 msgr2=0x7f4dec19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 0x7f4dec19cd80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c970 tx=0x7f4de800ce30 comp rx=0 tx=0).stop 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 shutdown_connections 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4dd80b6590 0x7f4dd80b8aa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dec108d70 0x7f4dec1a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4dec108420 0x7f4dec19cd80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 --2- 172.21.15.92:0/1944876248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dec102ae0 0x7f4dec19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:23.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 >> 172.21.15.92:0/1944876248 conn(0x7f4dec079770 msgr2=0x7f4dec106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:23.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 shutdown_connections 2024-09-21T02:07:23.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:23.249+0000 7f4df3b55640 1 -- 172.21.15.92:0/1944876248 wait complete. 2024-09-21T02:07:24.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:07:24.341 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:24.727 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:24 smithi092 ceph-mon[31190]: pgmap v433: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:07:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:24 smithi136 ceph-mon[34754]: pgmap v433: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 -- 172.21.15.92:0/1043035384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38109570 msgr2=0x7fce38109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 --2- 172.21.15.92:0/1043035384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38109570 0x7fce38109980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fce20009900 tx=0x7fce2002a410 comp rx=0 tx=0).stop 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 -- 172.21.15.92:0/1043035384 shutdown_connections 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 --2- 172.21.15.92:0/1043035384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109ec0 0x7fce381106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 --2- 172.21.15.92:0/1043035384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38109570 0x7fce38109980 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 --2- 172.21.15.92:0/1043035384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce38103c30 0x7fce38104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 -- 172.21.15.92:0/1043035384 >> 172.21.15.92:0/1043035384 conn(0x7fce3807b8d0 msgr2=0x7fce3807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 -- 172.21.15.92:0/1043035384 shutdown_connections 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 -- 172.21.15.92:0/1043035384 wait complete. 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.818+0000 7fce3ce17640 1 Processor -- start 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 -- start start 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 0x7fce3819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109570 0x7fce3819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce38109ec0 0x7fce381a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce38113ae0 con 0x7fce38103c30 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce38113960 con 0x7fce38109570 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3ce17640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce38113c60 con 0x7fce38109ec0 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 0x7fce3819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 0x7fce3819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52382/0 (socket says 172.21.15.92:52382) 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 -- 172.21.15.92:0/3982526194 learned_addr learned my addr 172.21.15.92:0/3982526194 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:24.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109570 0x7fce3819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce36d9d640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce38109ec0 0x7fce381a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 -- 172.21.15.92:0/3982526194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce38109ec0 msgr2=0x7fce381a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce38109ec0 0x7fce381a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 -- 172.21.15.92:0/3982526194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109570 msgr2=0x7fce3819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109570 0x7fce3819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce3659c640 1 -- 172.21.15.92:0/3982526194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce381a2680 con 0x7fce38103c30 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.819+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109570 0x7fce3819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.820+0000 7fce3659c640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 0x7fce3819c710 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fce2800d770 tx=0x7fce2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.820+0000 7fce2ffff640 1 -- 172.21.15.92:0/3982526194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28017070 con 0x7fce38103c30 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.820+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce381a2970 con 0x7fce38103c30 2024-09-21T02:07:24.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.820+0000 7fce2ffff640 1 -- 172.21.15.92:0/3982526194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280054d0 con 0x7fce38103c30 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.820+0000 7fce2ffff640 1 -- 172.21.15.92:0/3982526194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28004c30 con 0x7fce38103c30 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.820+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce381a90d0 con 0x7fce38103c30 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.822+0000 7fce2ffff640 1 -- 172.21.15.92:0/3982526194 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fce28010460 con 0x7fce38103c30 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 Processor -- start 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 -- start start 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190294070 0x56019063b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 0x56019063b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019063bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601905b1ea0 con 0x56019026b1b0 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5601905a12d0 con 0x560190294070 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce4338b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601905a50e0 con 0x560190273390 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce3659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 0x56019063b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce3659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 0x56019063b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52396/0 (socket says 172.21.15.92:52396) 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce3659c640 1 -- 172.21.15.92:0/4151244484 learned_addr learned my addr 172.21.15.92:0/4151244484 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce2ffff640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce240b6510 0x7fce240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019063bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.823+0000 7fce36d9d640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce240b6510 0x7fce240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce36d9d640 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190294070 0x56019063b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce2ffff640 1 -- 172.21.15.92:0/3982526194 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fce280e6f30 con 0x7fce38103c30 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce3659c640 1 -- 172.21.15.92:0/4151244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 msgr2=0x56019063bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce3659c640 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019063bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce3659c640 1 -- 172.21.15.92:0/4151244484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190294070 msgr2=0x56019063b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce3659c640 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190294070 0x56019063b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce3659c640 1 -- 172.21.15.92:0/4151244484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601902ff650 con 0x56019026b1b0 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce3659c640 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 0x56019063b800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fce280a9670 tx=0x7fce28009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e07e4ea98a7b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:24.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce36d9d640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce240b6510 0x7fce240b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce30005fd0 tx=0x7fce3000abc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce2cd99640 1 -- 172.21.15.92:0/4151244484 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28020030 con 0x56019026b1b0 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce2cd99640 1 -- 172.21.15.92:0/4151244484 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280acbb0 con 0x56019026b1b0 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.824+0000 7fce2cd99640 1 -- 172.21.15.92:0/4151244484 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280abd70 con 0x56019026b1b0 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 -- 172.21.15.92:0/4151244484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 msgr2=0x56019063b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 0x56019063b800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fce280a9670 tx=0x7fce28009f90 comp rx=0 tx=0).stop 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 -- 172.21.15.92:0/4151244484 shutdown_connections 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019063bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56019026b1b0 0x56019063b800 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 --2- 172.21.15.92:0/4151244484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190294070 0x56019063b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 -- 172.21.15.92:0/4151244484 >> 172.21.15.92:0/4151244484 conn(0x560190288460 msgr2=0x5601906391b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 -- 172.21.15.92:0/4151244484 shutdown_connections 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.825+0000 7fce4338b740 1 -- 172.21.15.92:0/4151244484 wait complete. 2024-09-21T02:07:24.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 -- start start 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 Processor -- start 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x560190646940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 0x560190646e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190294070 0x560190652d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560190402700 con 0x560190273390 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56019053c160 con 0x56019026b1b0 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601906499e0 con 0x560190294070 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce4338b740 10 client.0 _ll_register_callbacks cb 0x5601905559e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 0x560190646e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce2dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 0x560190646e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52410/0 (socket says 172.21.15.92:52410) 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce2dd9b640 1 -- 172.21.15.92:0/1618105960 learned_addr learned my addr 172.21.15.92:0/1618105960 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce36d9d640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190294070 0x560190652d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.826+0000 7fce3659c640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x560190646940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce2dd9b640 1 -- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190294070 msgr2=0x560190652d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190294070 0x560190652d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce2dd9b640 1 -- 172.21.15.92:0/1618105960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 msgr2=0x560190646940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x560190646940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce2dd9b640 1 -- 172.21.15.92:0/1618105960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560190569c90 con 0x560190273390 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce3659c640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x560190646940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce36d9d640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190294070 0x560190652d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 0x560190646e80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fce20005ab0 tx=0x7fce200316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc885fe6e8a636e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce1dffb640 1 -- 172.21.15.92:0/1618105960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20040070 con 0x560190273390 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce1dffb640 1 -- 172.21.15.92:0/1618105960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fce20004240 con 0x560190273390 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce280acbb0 con 0x560190273390 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce1dffb640 1 -- 172.21.15.92:0/1618105960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce20035040 con 0x560190273390 2024-09-21T02:07:24.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce4338b740 10 client.16257 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560190655710 con 0x560190273390 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.827+0000 7fce1dffb640 1 -- 172.21.15.92:0/1618105960 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280acbb0 con 0x560190273390 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce1dffb640 1 -- 172.21.15.92:0/1618105960 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce20004a40 con 0x560190273390 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce4338b740 10 client.16257 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce4338b740 10 client.16257 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce4338b740 20 client.16257 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce4338b740 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5601902529f0 0x560190658ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce4338b740 4 client.16257 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601905b3df0 con 0x5601902529f0 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce3659c640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5601902529f0 0x560190658ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce3659c640 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5601902529f0 0x560190658ca0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae3f4790bfec6131 server_cookie=c23c555805200b2a in_seq=0 out_seq=0 2024-09-21T02:07:24.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.828+0000 7fce1dffb640 10 client.16257 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:07:24.838 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:24 smithi099 ceph-mon[34404]: pgmap v433: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:07:24.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce1dffb640 1 -- 172.21.15.92:0/1618105960 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601905b3df0 con 0x5601902529f0 2024-09-21T02:07:24.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce1dffb640 10 client.16257 handle_command_reply: tid=0 2024-09-21T02:07:24.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 client.16257 shutdown 2024-09-21T02:07:24.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 msgr2=0x560190646e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 0x560190646e80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fce20005ab0 tx=0x7fce200316d0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 shutdown_connections 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5601902529f0 0x560190658ca0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce1dffb640 0 client.16257 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190294070 0x560190652d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190273390 0x560190646e80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 --2- 172.21.15.92:0/1618105960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x560190646940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.829+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 >> 172.21.15.92:0/1618105960 conn(0x560190288460 msgr2=0x56019063e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.830+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 shutdown_connections 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.830+0000 7fce4338b740 1 -- 172.21.15.92:0/1618105960 wait complete. 2024-09-21T02:07:24.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.830+0000 7fce4338b740 20 client.16257 trim_cache size 0 max 16384 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.842+0000 7fce4338b740 1 Processor -- start 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.842+0000 7fce4338b740 1 -- start start 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce4338b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56019026b1b0 0x56019063ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce4338b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190273390 0x560190648570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce4338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x560190652c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce4338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce381b1690 con 0x560190294070 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce4338b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce381b31d0 con 0x560190273390 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce4338b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce381b1d60 con 0x56019026b1b0 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x560190652c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x560190652c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52414/0 (socket says 172.21.15.92:52414) 2024-09-21T02:07:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 -- 172.21.15.92:0/1188117515 learned_addr learned my addr 172.21.15.92:0/1188117515 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce3659c640 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56019026b1b0 0x56019063ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190273390 0x560190648570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 -- 172.21.15.92:0/1188117515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56019026b1b0 msgr2=0x56019063ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56019026b1b0 0x56019063ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 -- 172.21.15.92:0/1188117515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190273390 msgr2=0x560190648570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190273390 0x560190648570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 -- 172.21.15.92:0/1188117515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560190633460 con 0x560190294070 2024-09-21T02:07:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190273390 0x560190648570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:24.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.843+0000 7fce36d9d640 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x560190652c00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fce3000fbd0 tx=0x7fce3001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9838bf1f0d3379 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:24.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce1ffff640 1 -- 172.21.15.92:0/1188117515 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3001dbe0 con 0x560190294070 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce1ffff640 1 -- 172.21.15.92:0/1188117515 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3001dd80 con 0x560190294070 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce1ffff640 1 -- 172.21.15.92:0/1188117515 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30005920 con 0x560190294070 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 -- 172.21.15.92:0/1188117515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 msgr2=0x560190652c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x560190652c00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fce3000fbd0 tx=0x7fce3001a000 comp rx=0 tx=0).stop 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 -- 172.21.15.92:0/1188117515 shutdown_connections 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x560190652c00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560190273390 0x560190648570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 --2- 172.21.15.92:0/1188117515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56019026b1b0 0x56019063ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 -- 172.21.15.92:0/1188117515 >> 172.21.15.92:0/1188117515 conn(0x560190288460 msgr2=0x560190647e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 -- 172.21.15.92:0/1188117515 shutdown_connections 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.844+0000 7fce4338b740 1 -- 172.21.15.92:0/1188117515 wait complete. 2024-09-21T02:07:24.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 -- start start 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 Processor -- start 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x56019064ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019064b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x56019064b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce381b2b00 con 0x560190294070 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce381b2430 con 0x56019026b1b0 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce381af670 con 0x560190273390 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce4338b740 10 client.0 _ll_register_callbacks cb 0x7fce20005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce36d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x56019064b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce36d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x56019064b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52416/0 (socket says 172.21.15.92:52416) 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce36d9d640 1 -- 172.21.15.92:0/319250755 learned_addr learned my addr 172.21.15.92:0/319250755 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.845+0000 7fce2dd9b640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019064b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce3659c640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x56019064ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce36d9d640 1 -- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 msgr2=0x56019064b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce36d9d640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019064b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce36d9d640 1 -- 172.21.15.92:0/319250755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 msgr2=0x56019064ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce36d9d640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x56019064ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce36d9d640 1 -- 172.21.15.92:0/319250755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601906c2750 con 0x560190294070 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce36d9d640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x56019064b7c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fce30005540 tx=0x7fce300058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb2beda7758c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce1dffb640 1 -- 172.21.15.92:0/319250755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30034020 con 0x560190294070 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce1dffb640 1 -- 172.21.15.92:0/319250755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fce30004ec0 con 0x560190294070 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3001dd80 con 0x560190294070 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce1dffb640 1 -- 172.21.15.92:0/319250755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce300050c0 con 0x560190294070 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce4338b740 10 client.16263 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601906c3940 con 0x560190294070 2024-09-21T02:07:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.846+0000 7fce1dffb640 1 -- 172.21.15.92:0/319250755 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3001dd80 con 0x560190294070 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce1dffb640 1 -- 172.21.15.92:0/319250755 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce30025070 con 0x560190294070 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce4338b740 10 client.16263 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce4338b740 10 client.16263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce4338b740 20 client.16263 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce4338b740 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560190227da0 0x5601906c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce4338b740 4 client.16263 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601905af8a0 con 0x560190227da0 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce3659c640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560190227da0 0x5601906c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce1dffb640 10 client.16263 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.847+0000 7fce3659c640 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560190227da0 0x5601906c60c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=568c7252e31ae45e server_cookie=26bdd06754839dcf in_seq=0 out_seq=0 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce1dffb640 1 -- 172.21.15.92:0/319250755 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601905af8a0 con 0x560190227da0 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce1dffb640 10 client.16263 handle_command_reply: tid=0 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 client.16263 shutdown 2024-09-21T02:07:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 msgr2=0x56019064b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x56019064b7c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fce30005540 tx=0x7fce300058d0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 shutdown_connections 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560190227da0 0x5601906c60c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560190294070 0x56019064b7c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560190273390 0x56019064b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce1dffb640 0 client.16263 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 --2- 172.21.15.92:0/319250755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56019026b1b0 0x56019064ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 >> 172.21.15.92:0/319250755 conn(0x560190288460 msgr2=0x56019063ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:24.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 shutdown_connections 2024-09-21T02:07:24.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 1 -- 172.21.15.92:0/319250755 wait complete. 2024-09-21T02:07:24.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.848+0000 7fce4338b740 20 client.16263 trim_cache size 0 max 16384 2024-09-21T02:07:24.855 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce240b6510 msgr2=0x7fce240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce240b6510 0x7fce240b8a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce30005fd0 tx=0x7fce3000abc0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 msgr2=0x7fce3819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 0x7fce3819c710 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fce2800d770 tx=0x7fce2800dc30 comp rx=0 tx=0).stop 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 shutdown_connections 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fce240b6510 0x7fce240b8a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce38109ec0 0x7fce381a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38109570 0x7fce3819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 --2- 172.21.15.92:0/3982526194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce38103c30 0x7fce3819c710 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 >> 172.21.15.92:0/3982526194 conn(0x7fce3807b8d0 msgr2=0x7fce381053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 shutdown_connections 2024-09-21T02:07:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:24.849+0000 7fce3ce17640 1 -- 172.21.15.92:0/3982526194 wait complete. 2024-09-21T02:07:25.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:07:25.973 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 -- 172.21.15.92:0/2237842179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708109df0 msgr2=0x7f8708110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 --2- 172.21.15.92:0/2237842179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708109df0 0x7f8708110620 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8700009ca0 tx=0x7f870002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 -- 172.21.15.92:0/2237842179 shutdown_connections 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 --2- 172.21.15.92:0/2237842179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708109df0 0x7f8708110620 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 --2- 172.21.15.92:0/2237842179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87081094a0 0x7f87081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 --2- 172.21.15.92:0/2237842179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8708103b60 0x7f8708103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 -- 172.21.15.92:0/2237842179 >> 172.21.15.92:0/2237842179 conn(0x7f870807b850 msgr2=0x7f870807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 -- 172.21.15.92:0/2237842179 shutdown_connections 2024-09-21T02:07:26.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 -- 172.21.15.92:0/2237842179 wait complete. 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 Processor -- start 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.460+0000 7f870e26e640 1 -- start start 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870e26e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 0x7f870819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870e26e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87081094a0 0x7f870819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870e26e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708109df0 0x7f870819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870e26e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8708113ab0 con 0x7f8708103b60 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870e26e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8708113930 con 0x7f8708109df0 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870e26e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8708113c30 con 0x7f87081094a0 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 0x7f870819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 0x7f870819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52430/0 (socket says 172.21.15.92:52430) 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 -- 172.21.15.92:0/2997227824 learned_addr learned my addr 172.21.15.92:0/2997227824 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f87077fe640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87081094a0 0x7f870819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f870c80b640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708109df0 0x7f870819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 -- 172.21.15.92:0/2997227824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87081094a0 msgr2=0x7f870819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87081094a0 0x7f870819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 -- 172.21.15.92:0/2997227824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708109df0 msgr2=0x7f870819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708109df0 0x7f870819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.461+0000 7f8707fff640 1 -- 172.21.15.92:0/2997227824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87081a02f0 con 0x7f8708103b60 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.462+0000 7f8707fff640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 0x7f870819a440 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f86f800d770 tx=0x7f86f800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.462+0000 7f87057fa640 1 -- 172.21.15.92:0/2997227824 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8017070 con 0x7f8708103b60 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.462+0000 7f87057fa640 1 -- 172.21.15.92:0/2997227824 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f8009820 con 0x7f8708103b60 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.462+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87081a05e0 con 0x7f8708103b60 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.462+0000 7f87057fa640 1 -- 172.21.15.92:0/2997227824 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004bc0 con 0x7f8708103b60 2024-09-21T02:07:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.462+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87081a6e00 con 0x7f8708103b60 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 Processor -- start 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 -- start start 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a0e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a0ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565226984ea0 con 0x56522663e1b0 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5652269742d0 con 0x565226667070 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87147e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652269780e0 con 0x565226646390 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.465+0000 7f87057fa640 1 -- 172.21.15.92:0/2997227824 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f86f80099d0 con 0x7f8708103b60 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f870c80b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a0e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a0ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f870c80b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a0e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:57834/0 (socket says 172.21.15.92:57834) 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f870c80b640 1 -- 172.21.15.92:0/2189905463 learned_addr learned my addr 172.21.15.92:0/2189905463 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f87077fe640 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 -- 172.21.15.92:0/2189905463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 msgr2=0x565226a0f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a0f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 -- 172.21.15.92:0/2189905463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 msgr2=0x565226a0e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a0e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 -- 172.21.15.92:0/2189905463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652266d2650 con 0x56522663e1b0 2024-09-21T02:07:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f8707fff640 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a0ebe0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f86f80e47d0 tx=0x7f86f80ef790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ae37663eb94753 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.466+0000 7f87057fa640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86f00b6640 0x7f86f00b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.467+0000 7f870c80b640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86f00b6640 0x7f86f00b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.467+0000 7f87057fa640 1 -- 172.21.15.92:0/2997227824 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86f80e6db0 con 0x7f8708103b60 2024-09-21T02:07:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.467+0000 7f870c80b640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86f00b6640 0x7f86f00b8b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f870002ade0 tx=0x7f8700005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.469+0000 7f86ee7fc640 1 -- 172.21.15.92:0/2189905463 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80a9510 con 0x56522663e1b0 2024-09-21T02:07:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.469+0000 7f86ee7fc640 1 -- 172.21.15.92:0/2189905463 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f80f4070 con 0x56522663e1b0 2024-09-21T02:07:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.469+0000 7f86ee7fc640 1 -- 172.21.15.92:0/2189905463 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80ac630 con 0x56522663e1b0 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 -- 172.21.15.92:0/2189905463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 msgr2=0x565226a0ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a0ebe0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f86f80e47d0 tx=0x7f86f80ef790 comp rx=0 tx=0).stop 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 -- 172.21.15.92:0/2189905463 shutdown_connections 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a0f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a0ebe0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 --2- 172.21.15.92:0/2189905463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a0e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 -- 172.21.15.92:0/2189905463 >> 172.21.15.92:0/2189905463 conn(0x56522665b460 msgr2=0x565226a0c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 -- 172.21.15.92:0/2189905463 shutdown_connections 2024-09-21T02:07:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 -- 172.21.15.92:0/2189905463 wait complete. 2024-09-21T02:07:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.470+0000 7f87147e2740 1 -- start start 2024-09-21T02:07:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 Processor -- start 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a19e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226646390 0x565226a1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226667070 0x565226a261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5652267d5700 con 0x56522663e1b0 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56522690f160 con 0x565226646390 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565226a1ce80 con 0x565226667070 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a19e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a19e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52456/0 (socket says 172.21.15.92:52456) 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 -- 172.21.15.92:0/887225329 learned_addr learned my addr 172.21.15.92:0/887225329 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87147e2740 10 client.0 _ll_register_callbacks cb 0x5652269289e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f870c80b640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226667070 0x565226a261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f87077fe640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226646390 0x565226a1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 -- 172.21.15.92:0/887225329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226667070 msgr2=0x565226a261a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226667070 0x565226a261a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 -- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226646390 msgr2=0x565226a1a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226646390 0x565226a1a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.471+0000 7f8707fff640 1 -- 172.21.15.92:0/887225329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56522693cc90 con 0x56522663e1b0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f8707fff640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a19e00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f86f80aa130 tx=0x7f86f80e4800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae10a3fcd4bc9ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f86e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f86e77fe640 1 -- 172.21.15.92:0/887225329 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8020070 con 0x56522663e1b0 2024-09-21T02:07:26.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f86e77fe640 1 -- 172.21.15.92:0/887225329 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86f80a9130 con 0x56522663e1b0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f80f4070 con 0x56522663e1b0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f86e77fe640 1 -- 172.21.15.92:0/887225329 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86f80ac810 con 0x56522663e1b0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 10 client.16275 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565226a28c00 con 0x56522663e1b0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f86e77fe640 1 -- 172.21.15.92:0/887225329 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f80f4070 con 0x56522663e1b0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f86e77fe640 1 -- 172.21.15.92:0/887225329 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86f8092ce0 con 0x56522663e1b0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 10 client.16275 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 10 client.16275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 20 client.16275 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652266259f0 0x565226a2bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 4 client.16275 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.472+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565226986df0 con 0x5652266259f0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.473+0000 7f87077fe640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652266259f0 0x565226a2bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.473+0000 7f86e77fe640 10 client.16275 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:07:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.473+0000 7f87077fe640 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652266259f0 0x565226a2bf60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa45221a1871e55c server_cookie=6625082df9e59851 in_seq=0 out_seq=0 2024-09-21T02:07:26.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f86e77fe640 1 -- 172.21.15.92:0/887225329 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565226986df0 con 0x5652266259f0 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f86e77fe640 10 client.16275 handle_command_reply: tid=0 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 client.16275 shutdown 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 msgr2=0x565226a19e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a19e00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f86f80aa130 tx=0x7f86f80e4800 comp rx=0 tx=0).stop 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 shutdown_connections 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226667070 0x565226a261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226646390 0x565226a1a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652266259f0 0x565226a2bf60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 --2- 172.21.15.92:0/887225329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a19e00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f86e77fe640 0 client.16275 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 >> 172.21.15.92:0/887225329 conn(0x56522665b460 msgr2=0x565226a11830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 shutdown_connections 2024-09-21T02:07:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 1 -- 172.21.15.92:0/887225329 wait complete. 2024-09-21T02:07:26.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.474+0000 7f87147e2740 20 client.16275 trim_cache size 0 max 16384 2024-09-21T02:07:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.487+0000 7f87147e2740 1 Processor -- start 2024-09-21T02:07:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.487+0000 7f87147e2740 1 -- start start 2024-09-21T02:07:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f87147e2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a1a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f87147e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f87147e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a11ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f87147e2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87081af470 con 0x56522663e1b0 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f87147e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b0fb0 con 0x565226667070 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f87147e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87081afb40 con 0x565226646390 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a1a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a1a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52462/0 (socket says 172.21.15.92:52462) 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 -- 172.21.15.92:0/2147837666 learned_addr learned my addr 172.21.15.92:0/2147837666 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f870c80b640 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a11ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 -- 172.21.15.92:0/2147837666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 msgr2=0x565226a117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 -- 172.21.15.92:0/2147837666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 msgr2=0x565226a11ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a11ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 -- 172.21.15.92:0/2147837666 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565226a0ce80 con 0x56522663e1b0 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f870c80b640 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:26.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f8707fff640 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a1a2c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f86f80ac720 tx=0x7f86f80ef790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cbf0f22b40c711 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:26.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f86edffb640 1 -- 172.21.15.92:0/2147837666 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8020020 con 0x56522663e1b0 2024-09-21T02:07:26.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f86edffb640 1 -- 172.21.15.92:0/2147837666 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f80f4070 con 0x56522663e1b0 2024-09-21T02:07:26.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.488+0000 7f86edffb640 1 -- 172.21.15.92:0/2147837666 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f80d4040 con 0x56522663e1b0 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 -- 172.21.15.92:0/2147837666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 msgr2=0x565226a1a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a1a2c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f86f80ac720 tx=0x7f86f80ef790 comp rx=0 tx=0).stop 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 -- 172.21.15.92:0/2147837666 shutdown_connections 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 --2- 172.21.15.92:0/2147837666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a1a2c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 -- 172.21.15.92:0/2147837666 >> 172.21.15.92:0/2147837666 conn(0x56522665b460 msgr2=0x565226a19bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 -- 172.21.15.92:0/2147837666 shutdown_connections 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.489+0000 7f87147e2740 1 -- 172.21.15.92:0/2147837666 wait complete. 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 -- start start 2024-09-21T02:07:26.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 Processor -- start 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a2d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a2d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b08e0 con 0x56522663e1b0 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b0210 con 0x565226667070 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87081ad450 con 0x565226646390 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f87147e2740 10 client.0 _ll_register_callbacks cb 0x7f8700009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a2d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f8707fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a2d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52476/0 (socket says 172.21.15.92:52476) 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f8707fff640 1 -- 172.21.15.92:0/2280807278 learned_addr learned my addr 172.21.15.92:0/2280807278 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.490+0000 7f870c80b640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f87077fe640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a2d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f8707fff640 1 -- 172.21.15.92:0/2280807278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 msgr2=0x565226a2d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f8707fff640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a2d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f8707fff640 1 -- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 msgr2=0x565226a956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f8707fff640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f8707fff640 1 -- 172.21.15.92:0/2280807278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565226a2e0d0 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f8707fff640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a2d2c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f86f80aa130 tx=0x7f86f80ac9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edc370edbd55b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f86e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f86e77fe640 1 -- 172.21.15.92:0/2280807278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f805c070 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f86e77fe640 1 -- 172.21.15.92:0/2280807278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86f8009cf0 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f80f4070 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f86e77fe640 1 -- 172.21.15.92:0/2280807278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86f80c7020 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f87147e2740 10 client.16281 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565226a2e0d0 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f86e77fe640 1 -- 172.21.15.92:0/2280807278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f80f4070 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.491+0000 7f86e77fe640 1 -- 172.21.15.92:0/2280807278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86f8092ce0 con 0x56522663e1b0 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 10 client.16281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:07:26.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 20 client.16281 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652265fada0 0x565226a98fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 4 client.16281 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652269828a0 con 0x5652265fada0 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87077fe640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652265fada0 0x565226a98fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87077fe640 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652265fada0 0x565226a98fe0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eba041c29dfd4d8d server_cookie=ca8a7bb7695a09ca in_seq=0 out_seq=0 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f86e77fe640 10 client.16281 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f86e77fe640 1 -- 172.21.15.92:0/2280807278 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652269828a0 con 0x5652265fada0 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f86e77fe640 10 client.16281 handle_command_reply: tid=0 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.492+0000 7f87147e2740 1 client.16281 shutdown 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 msgr2=0x565226a2d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a2d2c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f86f80aa130 tx=0x7f86f80ac9d0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 shutdown_connections 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565226667070 0x565226a956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565226646390 0x565226a2d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5652265fada0 0x565226a98fe0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 --2- 172.21.15.92:0/2280807278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56522663e1b0 0x565226a2d2c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 >> 172.21.15.92:0/2280807278 conn(0x56522665b460 msgr2=0x565226a0f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f86e77fe640 0 client.16281 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 shutdown_connections 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 1 -- 172.21.15.92:0/2280807278 wait complete. 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f87147e2740 20 client.16281 trim_cache size 0 max 16384 2024-09-21T02:07:26.498 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86f00b6640 msgr2=0x7f86f00b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f870e26e640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86f00b6640 0x7f86f00b8b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f870002ade0 tx=0x7f8700005c50 comp rx=0 tx=0).stop 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 msgr2=0x7f870819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.493+0000 7f870e26e640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 0x7f870819a440 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f86f800d770 tx=0x7f86f800db70 comp rx=0 tx=0).stop 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 shutdown_connections 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86f00b6640 0x7f86f00b8b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708109df0 0x7f870819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87081094a0 0x7f870819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 --2- 172.21.15.92:0/2997227824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8708103b60 0x7f870819a440 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 >> 172.21.15.92:0/2997227824 conn(0x7f870807b850 msgr2=0x7f8708105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:26.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 shutdown_connections 2024-09-21T02:07:26.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:26.494+0000 7f870e26e640 1 -- 172.21.15.92:0/2997227824 wait complete. 2024-09-21T02:07:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:26 smithi136 ceph-mon[34754]: pgmap v434: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:26.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:26 smithi092 ceph-mon[31190]: pgmap v434: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:26.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:26 smithi099 ceph-mon[34404]: pgmap v434: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:27.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:07:27.427 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 -- 172.21.15.92:0/3192745976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe81083c0 msgr2=0x7f1fe81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 --2- 172.21.15.92:0/3192745976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe81083c0 0x7f1fe81087d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc02a410 comp rx=0 tx=0).stop 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 -- 172.21.15.92:0/3192745976 shutdown_connections 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 --2- 172.21.15.92:0/3192745976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe8108d10 0x7f1fe810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 --2- 172.21.15.92:0/3192745976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe81083c0 0x7f1fe81087d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 --2- 172.21.15.92:0/3192745976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe8102a80 0x7f1fe8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 -- 172.21.15.92:0/3192745976 >> 172.21.15.92:0/3192745976 conn(0x7f1fe8079740 msgr2=0x7f1fe8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 -- 172.21.15.92:0/3192745976 shutdown_connections 2024-09-21T02:07:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 -- 172.21.15.92:0/3192745976 wait complete. 2024-09-21T02:07:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 Processor -- start 2024-09-21T02:07:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.934+0000 7f1fee93a640 1 -- start start 2024-09-21T02:07:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fee93a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 0x7f1fe810ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fee93a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe81083c0 0x7f1fe8110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fee93a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe8108d10 0x7f1fe8110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fee93a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8113ae0 con 0x7f1fe8102a80 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fee93a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8113960 con 0x7f1fe81083c0 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fee93a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8113c60 con 0x7f1fe8108d10 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 0x7f1fe810ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 0x7f1fe810ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52488/0 (socket says 172.21.15.92:52488) 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 -- 172.21.15.92:0/652623112 learned_addr learned my addr 172.21.15.92:0/652623112 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe81083c0 0x7f1fe8110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1feced7640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe8108d10 0x7f1fe8110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 -- 172.21.15.92:0/652623112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe8108d10 msgr2=0x7f1fe8110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe8108d10 0x7f1fe8110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 -- 172.21.15.92:0/652623112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe81083c0 msgr2=0x7f1fe8110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe81083c0 0x7f1fe8110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.935+0000 7f1fe7fff640 1 -- 172.21.15.92:0/652623112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe81a87b0 con 0x7f1fe8102a80 2024-09-21T02:07:27.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.936+0000 7f1fe7fff640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 0x7f1fe810ffd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d770 tx=0x7f1fd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:27.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.936+0000 7f1fe57fa640 1 -- 172.21.15.92:0/652623112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8017070 con 0x7f1fe8102a80 2024-09-21T02:07:27.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.936+0000 7f1fe57fa640 1 -- 172.21.15.92:0/652623112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd80054d0 con 0x7f1fe8102a80 2024-09-21T02:07:27.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.936+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81a8aa0 con 0x7f1fe8102a80 2024-09-21T02:07:27.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.936+0000 7f1fe57fa640 1 -- 172.21.15.92:0/652623112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8004c30 con 0x7f1fe8102a80 2024-09-21T02:07:27.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.936+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a9030 con 0x7f1fe8102a80 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 Processor -- start 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1fe57fa640 1 -- 172.21.15.92:0/652623112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1fd8004e10 con 0x7f1fe8102a80 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 -- start start 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f905070 0x55f19fcac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcacbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 0x55f19fcad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f19fc22ea0 con 0x55f19f8e4390 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f19fc122d0 con 0x55f19f905070 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f19fc160e0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 0x55f19fcad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 0x55f19fcad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52500/0 (socket says 172.21.15.92:52500) 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1fe77fe640 1 -- 172.21.15.92:0/315642438 learned_addr learned my addr 172.21.15.92:0/315642438 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1fe7fff640 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcacbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.939+0000 7f1feced7640 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f905070 0x55f19fcac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe77fe640 1 -- 172.21.15.92:0/315642438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 msgr2=0x55f19fcacbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcacbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe77fe640 1 -- 172.21.15.92:0/315642438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f905070 msgr2=0x55f19fcac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f905070 0x55f19fcac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe57fa640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1fd00b6590 0x7f1fd00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe77fe640 1 -- 172.21.15.92:0/315642438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19f970650 con 0x55f19f8e4390 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1feced7640 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f905070 0x55f19fcac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 0x55f19fcad120 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a03548b393c3c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:27.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe57fa640 1 -- 172.21.15.92:0/652623112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1fd80e6df0 con 0x7f1fe8102a80 2024-09-21T02:07:27.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fe7fff640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1fd00b6590 0x7f1fd00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fce7fc640 1 -- 172.21.15.92:0/315642438 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc040070 con 0x55f19f8e4390 2024-09-21T02:07:27.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fce7fc640 1 -- 172.21.15.92:0/315642438 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc005db0 con 0x55f19f8e4390 2024-09-21T02:07:27.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.940+0000 7f1fce7fc640 1 -- 172.21.15.92:0/315642438 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc035040 con 0x55f19f8e4390 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 -- 172.21.15.92:0/315642438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 msgr2=0x55f19fcad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1fe7fff640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1fd00b6590 0x7f1fd00b8aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80a94d0 tx=0x7f1fd80ea830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 0x55f19fcad120 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc032df0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 -- 172.21.15.92:0/315642438 shutdown_connections 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8e4390 0x55f19fcad120 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcacbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/315642438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f905070 0x55f19fcac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 -- 172.21.15.92:0/315642438 >> 172.21.15.92:0/315642438 conn(0x55f19f8f9460 msgr2=0x55f19fcaa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:27.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 -- 172.21.15.92:0/315642438 shutdown_connections 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.941+0000 7f1ff4eae740 1 -- 172.21.15.92:0/315642438 wait complete. 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 -- start start 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 Processor -- start 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcb7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 0x55f19fcc4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f19fa73700 con 0x55f19f905070 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f19fbad160 con 0x55f19f8e4390 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f19fcbae20 con 0x55f19f8dc1b0 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1ff4eae740 10 client.0 _ll_register_callbacks cb 0x55f19fbc69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1feced7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 0x55f19fcc4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1feced7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 0x55f19fcc4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52502/0 (socket says 172.21.15.92:52502) 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.942+0000 7f1feced7640 1 -- 172.21.15.92:0/3237036489 learned_addr learned my addr 172.21.15.92:0/3237036489 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcb7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 -- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 msgr2=0x55f19fcb7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcb7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 -- 172.21.15.92:0/3237036489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 msgr2=0x55f19fcb82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcb82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 -- 172.21.15.92:0/3237036489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19fbdac90 con 0x55f19f905070 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1feced7640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 0x55f19fcc4140 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009fd0 tx=0x7f1fe0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443e22a005af2ed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3237036489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00116f0 con 0x55f19f905070 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3237036489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1fe0011890 con 0x55f19f905070 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fdc005db0 con 0x55f19f905070 2024-09-21T02:07:27.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3237036489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fe0011a90 con 0x55f19f905070 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1ff4eae740 10 client.16293 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.943+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f19fcc6bc0 con 0x55f19f905070 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3237036489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc005db0 con 0x55f19f905070 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3237036489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1fe001cb40 con 0x55f19f905070 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1ff4eae740 10 client.16293 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1ff4eae740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1ff4eae740 20 client.16293 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f8c39f0 0x55f19fcca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1ff4eae740 4 client.16293 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f19fc24df0 con 0x55f19f8c39f0 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.944+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f8c39f0 0x55f19fcca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.945+0000 7f1fc77fe640 10 client.16293 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:27.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.945+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f8c39f0 0x55f19fcca2f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69aecf9a80c5c6cf server_cookie=7e4b105ceb145d6 in_seq=0 out_seq=0 2024-09-21T02:07:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.945+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3237036489 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f19fc24df0 con 0x55f19f8c39f0 2024-09-21T02:07:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.945+0000 7f1fc77fe640 10 client.16293 handle_command_reply: tid=0 2024-09-21T02:07:27.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.945+0000 7f1ff4eae740 1 client.16293 shutdown 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 msgr2=0x55f19fcc4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 0x55f19fcc4140 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009fd0 tx=0x7f1fe0009b20 comp rx=0 tx=0).stop 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 shutdown_connections 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f8c39f0 0x55f19fcca2f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f905070 0x55f19fcc4140 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1fc77fe640 0 client.16293 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcb82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3237036489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fcb7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 >> 172.21.15.92:0/3237036489 conn(0x55f19f8f9460 msgr2=0x55f19fcaf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 shutdown_connections 2024-09-21T02:07:27.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3237036489 wait complete. 2024-09-21T02:07:27.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.946+0000 7f1ff4eae740 20 client.16293 trim_cache size 0 max 16384 2024-09-21T02:07:27.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 Processor -- start 2024-09-21T02:07:27.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 -- start start 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 0x55f19fcb99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fca9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b1630 con 0x55f19f8c39f0 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b3170 con 0x55f19f8e4390 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b1d00 con 0x55f19f8dc1b0 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1feced7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 0x55f19fcb99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1feced7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 0x55f19fcb99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52516/0 (socket says 172.21.15.92:52516) 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.959+0000 7f1feced7640 1 -- 172.21.15.92:0/1107776769 learned_addr learned my addr 172.21.15.92:0/1107776769 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fca9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1feced7640 1 -- 172.21.15.92:0/1107776769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 msgr2=0x55f19fca9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1feced7640 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fca9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1feced7640 1 -- 172.21.15.92:0/1107776769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 msgr2=0x55f19fcaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1feced7640 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1feced7640 1 -- 172.21.15.92:0/1107776769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19fcae990 con 0x55f19f8c39f0 2024-09-21T02:07:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1feced7640 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 0x55f19fcb99a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1fe001cee0 tx=0x7f1fe0007fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c641acccece30e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1fcdffb640 1 -- 172.21.15.92:0/1107776769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00097c0 con 0x55f19f8c39f0 2024-09-21T02:07:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1fcdffb640 1 -- 172.21.15.92:0/1107776769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe0026e40 con 0x55f19f8c39f0 2024-09-21T02:07:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.960+0000 7f1fcdffb640 1 -- 172.21.15.92:0/1107776769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0011d00 con 0x55f19f8c39f0 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 -- 172.21.15.92:0/1107776769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 msgr2=0x55f19fcb99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 0x55f19fcb99a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1fe001cee0 tx=0x7f1fe0007fa0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 -- 172.21.15.92:0/1107776769 shutdown_connections 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcaa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8dc1b0 0x55f19fca9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/1107776769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8c39f0 0x55f19fcb99a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 -- 172.21.15.92:0/1107776769 >> 172.21.15.92:0/1107776769 conn(0x55f19f8f9460 msgr2=0x55f19fcb9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 -- 172.21.15.92:0/1107776769 shutdown_connections 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 -- 172.21.15.92:0/1107776769 wait complete. 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 -- start start 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.961+0000 7f1ff4eae740 1 Processor -- start 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8c39f0 0x55f19fcbc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 0x55f19fcbc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcbcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b2aa0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81b23d0 con 0x55f19f8e4390 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81af610 con 0x55f19f8c39f0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1ff4eae740 10 client.0 _ll_register_callbacks cb 0x7f1fe0009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 0x55f19fcbc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 0x55f19fcbc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52524/0 (socket says 172.21.15.92:52524) 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 -- 172.21.15.92:0/3072549634 learned_addr learned my addr 172.21.15.92:0/3072549634 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1feced7640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8c39f0 0x55f19fcbc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1feced7640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcbcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 -- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8c39f0 msgr2=0x55f19fcbc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8c39f0 0x55f19fcbc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 -- 172.21.15.92:0/3072549634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 msgr2=0x55f19fcbcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcbcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 -- 172.21.15.92:0/3072549634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19fd33e30 con 0x55f19f8dc1b0 2024-09-21T02:07:27.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1feced7640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8c39f0 0x55f19fcbc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fe77fe640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 0x55f19fcbc740 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0052d0 tx=0x7f1fdc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fcae68ea4ea55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.962+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3072549634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc055060 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3072549634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1fdc0314a0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f19fc208a0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3072549634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fdc035040 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 10 client.16299 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f19fd350b0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3072549634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f19fc208a0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3072549634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1fdc0035f0 con 0x55f19f8dc1b0 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 10 client.16299 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 20 client.16299 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f905070 0x55f19fd37450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 4 client.16299 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.963+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f19fc1dcd0 con 0x55f19f905070 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1feced7640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f905070 0x55f19fd37450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1fc77fe640 10 client.16299 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1feced7640 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f905070 0x55f19fd37450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f968485278a654 server_cookie=a90aee72f53080d4 in_seq=0 out_seq=0 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1fc77fe640 1 -- 172.21.15.92:0/3072549634 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f19fc1dcd0 con 0x55f19f905070 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1fc77fe640 10 client.16299 handle_command_reply: tid=0 2024-09-21T02:07:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 client.16299 shutdown 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 msgr2=0x55f19fcbc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 0x55f19fcbc740 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0052d0 tx=0x7f1fdc02a410 comp rx=0 tx=0).stop 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 shutdown_connections 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f19f905070 0x55f19fd37450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f19f8e4390 0x55f19fcbcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f19f8dc1b0 0x55f19fcbc740 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 --2- 172.21.15.92:0/3072549634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f8c39f0 0x55f19fcbc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 >> 172.21.15.92:0/3072549634 conn(0x55f19f8f9460 msgr2=0x55f19fcb9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.964+0000 7f1fc77fe640 0 client.16299 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 shutdown_connections 2024-09-21T02:07:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1ff4eae740 1 -- 172.21.15.92:0/3072549634 wait complete. 2024-09-21T02:07:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1ff4eae740 20 client.16299 trim_cache size 0 max 16384 2024-09-21T02:07:27.970 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:07:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1fd00b6590 msgr2=0x7f1fd00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1fd00b6590 0x7f1fd00b8aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80a94d0 tx=0x7f1fd80ea830 comp rx=0 tx=0).stop 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 msgr2=0x7f1fe810ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 0x7f1fe810ffd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d770 tx=0x7f1fd800dc30 comp rx=0 tx=0).stop 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 shutdown_connections 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1fd00b6590 0x7f1fd00b8aa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe8108d10 0x7f1fe8110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe81083c0 0x7f1fe8110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 --2- 172.21.15.92:0/652623112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fe8102a80 0x7f1fe810ffd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 >> 172.21.15.92:0/652623112 conn(0x7f1fe8079740 msgr2=0x7f1fe81052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 shutdown_connections 2024-09-21T02:07:27.971 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:27.965+0000 7f1fee93a640 1 -- 172.21.15.92:0/652623112 wait complete. 2024-09-21T02:07:28.719 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:28 smithi092 ceph-mon[31190]: pgmap v435: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:28.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:28 smithi136 ceph-mon[34754]: pgmap v435: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:28 smithi099 ceph-mon[34404]: pgmap v435: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:29.746 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:07:29.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:07:30.004 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 -- 172.21.15.92:0/366544010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a0108d10 msgr2=0x7fb2a010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/366544010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a0108d10 0x7fb2a010f540 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb298009ca0 tx=0x7fb29802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 -- 172.21.15.92:0/366544010 shutdown_connections 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/366544010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a0108d10 0x7fb2a010f540 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/366544010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a01083c0 0x7fb2a01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/366544010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2a0102a80 0x7fb2a0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 -- 172.21.15.92:0/366544010 >> 172.21.15.92:0/366544010 conn(0x7fb2a0079740 msgr2=0x7fb2a0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 -- 172.21.15.92:0/366544010 shutdown_connections 2024-09-21T02:07:30.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 -- 172.21.15.92:0/366544010 wait complete. 2024-09-21T02:07:30.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.446+0000 7fb2a62fe640 1 Processor -- start 2024-09-21T02:07:30.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 -- start start 2024-09-21T02:07:30.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2a0102a80 0x7fb2a019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 0x7fb2a019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a0108d10 0x7fb2a019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01129e0 con 0x7fb2a01083c0 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a0112860 con 0x7fb2a0102a80 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a62fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a0112b60 con 0x7fb2a0108d10 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 0x7fb2a019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 0x7fb2a019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52542/0 (socket says 172.21.15.92:52542) 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb29f7fe640 1 -- 172.21.15.92:0/2188509519 learned_addr learned my addr 172.21.15.92:0/2188509519 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb29ffff640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2a0102a80 0x7fb2a019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.447+0000 7fb2a489b640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a0108d10 0x7fb2a019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29f7fe640 1 -- 172.21.15.92:0/2188509519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a0108d10 msgr2=0x7fb2a019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a0108d10 0x7fb2a019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29f7fe640 1 -- 172.21.15.92:0/2188509519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2a0102a80 msgr2=0x7fb2a019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2a0102a80 0x7fb2a019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29f7fe640 1 -- 172.21.15.92:0/2188509519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a01a64b0 con 0x7fb2a01083c0 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 0x7fb2a019b7f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c970 tx=0x7fb29400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29d7fa640 1 -- 172.21.15.92:0/2188509519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294014070 con 0x7fb2a01083c0 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a01a67a0 con 0x7fb2a01083c0 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29d7fa640 1 -- 172.21.15.92:0/2188509519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb294004440 con 0x7fb2a01083c0 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb29d7fa640 1 -- 172.21.15.92:0/2188509519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294003d30 con 0x7fb2a01083c0 2024-09-21T02:07:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.448+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a01a6db0 con 0x7fb2a01083c0 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.450+0000 7fb2ac872740 1 Processor -- start 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 -- start start 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb29d7fa640 1 -- 172.21.15.92:0/2188509519 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb294017070 con 0x7fb2a01083c0 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a215233430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215233970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215233eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2151a9e60 con 0x55a214e6b670 2024-09-21T02:07:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a215199290 con 0x55a214e63490 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2ac872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a21519d0a0 con 0x55a214e8c350 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215233eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2a489b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a215233430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2a489b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a215233430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:47722/0 (socket says 172.21.15.92:47722) 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb2a489b640 1 -- 172.21.15.92:0/1036119318 learned_addr learned my addr 172.21.15.92:0/1036119318 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.451+0000 7fb29ffff640 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215233970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2a489b640 1 -- 172.21.15.92:0/1036119318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 msgr2=0x55a215233970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2a489b640 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215233970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2a489b640 1 -- 172.21.15.92:0/1036119318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 msgr2=0x55a215233eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2a489b640 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215233eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2a489b640 1 -- 172.21.15.92:0/1036119318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a214ef7930 con 0x55a214e8c350 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215233eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:07:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb29d7fa640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2880b6540 0x7fb2880b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb29ffff640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2880b6540 0x7fb2880b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2a489b640 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a215233430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb298009930 tx=0x7fb298009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b6bd844bcd84123 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb29d7fa640 1 -- 172.21.15.92:0/2188509519 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb2940e6e30 con 0x7fb2a01083c0 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2867fc640 1 -- 172.21.15.92:0/1036119318 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298031df0 con 0x55a214e8c350 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb29ffff640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2880b6540 0x7fb2880b8a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a2152320b0 tx=0x7fb290008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2867fc640 1 -- 172.21.15.92:0/1036119318 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298036040 con 0x55a214e8c350 2024-09-21T02:07:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.452+0000 7fb2867fc640 1 -- 172.21.15.92:0/1036119318 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2980042a0 con 0x55a214e8c350 2024-09-21T02:07:30.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 -- 172.21.15.92:0/1036119318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 msgr2=0x55a215233430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a215233430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb298009930 tx=0x7fb298009960 comp rx=0 tx=0).stop 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 -- 172.21.15.92:0/1036119318 shutdown_connections 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215233eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215233970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1036119318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a215233430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 -- 172.21.15.92:0/1036119318 >> 172.21.15.92:0/1036119318 conn(0x55a214e80740 msgr2=0x55a215231320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 -- 172.21.15.92:0/1036119318 shutdown_connections 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.453+0000 7fb2ac872740 1 -- 172.21.15.92:0/1036119318 wait complete. 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 -- start start 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 Processor -- start 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215243480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a2152439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a21524f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a214ffa890 con 0x55a214e6b670 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a215133d40 con 0x55a214e63490 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.454+0000 7fb2ac872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a215245f10 con 0x55a214e8c350 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb2ac872740 10 client.0 _ll_register_callbacks cb 0x55a215197c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a2152439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a2152439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52572/0 (socket says 172.21.15.92:52572) 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 -- 172.21.15.92:0/1285650026 learned_addr learned my addr 172.21.15.92:0/1285650026 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb2a489b640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a21524f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29ffff640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215243480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 -- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 msgr2=0x55a21524f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a21524f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 -- 172.21.15.92:0/1285650026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 msgr2=0x55a215243480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215243480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 -- 172.21.15.92:0/1285650026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a21524f7d0 con 0x55a214e6b670 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29ffff640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215243480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a2152439c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb2940acf30 tx=0x7fb29400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0d3018d17085e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb27f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb27f7fe640 1 -- 172.21.15.92:0/1285650026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2940a8e40 con 0x55a214e6b670 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb27f7fe640 1 -- 172.21.15.92:0/1285650026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb2940a8b00 con 0x55a214e6b670 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb298036040 con 0x55a214e6b670 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.455+0000 7fb27f7fe640 1 -- 172.21.15.92:0/1285650026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2940ac710 con 0x55a214e6b670 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 10 client.16311 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:30.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a215252160 con 0x55a214e6b670 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb27f7fe640 1 -- 172.21.15.92:0/1285650026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298036040 con 0x55a214e6b670 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb27f7fe640 1 -- 172.21.15.92:0/1285650026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2940981a0 con 0x55a214e6b670 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 10 client.16311 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 20 client.16311 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e4acd0 0x55a215255580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 4 client.16311 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2151abdb0 con 0x55a214e4acd0 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.456+0000 7fb29ffff640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e4acd0 0x55a215255580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.457+0000 7fb27f7fe640 10 client.16311 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.457+0000 7fb29ffff640 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e4acd0 0x55a215255580 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e605173719e949 server_cookie=82daa0a23367a698 in_seq=0 out_seq=0 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb27f7fe640 1 -- 172.21.15.92:0/1285650026 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2151abdb0 con 0x55a214e4acd0 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb27f7fe640 10 client.16311 handle_command_reply: tid=0 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 client.16311 shutdown 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 msgr2=0x55a2152439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a2152439c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb2940acf30 tx=0x7fb29400a620 comp rx=0 tx=0).stop 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 shutdown_connections 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e4acd0 0x55a215255580 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e8c350 0x55a21524f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a2152439c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 --2- 172.21.15.92:0/1285650026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e63490 0x55a215243480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 >> 172.21.15.92:0/1285650026 conn(0x55a214e80740 msgr2=0x55a21522f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb27f7fe640 0 client.16311 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 shutdown_connections 2024-09-21T02:07:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 1 -- 172.21.15.92:0/1285650026 wait complete. 2024-09-21T02:07:30.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.458+0000 7fb2ac872740 20 client.16311 trim_cache size 0 max 16384 2024-09-21T02:07:30.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 Processor -- start 2024-09-21T02:07:30.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 -- start start 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 0x55a21523ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e6b670 0x55a21522ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a215240470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01ad2e0 con 0x55a214e63490 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01aee20 con 0x55a214e8c350 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2ac872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01ad9b0 con 0x55a214e6b670 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 0x55a21523ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 0x55a21523ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52588/0 (socket says 172.21.15.92:52588) 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 -- 172.21.15.92:0/794950188 learned_addr learned my addr 172.21.15.92:0/794950188 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e6b670 0x55a21522ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb2a489b640 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a215240470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 -- 172.21.15.92:0/794950188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e6b670 msgr2=0x55a21522ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e6b670 0x55a21522ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 -- 172.21.15.92:0/794950188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 msgr2=0x55a215240470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a215240470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.476+0000 7fb29ffff640 1 -- 172.21.15.92:0/794950188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a215236260 con 0x55a214e63490 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.477+0000 7fb29ffff640 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 0x55a21523ea30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb2a006c6a0 tx=0x7fb290017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36abf1604a38e2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.477+0000 7fb285ffb640 1 -- 172.21.15.92:0/794950188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290015e60 con 0x55a214e63490 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.477+0000 7fb285ffb640 1 -- 172.21.15.92:0/794950188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb29002e070 con 0x55a214e63490 2024-09-21T02:07:30.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.477+0000 7fb285ffb640 1 -- 172.21.15.92:0/794950188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290014640 con 0x55a214e63490 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 -- 172.21.15.92:0/794950188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 msgr2=0x55a21523ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 0x55a21523ea30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb2a006c6a0 tx=0x7fb290017000 comp rx=0 tx=0).stop 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 -- 172.21.15.92:0/794950188 shutdown_connections 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a215240470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e6b670 0x55a21522ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 --2- 172.21.15.92:0/794950188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e63490 0x55a21523ea30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 -- 172.21.15.92:0/794950188 >> 172.21.15.92:0/794950188 conn(0x55a214e80740 msgr2=0x55a21522f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 -- 172.21.15.92:0/794950188 shutdown_connections 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 -- 172.21.15.92:0/794950188 wait complete. 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 -- start start 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.478+0000 7fb2ac872740 1 Processor -- start 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e63490 0x55a21524b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215246160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a2152466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01ae750 con 0x55a214e6b670 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01ae080 con 0x55a214e8c350 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a01ab2c0 con 0x55a214e63490 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 10 client.0 _ll_register_callbacks cb 0x7fb294003ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215246160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215246160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52596/0 (socket says 172.21.15.92:52596) 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 -- 172.21.15.92:0/808933501 learned_addr learned my addr 172.21.15.92:0/808933501 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29ffff640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e63490 0x55a21524b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2a489b640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a2152466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 -- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e63490 msgr2=0x55a21524b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e63490 0x55a21524b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 -- 172.21.15.92:0/808933501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 msgr2=0x55a2152466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a2152466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 -- 172.21.15.92:0/808933501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a215234220 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb29f7fe640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215246160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb294003810 tx=0x7fb2940a8d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcd0c10ca74ffcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb27f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb27f7fe640 1 -- 172.21.15.92:0/808933501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2940ac8f0 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb27f7fe640 1 -- 172.21.15.92:0/808933501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb2940ea020 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.479+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb29002e070 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb27f7fe640 1 -- 172.21.15.92:0/808933501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2940acd70 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 10 client.16317 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2152c0c20 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb27f7fe640 1 -- 172.21.15.92:0/808933501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb29002e070 con 0x55a214e6b670 2024-09-21T02:07:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb27f7fe640 1 -- 172.21.15.92:0/808933501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb29405c0f0 con 0x55a214e6b670 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 10 client.16317 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 20 client.16317 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e20080 0x55a2152c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 4 client.16317 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a2151a7860 con 0x55a214e20080 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.480+0000 7fb29ffff640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e20080 0x55a2152c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.481+0000 7fb29ffff640 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e20080 0x55a2152c3220 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802325768656c6bc server_cookie=d86f9c04ad1d16f2 in_seq=0 out_seq=0 2024-09-21T02:07:30.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.481+0000 7fb27f7fe640 10 client.16317 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.482+0000 7fb27f7fe640 1 -- 172.21.15.92:0/808933501 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a2151a7860 con 0x55a214e20080 2024-09-21T02:07:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.482+0000 7fb27f7fe640 10 client.16317 handle_command_reply: tid=0 2024-09-21T02:07:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.482+0000 7fb2ac872740 1 client.16317 shutdown 2024-09-21T02:07:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 msgr2=0x55a215246160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215246160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb294003810 tx=0x7fb2940a8d30 comp rx=0 tx=0).stop 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 shutdown_connections 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a214e20080 0x55a2152c3220 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a214e8c350 0x55a2152466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a214e6b670 0x55a215246160 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb27f7fe640 0 client.16317 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 --2- 172.21.15.92:0/808933501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a214e63490 0x55a21524b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 >> 172.21.15.92:0/808933501 conn(0x55a214e80740 msgr2=0x55a2152343f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 shutdown_connections 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 1 -- 172.21.15.92:0/808933501 wait complete. 2024-09-21T02:07:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2ac872740 20 client.16317 trim_cache size 0 max 16384 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4", 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2880b6540 msgr2=0x7fb2880b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2880b6540 0x7fb2880b8a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a2152320b0 tx=0x7fb290008040 comp rx=0 tx=0).stop 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 msgr2=0x7fb2a019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.483+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 0x7fb2a019b7f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c970 tx=0x7fb29400cd70 comp rx=0 tx=0).stop 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 shutdown_connections 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb2880b6540 0x7fb2880b8a50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2a0108d10 0x7fb2a019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a01083c0 0x7fb2a019b7f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 --2- 172.21.15.92:0/2188509519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2a0102a80 0x7fb2a019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 >> 172.21.15.92:0/2188509519 conn(0x7fb2a0079740 msgr2=0x7fb2a0105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 shutdown_connections 2024-09-21T02:07:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:30.484+0000 7fb2a62fe640 1 -- 172.21.15.92:0/2188509519 wait complete. 2024-09-21T02:07:30.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:30 smithi136 ceph-mon[34754]: pgmap v436: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:07:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:30 smithi092 ceph-mon[31190]: pgmap v436: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:07:30.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:30 smithi099 ceph-mon[34404]: pgmap v436: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:07:31.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:07:31.584 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:07:31.734 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:31 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:07:31.735 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:31 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:07:31.735 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:31 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:07:31.735 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:31 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:07:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:31 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:07:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:31 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:07:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:31 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:07:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:31 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:07:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:31 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:07:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:31 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:07:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:31 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:07:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:31 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:07:32.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 -- 172.21.15.92:0/3185516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 msgr2=0x7fa43c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 --2- 172.21.15.92:0/3185516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c102ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009900 tx=0x7fa42c02a410 comp rx=0 tx=0).stop 2024-09-21T02:07:32.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 -- 172.21.15.92:0/3185516382 shutdown_connections 2024-09-21T02:07:32.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 --2- 172.21.15.92:0/3185516382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c108d20 0x7fa43c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 --2- 172.21.15.92:0/3185516382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c1083d0 0x7fa43c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 --2- 172.21.15.92:0/3185516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c102ef0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 -- 172.21.15.92:0/3185516382 >> 172.21.15.92:0/3185516382 conn(0x7fa43c079760 msgr2=0x7fa43c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 -- 172.21.15.92:0/3185516382 shutdown_connections 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.078+0000 7fa441cdc640 1 -- 172.21.15.92:0/3185516382 wait complete. 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 Processor -- start 2024-09-21T02:07:32.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 -- start start 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c1083d0 0x7fa43c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108d20 0x7fa43c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c113b10 con 0x7fa43c102ae0 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c113990 con 0x7fa43c108d20 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa441cdc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c113c90 con 0x7fa43c1083d0 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52626/0 (socket says 172.21.15.92:52626) 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.079+0000 7fa43b7fe640 1 -- 172.21.15.92:0/254815261 learned_addr learned my addr 172.21.15.92:0/254815261 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa432ffd640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c1083d0 0x7fa43c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43bfff640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108d20 0x7fa43c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43b7fe640 1 -- 172.21.15.92:0/254815261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c1083d0 msgr2=0x7fa43c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c1083d0 0x7fa43c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43b7fe640 1 -- 172.21.15.92:0/254815261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108d20 msgr2=0x7fa43c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108d20 0x7fa43c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43b7fe640 1 -- 172.21.15.92:0/254815261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c1a0470 con 0x7fa43c102ae0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43bfff640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108d20 0x7fa43c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa432ffd640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c1083d0 0x7fa43c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c19a500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa42c005940 tx=0x7fa42c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa4397fa640 1 -- 172.21.15.92:0/254815261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03f030 con 0x7fa43c102ae0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1a0700 con 0x7fa43c102ae0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa4397fa640 1 -- 172.21.15.92:0/254815261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42c003cb0 con 0x7fa43c102ae0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa4397fa640 1 -- 172.21.15.92:0/254815261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c035070 con 0x7fa43c102ae0 2024-09-21T02:07:32.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.080+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1a6ec0 con 0x7fa43c102ae0 2024-09-21T02:07:32.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 Processor -- start 2024-09-21T02:07:32.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa4397fa640 1 -- 172.21.15.92:0/254815261 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa42c042030 con 0x7fa43c102ae0 2024-09-21T02:07:32.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 -- start start 2024-09-21T02:07:32.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56101b14de50 con 0x56101ae0f390 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56101b13d280 con 0x56101ae071b0 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa448250740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56101b141090 con 0x56101ae30070 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa432ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa432ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52634/0 (socket says 172.21.15.92:52634) 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.083+0000 7fa432ffd640 1 -- 172.21.15.92:0/262768382 learned_addr learned my addr 172.21.15.92:0/262768382 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa43bfff640 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa432ffd640 1 -- 172.21.15.92:0/262768382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 msgr2=0x56101b1d7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa432ffd640 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1d7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa4397fa640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4280b6510 0x7fa4280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa432ffd640 1 -- 172.21.15.92:0/262768382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 msgr2=0x56101b1d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa432ffd640 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1d7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa432ffd640 1 -- 172.21.15.92:0/262768382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101ae9b650 con 0x56101ae0f390 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa43bfff640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4280b6510 0x7fa4280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa432ffd640 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1d80d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa42400a9a0 tx=0x7fa42400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e139077d4c0b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa4397fa640 1 -- 172.21.15.92:0/254815261 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa42c107c70 con 0x7fa43c102ae0 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa431ffb640 1 -- 172.21.15.92:0/262768382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424015070 con 0x56101ae0f390 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa431ffb640 1 -- 172.21.15.92:0/262768382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42400cc00 con 0x56101ae0f390 2024-09-21T02:07:32.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.084+0000 7fa431ffb640 1 -- 172.21.15.92:0/262768382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240105b0 con 0x56101ae0f390 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 -- 172.21.15.92:0/262768382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 msgr2=0x56101b1d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa43bfff640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4280b6510 0x7fa4280b8a20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa434005fd0 tx=0x7fa434009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1d80d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa42400a9a0 tx=0x7fa42400ae60 comp rx=0 tx=0).stop 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 -- 172.21.15.92:0/262768382 shutdown_connections 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1d80d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 --2- 172.21.15.92:0/262768382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 -- 172.21.15.92:0/262768382 >> 172.21.15.92:0/262768382 conn(0x56101ae24460 msgr2=0x56101b1d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 -- 172.21.15.92:0/262768382 shutdown_connections 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.085+0000 7fa448250740 1 -- 172.21.15.92:0/262768382 wait complete. 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 -- start start 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 Processor -- start 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56101af9e700 con 0x56101ae30070 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56101b0d8110 con 0x56101ae071b0 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56101b1e5e30 con 0x56101ae0f390 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa448250740 10 client.0 _ll_register_callbacks cb 0x56101b0f1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1e36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52640/0 (socket says 172.21.15.92:52640) 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.086+0000 7fa43bfff640 1 -- 172.21.15.92:0/784482358 learned_addr learned my addr 172.21.15.92:0/784482358 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa432ffd640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43bfff640 1 -- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 msgr2=0x56101b1e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43bfff640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1e3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43bfff640 1 -- 172.21.15.92:0/784482358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 msgr2=0x56101b1e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43bfff640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43bfff640 1 -- 172.21.15.92:0/784482358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101b105c40 con 0x56101ae30070 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa432ffd640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa43bfff640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1e36d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa434009950 tx=0x7fa434004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bbc6ca3303dfb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa41affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa41affd640 1 -- 172.21.15.92:0/784482358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434004d20 con 0x56101ae30070 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa41affd640 1 -- 172.21.15.92:0/784482358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa43401e590 con 0x56101ae30070 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42400cc00 con 0x56101ae30070 2024-09-21T02:07:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa41affd640 1 -- 172.21.15.92:0/784482358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa43401e790 con 0x56101ae30070 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa448250740 10 client.16329 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.087+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56101b1f1b10 con 0x56101ae30070 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa41affd640 1 -- 172.21.15.92:0/784482358 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42400cc00 con 0x56101ae30070 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa41affd640 1 -- 172.21.15.92:0/784482358 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa434005540 con 0x56101ae30070 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa448250740 10 client.16329 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa448250740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa448250740 20 client.16329 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa448250740 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adee9f0 0x56101b1f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa448250740 4 client.16329 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:32.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56101b14fda0 con 0x56101adee9f0 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.088+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adee9f0 0x56101b1f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.089+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adee9f0 0x56101b1f5090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6cd6b9bf57adf6 server_cookie=5919d5a2f9cb8c1c in_seq=0 out_seq=0 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.089+0000 7fa41affd640 10 client.16329 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.089+0000 7fa41affd640 1 -- 172.21.15.92:0/784482358 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56101b14fda0 con 0x56101adee9f0 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.089+0000 7fa41affd640 10 client.16329 handle_command_reply: tid=0 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.089+0000 7fa448250740 1 client.16329 shutdown 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 msgr2=0x56101b1e36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1e36d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa434009950 tx=0x7fa434004a40 comp rx=0 tx=0).stop 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 shutdown_connections 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adee9f0 0x56101b1f5090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1e36d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 --2- 172.21.15.92:0/784482358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 >> 172.21.15.92:0/784482358 conn(0x56101ae24460 msgr2=0x56101b1da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa41affd640 0 client.16329 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 shutdown_connections 2024-09-21T02:07:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 1 -- 172.21.15.92:0/784482358 wait complete. 2024-09-21T02:07:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.090+0000 7fa448250740 20 client.16329 trim_cache size 0 max 16384 2024-09-21T02:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 Processor -- start 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 -- start start 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1d50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1af530 con 0x56101ae30070 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b1070 con 0x56101ae071b0 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa448250740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1afc00 con 0x56101ae0f390 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1d50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1d50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52646/0 (socket says 172.21.15.92:52646) 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa43bfff640 1 -- 172.21.15.92:0/2254944193 learned_addr learned my addr 172.21.15.92:0/2254944193 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa432ffd640 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.107+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43bfff640 1 -- 172.21.15.92:0/2254944193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 msgr2=0x56101b1d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43bfff640 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43bfff640 1 -- 172.21.15.92:0/2254944193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 msgr2=0x56101b1e4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43bfff640 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43bfff640 1 -- 172.21.15.92:0/2254944193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101b1e1bb0 con 0x56101ae30070 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa43bfff640 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1d50b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa4340059e0 tx=0x7fa434009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bff1395fef95f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa4317fa640 1 -- 172.21.15.92:0/2254944193 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43401eb30 con 0x56101ae30070 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa4317fa640 1 -- 172.21.15.92:0/2254944193 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa43401ecd0 con 0x56101ae30070 2024-09-21T02:07:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.108+0000 7fa4317fa640 1 -- 172.21.15.92:0/2254944193 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43401d6f0 con 0x56101ae30070 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 -- 172.21.15.92:0/2254944193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 msgr2=0x56101b1d50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1d50b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa4340059e0 tx=0x7fa434009850 comp rx=0 tx=0).stop 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 -- 172.21.15.92:0/2254944193 shutdown_connections 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae30070 0x56101b1d50b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae0f390 0x56101b1d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 --2- 172.21.15.92:0/2254944193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 -- 172.21.15.92:0/2254944193 >> 172.21.15.92:0/2254944193 conn(0x56101ae24460 msgr2=0x56101b1e4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 -- 172.21.15.92:0/2254944193 shutdown_connections 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 -- 172.21.15.92:0/2254944193 wait complete. 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 -- start start 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.109+0000 7fa448250740 1 Processor -- start 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b09a0 con 0x56101ae0f390 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1b02d0 con 0x56101ae071b0 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ad510 con 0x56101ae30070 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa448250740 10 client.0 _ll_register_callbacks cb 0x7fa434009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1e6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52662/0 (socket says 172.21.15.92:52662) 2024-09-21T02:07:32.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 -- 172.21.15.92:0/2040006938 learned_addr learned my addr 172.21.15.92:0/2040006938 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa43bfff640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 -- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 msgr2=0x56101b1e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1e7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 -- 172.21.15.92:0/2040006938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 msgr2=0x56101b1eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 -- 172.21.15.92:0/2040006938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101b262e60 con 0x56101ae0f390 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa432ffd640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1e6f60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa424011d30 tx=0x7fa424011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6757903eae215fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.110+0000 7fa41affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa41affd640 1 -- 172.21.15.92:0/2040006938 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424015040 con 0x56101ae0f390 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa41affd640 1 -- 172.21.15.92:0/2040006938 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa4240115f0 con 0x56101ae0f390 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43401ecd0 con 0x56101ae0f390 2024-09-21T02:07:32.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa41affd640 1 -- 172.21.15.92:0/2040006938 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4240117f0 con 0x56101ae0f390 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 10 client.16335 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56101b263f90 con 0x56101ae0f390 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa41affd640 1 -- 172.21.15.92:0/2040006938 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa43401ecd0 con 0x56101ae0f390 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa41affd640 1 -- 172.21.15.92:0/2040006938 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa42401a450 con 0x56101ae0f390 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 10 client.16335 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 20 client.16335 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adc3da0 0x56101b266460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 4 client.16335 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.111+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56101b14b850 con 0x56101adc3da0 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.112+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adc3da0 0x56101b266460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.112+0000 7fa41affd640 10 client.16335 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.112+0000 7fa43b7fe640 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adc3da0 0x56101b266460 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95998c2f5121393a server_cookie=feb73052deb24867 in_seq=0 out_seq=0 2024-09-21T02:07:32.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.112+0000 7fa41affd640 1 -- 172.21.15.92:0/2040006938 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x56101b14b850 con 0x56101adc3da0 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.112+0000 7fa41affd640 10 client.16335 handle_command_reply: tid=0 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.112+0000 7fa448250740 1 client.16335 shutdown 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 msgr2=0x56101b1e6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1e6f60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa424011d30 tx=0x7fa424011db0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 shutdown_connections 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56101adc3da0 0x56101b266460 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56101ae30070 0x56101b1e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56101ae0f390 0x56101b1e6f60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 --2- 172.21.15.92:0/2040006938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56101ae071b0 0x56101b1eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 >> 172.21.15.92:0/2040006938 conn(0x56101ae24460 msgr2=0x56101b1d8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa41affd640 0 client.16335 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 shutdown_connections 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 1 -- 172.21.15.92:0/2040006938 wait complete. 2024-09-21T02:07:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa448250740 20 client.16335 trim_cache size 0 max 16384 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4", 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:32.119 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:32.120 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:32.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:07:32.122 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:07:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4280b6510 msgr2=0x7fa4280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4280b6510 0x7fa4280b8a20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa434005fd0 tx=0x7fa434009700 comp rx=0 tx=0).stop 2024-09-21T02:07:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 msgr2=0x7fa43c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c19a500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa42c005940 tx=0x7fa42c0042f0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 shutdown_connections 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4280b6510 0x7fa4280b8a20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108d20 0x7fa43c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c1083d0 0x7fa43c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 --2- 172.21.15.92:0/254815261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa43c102ae0 0x7fa43c19a500 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.113+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 >> 172.21.15.92:0/254815261 conn(0x7fa43c079760 msgr2=0x7fa43c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.114+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 shutdown_connections 2024-09-21T02:07:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:07:32.114+0000 7fa441cdc640 1 -- 172.21.15.92:0/254815261 wait complete. 2024-09-21T02:07:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:32 smithi136 ceph-mon[34754]: pgmap v437: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:32 smithi092 ceph-mon[31190]: pgmap v437: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:32 smithi099 ceph-mon[34404]: pgmap v437: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:32.838 INFO:tasks.cephfs.filesystem:scrub status for tag:43fb3d77-e7be-4f74-9a1f-0ad7813d2be4 - {'path': '/', 'tag': '43fb3d77-e7be-4f74-9a1f-0ad7813d2be4', 'options': 'recursive,force'} 2024-09-21T02:07:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.881+0000 7f39d8d39640 1 -- 172.21.15.92:0/3474984793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103640 msgr2=0x7f39d4103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.881+0000 7f39d8d39640 1 --2- 172.21.15.92:0/3474984793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103640 0x7f39d4103a50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009900 tx=0x7f39c802a410 comp rx=0 tx=0).stop 2024-09-21T02:07:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 -- 172.21.15.92:0/3474984793 shutdown_connections 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 --2- 172.21.15.92:0/3474984793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39d4103fe0 0x7f39d410a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 --2- 172.21.15.92:0/3474984793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103640 0x7f39d4103a50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 --2- 172.21.15.92:0/3474984793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39d41025c0 0x7f39d4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 -- 172.21.15.92:0/3474984793 >> 172.21.15.92:0/3474984793 conn(0x7f39d40fdf00 msgr2=0x7f39d41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 -- 172.21.15.92:0/3474984793 shutdown_connections 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 -- 172.21.15.92:0/3474984793 wait complete. 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 Processor -- start 2024-09-21T02:07:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.882+0000 7f39d8d39640 1 -- start start 2024-09-21T02:07:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39d41025c0 0x7f39d406c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39d4103640 0x7f39d419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 0x7f39d41a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410e210 con 0x7f39d4103fe0 2024-09-21T02:07:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410e090 con 0x7f39d41025c0 2024-09-21T02:07:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410e390 con 0x7f39d4103640 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 0x7f39d41a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 0x7f39d41a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52666/0 (socket says 172.21.15.92:52666) 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 -- 172.21.15.92:0/2048207244 learned_addr learned my addr 172.21.15.92:0/2048207244 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39d4103640 0x7f39d419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d259c640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39d41025c0 0x7f39d406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 -- 172.21.15.92:0/2048207244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39d4103640 msgr2=0x7f39d419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39d4103640 0x7f39d419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 -- 172.21.15.92:0/2048207244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39d41025c0 msgr2=0x7f39d406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39d41025c0 0x7f39d406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 -- 172.21.15.92:0/2048207244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d41a1a00 con 0x7f39d4103fe0 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d2d9d640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 0x7f39d41a1200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00eaa0 tx=0x7f39cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39c37fe640 1 -- 172.21.15.92:0/2048207244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00cdb0 con 0x7f39d4103fe0 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39c37fe640 1 -- 172.21.15.92:0/2048207244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc005020 con 0x7f39d4103fe0 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d41a1cf0 con 0x7f39d4103fe0 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39c37fe640 1 -- 172.21.15.92:0/2048207244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0116d0 con 0x7f39d4103fe0 2024-09-21T02:07:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.883+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d41a2190 con 0x7f39d4103fe0 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.886+0000 7f39dfca6740 1 Processor -- start 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.886+0000 7f39c37fe640 1 -- 172.21.15.92:0/2048207244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f39cc0051d0 con 0x7f39d4103fe0 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 -- start start 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac08e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cb4ab7b6f0 con 0x55cb4a8404d0 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb4ab6ab20 con 0x55cb4a8382f0 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39dfca6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb4ab6e930 con 0x55cb4a8611b0 2024-09-21T02:07:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac0a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52680/0 (socket says 172.21.15.92:52680) 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 -- 172.21.15.92:0/2683913985 learned_addr learned my addr 172.21.15.92:0/2683913985 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac08e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d259c640 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39c37fe640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39bc0b6510 0x7f39bc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 -- 172.21.15.92:0/2683913985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 msgr2=0x55cb4ac08e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac08e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 -- 172.21.15.92:0/2683913985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 msgr2=0x55cb4ac094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d1d9b640 1 -- 172.21.15.92:0/2683913985 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4ab30530 con 0x55cb4a8404d0 2024-09-21T02:07:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.887+0000 7f39d2d9d640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39bc0b6510 0x7f39bc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.888+0000 7f39c37fe640 1 -- 172.21.15.92:0/2048207244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f39cc0e73e0 con 0x7f39d4103fe0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.888+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac0a010 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f39c802a930 tx=0x7f39c8009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b73eccfa1c1597 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.888+0000 7f39c0ff9640 1 -- 172.21.15.92:0/2683913985 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8035040 con 0x55cb4a8404d0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.888+0000 7f39c0ff9640 1 -- 172.21.15.92:0/2683913985 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c8031cd0 con 0x55cb4a8404d0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.888+0000 7f39c0ff9640 1 -- 172.21.15.92:0/2683913985 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8003dc0 con 0x55cb4a8404d0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 -- 172.21.15.92:0/2683913985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 msgr2=0x55cb4ac0a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac0a010 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f39c802a930 tx=0x7f39c8009610 comp rx=0 tx=0).stop 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39d2d9d640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39bc0b6510 0x7f39bc0b8a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0aa870 tx=0x7f39cc0a1eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 -- 172.21.15.92:0/2683913985 shutdown_connections 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac0a010 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 --2- 172.21.15.92:0/2683913985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac08e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 -- 172.21.15.92:0/2683913985 >> 172.21.15.92:0/2683913985 conn(0x55cb4a855300 msgr2=0x55cb4ac048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 -- 172.21.15.92:0/2683913985 shutdown_connections 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 -- 172.21.15.92:0/2683913985 wait complete. 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 -- start start 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 Processor -- start 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8382f0 0x55cb4ac12620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.889+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8404d0 0x55cb4ac12ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac13850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cb4aaa2ab0 con 0x55cb4a8611b0 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb4ab10730 con 0x55cb4a8404d0 2024-09-21T02:07:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb4ac0aad0 con 0x55cb4a8382f0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 10 client.0 _ll_register_callbacks cb 0x55cb4ab646b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac13850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac13850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52690/0 (socket says 172.21.15.92:52690) 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 -- 172.21.15.92:0/1816875632 learned_addr learned my addr 172.21.15.92:0/1816875632 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8382f0 0x55cb4ac12620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d259c640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8404d0 0x55cb4ac12ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 -- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8382f0 msgr2=0x55cb4ac12620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8382f0 0x55cb4ac12620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 -- 172.21.15.92:0/1816875632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8404d0 msgr2=0x55cb4ac12ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8404d0 0x55cb4ac12ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 -- 172.21.15.92:0/1816875632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4a8cc790 con 0x55cb4a8611b0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac13850 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005ab0 tx=0x7f39c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd76eb7f1ee520e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39b9ffb640 1 -- 172.21.15.92:0/1816875632 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8035030 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39b9ffb640 1 -- 172.21.15.92:0/1816875632 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f39c8004340 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c8031cd0 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39b9ffb640 1 -- 172.21.15.92:0/1816875632 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39c803d480 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 10 client.16347 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.890+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb4a945620 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39b9ffb640 1 -- 172.21.15.92:0/1816875632 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c8031cd0 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39b9ffb640 1 -- 172.21.15.92:0/1816875632 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39c804b440 con 0x55cb4a8611b0 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39dfca6740 10 client.16347 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39dfca6740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39dfca6740 20 client.16347 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39dfca6740 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a81fd80 0x55cb4ac1f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39dfca6740 4 client.16347 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb4ab7d640 con 0x55cb4a81fd80 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.891+0000 7f39d259c640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a81fd80 0x55cb4ac1f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39b9ffb640 10 client.16347 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39d259c640 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a81fd80 0x55cb4ac1f930 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363950f901018de5 server_cookie=bd7dd71fab785ed0 in_seq=0 out_seq=0 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39b9ffb640 1 -- 172.21.15.92:0/1816875632 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb4ab7d640 con 0x55cb4a81fd80 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39b9ffb640 10 client.16347 handle_command_reply: tid=0 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39dfca6740 1 client.16347 shutdown 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 msgr2=0x55cb4ac13850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.892+0000 7f39dfca6740 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac13850 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005ab0 tx=0x7f39c802a410 comp rx=0 tx=0).stop 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 shutdown_connections 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a81fd80 0x55cb4ac1f930 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac13850 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8404d0 0x55cb4ac12ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39b9ffb640 0 client.16347 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 --2- 172.21.15.92:0/1816875632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8382f0 0x55cb4ac12620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 >> 172.21.15.92:0/1816875632 conn(0x55cb4a855300 msgr2=0x55cb4ac02ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 shutdown_connections 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 1 -- 172.21.15.92:0/1816875632 wait complete. 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.893+0000 7f39dfca6740 20 client.16347 trim_cache size 0 max 16384 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 Processor -- start 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 -- start start 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac08df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8404d0 0x55cb4ac09330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41aaff0 con 0x55cb4a8611b0 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41acb30 con 0x55cb4a8382f0 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39dfca6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41ab6c0 con 0x55cb4a8404d0 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac04d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52696/0 (socket says 172.21.15.92:52696) 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 -- 172.21.15.92:0/972291122 learned_addr learned my addr 172.21.15.92:0/972291122 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d259c640 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8404d0 0x55cb4ac09330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 -- 172.21.15.92:0/972291122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8404d0 msgr2=0x55cb4ac09330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8404d0 0x55cb4ac09330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 -- 172.21.15.92:0/972291122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 msgr2=0x55cb4ac08df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.907+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac08df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.908+0000 7f39d1d9b640 1 -- 172.21.15.92:0/972291122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4ac1fca0 con 0x55cb4a8611b0 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.908+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac04d80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005a80 tx=0x7f39c8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485b7c1bbaf68b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.908+0000 7f39bbfff640 1 -- 172.21.15.92:0/972291122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8035040 con 0x55cb4a8611b0 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.908+0000 7f39bbfff640 1 -- 172.21.15.92:0/972291122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c8009520 con 0x55cb4a8611b0 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.908+0000 7f39bbfff640 1 -- 172.21.15.92:0/972291122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c804bca0 con 0x55cb4a8611b0 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- 172.21.15.92:0/972291122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 msgr2=0x55cb4ac04d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac04d80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005a80 tx=0x7f39c8001e70 comp rx=0 tx=0).stop 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- 172.21.15.92:0/972291122 shutdown_connections 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8611b0 0x55cb4ac04d80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8404d0 0x55cb4ac09330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- 172.21.15.92:0/972291122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac08df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- 172.21.15.92:0/972291122 >> 172.21.15.92:0/972291122 conn(0x55cb4a855300 msgr2=0x55cb4abff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- 172.21.15.92:0/972291122 shutdown_connections 2024-09-21T02:07:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- 172.21.15.92:0/972291122 wait complete. 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- start start 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 Processor -- start 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41ac460 con 0x55cb4a8404d0 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41abd90 con 0x55cb4a8382f0 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39d41a8fd0 con 0x55cb4a8611b0 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39dfca6740 10 client.0 _ll_register_callbacks cb 0x7f39c8031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39d259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39d259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac17f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52698/0 (socket says 172.21.15.92:52698) 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.909+0000 7f39d259c640 1 -- 172.21.15.92:0/3759331272 learned_addr learned my addr 172.21.15.92:0/3759331272 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d259c640 1 -- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 msgr2=0x55cb4ac18ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d259c640 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac18ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d259c640 1 -- 172.21.15.92:0/3759331272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 msgr2=0x55cb4ac178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d259c640 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d259c640 1 -- 172.21.15.92:0/3759331272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4ac19230 con 0x55cb4a8404d0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39d259c640 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac17f60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004700 tx=0x7f39c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35779863a78db167 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39b9ffb640 1 -- 172.21.15.92:0/3759331272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c401d3b0 con 0x55cb4a8404d0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39b9ffb640 1 -- 172.21.15.92:0/3759331272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f39c401d550 con 0x55cb4a8404d0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c8009520 con 0x55cb4a8404d0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39b9ffb640 1 -- 172.21.15.92:0/3759331272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39c401d750 con 0x55cb4a8404d0 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39dfca6740 10 client.16353 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb4ac1a620 con 0x55cb4a8404d0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39b9ffb640 1 -- 172.21.15.92:0/3759331272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c8009520 con 0x55cb4a8404d0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.910+0000 7f39b9ffb640 1 -- 172.21.15.92:0/3759331272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39c40239a0 con 0x55cb4a8404d0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39dfca6740 10 client.16353 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39dfca6740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39dfca6740 20 client.16353 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39dfca6740 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a7f51c0 0x55cb4ac8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39dfca6740 4 client.16353 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb4ab790f0 con 0x55cb4a7f51c0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a7f51c0 0x55cb4ac8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39d1d9b640 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a7f51c0 0x55cb4ac8c670 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7e28b19f4eb0b server_cookie=4621d8a997a8ac08 in_seq=0 out_seq=0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.911+0000 7f39b9ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.924+0000 7f39b9ffb640 1 -- 172.21.15.92:0/3759331272 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb4ab790f0 con 0x55cb4a7f51c0 2024-09-21T02:07:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.924+0000 7f39b9ffb640 10 client.16353 handle_command_reply: tid=0 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.924+0000 7f39dfca6740 1 client.16353 shutdown 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 msgr2=0x55cb4ac17f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac17f60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004700 tx=0x7f39c4011810 comp rx=0 tx=0).stop 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 shutdown_connections 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55cb4a7f51c0 0x55cb4ac8c670 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb4a8611b0 0x55cb4ac18ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39b9ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cb4a8404d0 0x55cb4ac17f60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 --2- 172.21.15.92:0/3759331272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb4a8382f0 0x55cb4ac178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 >> 172.21.15.92:0/3759331272 conn(0x55cb4a855300 msgr2=0x55cb4ac0cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 shutdown_connections 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 1 -- 172.21.15.92:0/3759331272 wait complete. 2024-09-21T02:07:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39dfca6740 20 client.16353 trim_cache size 0 max 16384 2024-09-21T02:07:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:07:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:07:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:07:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:07:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39bc0b6510 msgr2=0x7f39bc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39bc0b6510 0x7f39bc0b8a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0aa870 tx=0x7f39cc0a1eb0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 msgr2=0x7f39d41a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 0x7f39d41a1200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00eaa0 tx=0x7f39cc00ef60 comp rx=0 tx=0).stop 2024-09-21T02:07:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 shutdown_connections 2024-09-21T02:07:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f39bc0b6510 0x7f39bc0b8a20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d4103fe0 0x7f39d41a1200 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39d4103640 0x7f39d419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 --2- 172.21.15.92:0/2048207244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39d41025c0 0x7f39d406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.925+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 >> 172.21.15.92:0/2048207244 conn(0x7f39d40fdf00 msgr2=0x7f39d4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.926+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 shutdown_connections 2024-09-21T02:07:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:32.926+0000 7f39d8d39640 1 -- 172.21.15.92:0/2048207244 wait complete. 2024-09-21T02:07:33.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:33 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:07:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:33 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:07:33.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:33 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:07:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:34 smithi136 ceph-mon[34754]: pgmap v438: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:07:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:34 smithi092 ceph-mon[31190]: pgmap v438: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:07:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:34 smithi099 ceph-mon[34404]: pgmap v438: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:07:36.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:36 smithi136 ceph-mon[34754]: pgmap v439: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:07:36.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:36 smithi092 ceph-mon[31190]: pgmap v439: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:07:36.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:36 smithi099 ceph-mon[34404]: pgmap v439: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:64/600: open fd = 3 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:65/600: open fd = 3 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.659 INFO:tasks.workunit.client.0.smithi092.stdout:66/600: open fd = 3 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:67/600: open fd = 3 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:68/600: open fd = 3 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.660 INFO:tasks.workunit.client.0.smithi092.stdout:69/600: open fd = 3 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:70/600: open fd = 3 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:71/600: open fd = 3 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:72/600: open fd = 3 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.661 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:73/600: open fd = 3 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:74/600: open fd = 3 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:75/600: open fd = 3 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.662 INFO:tasks.workunit.client.0.smithi092.stdout:76/600: open fd = 3 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:77/600: open fd = 3 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:78/600: open fd = 3 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:79/600: open fd = 3 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.663 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:80/600: open fd = 3 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:81/600: open fd = 3 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:82/600: open fd = 3 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.664 INFO:tasks.workunit.client.0.smithi092.stdout:83/600: open fd = 3 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:84/600: open fd = 3 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:85/600: open fd = 3 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:86/600: open fd = 3 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.665 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:87/600: open fd = 3 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:88/600: open fd = 3 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:89/600: open fd = 3 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.666 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:90/600: open fd = 3 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:91/600: open fd = 3 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:92/600: open fd = 3 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.667 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:93/600: open fd = 3 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:94/600: open fd = 3 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:95/600: open fd = 3 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.668 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:96/600: open fd = 3 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:97/600: open fd = 3 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:98/600: open fd = 3 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:99/600: open fd = 3 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.669 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:100/600: open fd = 3 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:101/600: open fd = 3 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:102/600: open fd = 3 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.670 INFO:tasks.workunit.client.0.smithi092.stdout:103/600: open fd = 3 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:104/600: open fd = 3 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:105/600: open fd = 3 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.671 INFO:tasks.workunit.client.0.smithi092.stdout:106/600: open fd = 3 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:107/600: open fd = 3 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:108/600: open fd = 3 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:109/600: open fd = 3 2024-09-21T02:07:38.672 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:110/600: open fd = 3 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:111/600: open fd = 3 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:112/600: open fd = 3 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.673 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:113/600: open fd = 3 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:114/600: open fd = 3 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:115/600: open fd = 3 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:116/600: open fd = 3 2024-09-21T02:07:38.674 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:117/600: open fd = 3 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:118/600: open fd = 3 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:119/600: open fd = 3 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.675 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:120/600: open fd = 3 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:121/600: open fd = 3 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:122/600: open fd = 3 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:123/600: open fd = 3 2024-09-21T02:07:38.676 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:124/600: open fd = 3 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:125/600: open fd = 3 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:126/600: open fd = 3 2024-09-21T02:07:38.677 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:07:38.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:38 smithi136 ceph-mon[34754]: pgmap v440: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:38.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:38 smithi092 ceph-mon[31190]: pgmap v440: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:38.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:38 smithi099 ceph-mon[34404]: pgmap v440: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:40.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:40 smithi136 ceph-mon[34754]: pgmap v441: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:07:40.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:07:40.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:40 smithi092 ceph-mon[31190]: pgmap v441: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:07:40.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:07:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:40 smithi099 ceph-mon[34404]: pgmap v441: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:07:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:07:41.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:07:41.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:07:41.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:07:41.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:07:41.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:07:41.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:07:41.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:07:41.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:07:41.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:07:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:42 smithi136 ceph-mon[34754]: pgmap v442: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:07:43.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:42 smithi092 ceph-mon[31190]: pgmap v442: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:07:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:42 smithi099 ceph-mon[34404]: pgmap v442: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:07:45.128 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:44 smithi092 ceph-mon[31190]: pgmap v443: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:07:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:44 smithi136 ceph-mon[34754]: pgmap v443: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:07:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:44 smithi099 ceph-mon[34404]: pgmap v443: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:07:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 -- 172.21.15.92:0/3357446887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b8103640 msgr2=0x7f88b8103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 --2- 172.21.15.92:0/3357446887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b8103640 0x7f88b8103a50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002a410 comp rx=0 tx=0).stop 2024-09-21T02:07:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 -- 172.21.15.92:0/3357446887 shutdown_connections 2024-09-21T02:07:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 --2- 172.21.15.92:0/3357446887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103fe0 0x7f88b810a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 --2- 172.21.15.92:0/3357446887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b8103640 0x7f88b8103a50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 --2- 172.21.15.92:0/3357446887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b81025c0 0x7f88b8102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 -- 172.21.15.92:0/3357446887 >> 172.21.15.92:0/3357446887 conn(0x7f88b80fdf00 msgr2=0x7f88b81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 -- 172.21.15.92:0/3357446887 shutdown_connections 2024-09-21T02:07:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.023+0000 7f88bcff1640 1 -- 172.21.15.92:0/3357446887 wait complete. 2024-09-21T02:07:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 Processor -- start 2024-09-21T02:07:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 -- start start 2024-09-21T02:07:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 0x7f88b806c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103640 0x7f88b819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b8103fe0 0x7f88b81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810e1f0 con 0x7f88b81025c0 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810e070 con 0x7f88b8103fe0 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88bcff1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f88b810e370 con 0x7f88b8103640 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88b659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 0x7f88b806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88b659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 0x7f88b806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47952/0 (socket says 172.21.15.92:47952) 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.024+0000 7f88b659c640 1 -- 172.21.15.92:0/1106188285 learned_addr learned my addr 172.21.15.92:0/1106188285 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b8103fe0 0x7f88b81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88affff640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103640 0x7f88b819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b659c640 1 -- 172.21.15.92:0/1106188285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103640 msgr2=0x7f88b819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b659c640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103640 0x7f88b819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b659c640 1 -- 172.21.15.92:0/1106188285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b8103fe0 msgr2=0x7f88b81a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b659c640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b8103fe0 0x7f88b81a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b659c640 1 -- 172.21.15.92:0/1106188285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a19d0 con 0x7f88b81025c0 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88affff640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103640 0x7f88b819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b8103fe0 0x7f88b81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88b659c640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 0x7f88b806c170 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d950 tx=0x7f88a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88af7fe640 1 -- 172.21.15.92:0/1106188285 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8017030 con 0x7f88b81025c0 2024-09-21T02:07:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88af7fe640 1 -- 172.21.15.92:0/1106188285 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a8004020 con 0x7f88b81025c0 2024-09-21T02:07:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a1cc0 con 0x7f88b81025c0 2024-09-21T02:07:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88af7fe640 1 -- 172.21.15.92:0/1106188285 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80047a0 con 0x7f88b81025c0 2024-09-21T02:07:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.025+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a21b0 con 0x7f88b81025c0 2024-09-21T02:07:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88af7fe640 1 -- 172.21.15.92:0/1106188285 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f88a8004940 con 0x7f88b81025c0 2024-09-21T02:07:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 Processor -- start 2024-09-21T02:07:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 -- start start 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39aa91b0 0x556e39e55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a802f0 0x556e39e558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 0x556e39e56420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556e39dc36f0 con 0x556e39a884d0 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556e39db2b20 con 0x556e39a802f0 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88c4041740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556e39db6930 con 0x556e39aa91b0 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.028+0000 7f88affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 0x556e39e56420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 0x556e39e56420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47958/0 (socket says 172.21.15.92:47958) 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 -- 172.21.15.92:0/3359176713 learned_addr learned my addr 172.21.15.92:0/3359176713 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39aa91b0 0x556e39e55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88b659c640 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a802f0 0x556e39e558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88af7fe640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f88a40b6590 0x7f88a40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 -- 172.21.15.92:0/3359176713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39aa91b0 msgr2=0x556e39e55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39aa91b0 0x556e39e55260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 -- 172.21.15.92:0/3359176713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a802f0 msgr2=0x556e39e558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a802f0 0x556e39e558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 -- 172.21.15.92:0/3359176713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e39d78530 con 0x556e39a884d0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f88a40b6590 0x7f88a40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88af7fe640 1 -- 172.21.15.92:0/1106188285 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f88a80e6e40 con 0x7f88b81025c0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88affff640 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 0x556e39e56420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671baba076d46fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88acff9640 1 -- 172.21.15.92:0/3359176713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a003f070 con 0x556e39a884d0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88acff9640 1 -- 172.21.15.92:0/3359176713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00318c0 con 0x556e39a884d0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.029+0000 7f88acff9640 1 -- 172.21.15.92:0/3359176713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0035030 con 0x556e39a884d0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 -- 172.21.15.92:0/3359176713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 msgr2=0x556e39e56420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f88a40b6590 0x7f88a40b8aa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007f70 tx=0x7f88b0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 0x556e39e56420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a00316d0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 -- 172.21.15.92:0/3359176713 shutdown_connections 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a884d0 0x556e39e56420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a802f0 0x556e39e558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 --2- 172.21.15.92:0/3359176713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39aa91b0 0x556e39e55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.030+0000 7f88c4041740 1 -- 172.21.15.92:0/3359176713 >> 172.21.15.92:0/3359176713 conn(0x556e39a9d300 msgr2=0x556e39e50cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 -- 172.21.15.92:0/3359176713 shutdown_connections 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 -- 172.21.15.92:0/3359176713 wait complete. 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 -- start start 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 Processor -- start 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a802f0 0x556e39e5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e5f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 0x556e39e5fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556e39ceaab0 con 0x556e39aa91b0 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556e39d58730 con 0x556e39a884d0 2024-09-21T02:07:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556e39e56ee0 con 0x556e39a802f0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88c4041740 10 client.0 _ll_register_callbacks cb 0x556e39dac6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88b6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 0x556e39e5fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88b6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 0x556e39e5fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47960/0 (socket says 172.21.15.92:47960) 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.031+0000 7f88b6d9d640 1 -- 172.21.15.92:0/2085409310 learned_addr learned my addr 172.21.15.92:0/2085409310 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b659c640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a802f0 0x556e39e5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b6d9d640 1 -- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a802f0 msgr2=0x556e39e5e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a802f0 0x556e39e5e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b6d9d640 1 -- 172.21.15.92:0/2085409310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 msgr2=0x556e39e5f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88affff640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e5f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e5f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b6d9d640 1 -- 172.21.15.92:0/2085409310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e39b14790 con 0x556e39aa91b0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 0x556e39e5fba0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88b0012e50 tx=0x7f88b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98187cadfee37bda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f889dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f889dffb640 1 -- 172.21.15.92:0/2085409310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0027670 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f889dffb640 1 -- 172.21.15.92:0/2085409310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f88b0027810 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e39dc5640 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f889dffb640 1 -- 172.21.15.92:0/2085409310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88b0027a10 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88c4041740 10 client.16365 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e39b8d620 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.032+0000 7f889dffb640 1 -- 172.21.15.92:0/2085409310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556e39dc5640 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f889dffb640 1 -- 172.21.15.92:0/2085409310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88b001b580 con 0x556e39aa91b0 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88c4041740 10 client.16365 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88c4041740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88c4041740 20 client.16365 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88c4041740 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39a67d80 0x556e39e6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88c4041740 4 client.16365 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e39dc10f0 con 0x556e39a67d80 2024-09-21T02:07:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.033+0000 7f88b659c640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39a67d80 0x556e39e6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.034+0000 7f889dffb640 10 client.16365 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.034+0000 7f88b659c640 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39a67d80 0x556e39e6c270 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8712d38c2db42090 server_cookie=57da68f5a8d999d3 in_seq=0 out_seq=0 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.034+0000 7f889dffb640 1 -- 172.21.15.92:0/2085409310 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e39dc10f0 con 0x556e39a67d80 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.034+0000 7f889dffb640 10 client.16365 handle_command_reply: tid=0 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.034+0000 7f88c4041740 1 client.16365 shutdown 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 msgr2=0x556e39e5fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 0x556e39e5fba0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88b0012e50 tx=0x7f88b0004ab0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 shutdown_connections 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39a67d80 0x556e39e6c270 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39aa91b0 0x556e39e5fba0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e5f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f889dffb640 0 client.16365 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 --2- 172.21.15.92:0/2085409310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a802f0 0x556e39e5e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 >> 172.21.15.92:0/2085409310 conn(0x556e39a9d300 msgr2=0x556e39e4aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 shutdown_connections 2024-09-21T02:07:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 1 -- 172.21.15.92:0/2085409310 wait complete. 2024-09-21T02:07:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.035+0000 7f88c4041740 20 client.16365 trim_cache size 0 max 16384 2024-09-21T02:07:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 Processor -- start 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 -- start start 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a67d80 0x556e39e6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a884d0 0x556e39e51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81a8da0 con 0x556e39a802f0 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81aa8e0 con 0x556e39a67d80 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88c4041740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81a9470 con 0x556e39a884d0 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47972/0 (socket says 172.21.15.92:47972) 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88affff640 1 -- 172.21.15.92:0/2222685756 learned_addr learned my addr 172.21.15.92:0/2222685756 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.049+0000 7f88b659c640 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a67d80 0x556e39e6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a884d0 0x556e39e51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88affff640 1 -- 172.21.15.92:0/2222685756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a884d0 msgr2=0x556e39e51cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88affff640 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a884d0 0x556e39e51cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88affff640 1 -- 172.21.15.92:0/2222685756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a67d80 msgr2=0x556e39e6c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88affff640 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a67d80 0x556e39e6c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88affff640 1 -- 172.21.15.92:0/2222685756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e39e5cb30 con 0x556e39a802f0 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f88affff640 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e517b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005940 tx=0x7f88a00097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c786b5a7a21c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f889ffff640 1 -- 172.21.15.92:0/2222685756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a004b020 con 0x556e39a802f0 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f889ffff640 1 -- 172.21.15.92:0/2222685756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a0031d90 con 0x556e39a802f0 2024-09-21T02:07:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.050+0000 7f889ffff640 1 -- 172.21.15.92:0/2222685756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0035070 con 0x556e39a802f0 2024-09-21T02:07:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 -- 172.21.15.92:0/2222685756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 msgr2=0x556e39e517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e517b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005940 tx=0x7f88a00097c0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 -- 172.21.15.92:0/2222685756 shutdown_connections 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a884d0 0x556e39e51cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e517b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 --2- 172.21.15.92:0/2222685756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a67d80 0x556e39e6c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 -- 172.21.15.92:0/2222685756 >> 172.21.15.92:0/2222685756 conn(0x556e39a9d300 msgr2=0x556e39e475c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 -- 172.21.15.92:0/2222685756 shutdown_connections 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 -- 172.21.15.92:0/2222685756 wait complete. 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 -- start start 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.051+0000 7f88c4041740 1 Processor -- start 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a67d80 0x556e39e663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e66ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81aa210 con 0x556e39a802f0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81a9b40 con 0x556e39a884d0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81a6d80 con 0x556e39a67d80 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 10 client.0 _ll_register_callbacks cb 0x7f88b0007c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e66ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e66ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47984/0 (socket says 172.21.15.92:47984) 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 -- 172.21.15.92:0/1050189222 learned_addr learned my addr 172.21.15.92:0/1050189222 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88b659c640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a67d80 0x556e39e663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88b6d9d640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 -- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a67d80 msgr2=0x556e39e663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a67d80 0x556e39e663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 -- 172.21.15.92:0/1050189222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 msgr2=0x556e39e65260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e65260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 -- 172.21.15.92:0/1050189222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e39e659f0 con 0x556e39a802f0 2024-09-21T02:07:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88b659c640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a67d80 0x556e39e663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88affff640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e66ab0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f88a0031440 tx=0x7f88a0003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ec89062a7e488c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f889dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f889dffb640 1 -- 172.21.15.92:0/1050189222 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a004b030 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f889dffb640 1 -- 172.21.15.92:0/1050189222 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f88a00040e0 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.052+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a0031d90 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f889dffb640 1 -- 172.21.15.92:0/1050189222 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88a0061050 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 10 client.16371 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e39e648e0 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f889dffb640 1 -- 172.21.15.92:0/1050189222 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a0031d90 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f889dffb640 1 -- 172.21.15.92:0/1050189222 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88a003bcb0 con 0x556e39a802f0 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 10 client.16371 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 20 client.16371 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39aa91b0 0x556e39ed8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 4 client.16371 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:07:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e39dbe520 con 0x556e39aa91b0 2024-09-21T02:07:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.053+0000 7f88b659c640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39aa91b0 0x556e39ed8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.054+0000 7f889dffb640 10 client.16371 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.054+0000 7f88b659c640 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39aa91b0 0x556e39ed8e50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56be309de5dc473f server_cookie=277c2ebcf5d0e56f in_seq=0 out_seq=0 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f889dffb640 1 -- 172.21.15.92:0/1050189222 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e39dbe520 con 0x556e39aa91b0 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f889dffb640 10 client.16371 handle_command_reply: tid=0 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 client.16371 shutdown 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 msgr2=0x556e39e66ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e66ab0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f88a0031440 tx=0x7f88a0003880 comp rx=0 tx=0).stop 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 shutdown_connections 2024-09-21T02:07:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556e39aa91b0 0x556e39ed8e50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e39a884d0 0x556e39e65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e39a802f0 0x556e39e66ab0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f889dffb640 0 client.16371 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 --2- 172.21.15.92:0/1050189222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556e39a67d80 0x556e39e663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 >> 172.21.15.92:0/1050189222 conn(0x556e39a9d300 msgr2=0x556e39e517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 shutdown_connections 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 1 -- 172.21.15.92:0/1050189222 wait complete. 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.067+0000 7f88c4041740 20 client.16371 trim_cache size 0 max 16384 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:07:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:07:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:07:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f88a40b6590 msgr2=0x7f88a40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f88a40b6590 0x7f88a40b8aa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007f70 tx=0x7f88b0004a40 comp rx=0 tx=0).stop 2024-09-21T02:07:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 msgr2=0x7f88b806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 0x7f88b806c170 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d950 tx=0x7f88a800de10 comp rx=0 tx=0).stop 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 shutdown_connections 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f88a40b6590 0x7f88a40b8aa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88b8103fe0 0x7f88b81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88b8103640 0x7f88b819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 --2- 172.21.15.92:0/1106188285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88b81025c0 0x7f88b806c170 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 >> 172.21.15.92:0/1106188285 conn(0x7f88b80fdf00 msgr2=0x7f88b8106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 shutdown_connections 2024-09-21T02:07:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:46.068+0000 7f88bcff1640 1 -- 172.21.15.92:0/1106188285 wait complete. 2024-09-21T02:07:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:46 smithi136 ceph-mon[34754]: pgmap v444: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:47.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:46 smithi092 ceph-mon[31190]: pgmap v444: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:47.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:46 smithi099 ceph-mon[34404]: pgmap v444: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:07:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:48 smithi136 ceph-mon[34754]: pgmap v445: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:07:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:48 smithi092 ceph-mon[31190]: pgmap v445: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:07:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:48 smithi099 ceph-mon[34404]: pgmap v445: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:07:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:50 smithi136 ceph-mon[34754]: pgmap v446: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:50 smithi092 ceph-mon[31190]: pgmap v446: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:50 smithi099 ceph-mon[34404]: pgmap v446: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:07:53.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:52 smithi136 ceph-mon[34754]: pgmap v447: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:52 smithi092 ceph-mon[31190]: pgmap v447: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:53.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:52 smithi099 ceph-mon[34404]: pgmap v447: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:07:55.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:54 smithi136 ceph-mon[34754]: pgmap v448: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:54 smithi092 ceph-mon[31190]: pgmap v448: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:54 smithi099 ceph-mon[34404]: pgmap v448: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:07:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:56 smithi136 ceph-mon[34754]: pgmap v449: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:07:57.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:56 smithi092 ceph-mon[31190]: pgmap v449: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:07:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:56 smithi099 ceph-mon[34404]: pgmap v449: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:07:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 -- 172.21.15.92:0/617387185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0390104560 msgr2=0x7f0390104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 --2- 172.21.15.92:0/617387185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0390104560 0x7f0390104970 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0378009900 tx=0x7f037802a410 comp rx=0 tx=0).stop 2024-09-21T02:07:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 -- 172.21.15.92:0/617387185 shutdown_connections 2024-09-21T02:07:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 --2- 172.21.15.92:0/617387185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390104f00 0x7f039010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 --2- 172.21.15.92:0/617387185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0390104560 0x7f0390104970 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 --2- 172.21.15.92:0/617387185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03901034e0 0x7f0390103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 -- 172.21.15.92:0/617387185 >> 172.21.15.92:0/617387185 conn(0x7f03900fee20 msgr2=0x7f03901012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 -- 172.21.15.92:0/617387185 shutdown_connections 2024-09-21T02:07:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.173+0000 7f03949fe640 1 -- 172.21.15.92:0/617387185 wait complete. 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 Processor -- start 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 -- start start 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 0x7f0390199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390104560 0x7f039019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0390104f00 0x7f039019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f039010f120 con 0x7f03901034e0 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f039010efa0 con 0x7f0390104560 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f03949fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f039010f2a0 con 0x7f0390104f00 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f038ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 0x7f0390199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f038ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 0x7f0390199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47604/0 (socket says 172.21.15.92:47604) 2024-09-21T02:07:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.174+0000 7f038ed9d640 1 -- 172.21.15.92:0/2759422788 learned_addr learned my addr 172.21.15.92:0/2759422788 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038f59e640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0390104f00 0x7f039019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038e59c640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390104560 0x7f039019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038ed9d640 1 -- 172.21.15.92:0/2759422788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0390104f00 msgr2=0x7f039019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038ed9d640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0390104f00 0x7f039019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038ed9d640 1 -- 172.21.15.92:0/2759422788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390104560 msgr2=0x7f039019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038ed9d640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390104560 0x7f039019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038ed9d640 1 -- 172.21.15.92:0/2759422788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039019b740 con 0x7f03901034e0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f038ed9d640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 0x7f0390199e00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f038000e9f0 tx=0x7f038000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f0387fff640 1 -- 172.21.15.92:0/2759422788 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038000cd80 con 0x7f03901034e0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039019ba30 con 0x7f03901034e0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f0387fff640 1 -- 172.21.15.92:0/2759422788 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03800054d0 con 0x7f03901034e0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f0387fff640 1 -- 172.21.15.92:0/2759422788 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380010690 con 0x7f03901034e0 2024-09-21T02:07:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.175+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03901a21b0 con 0x7f03901034e0 2024-09-21T02:07:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 Processor -- start 2024-09-21T02:07:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 -- start start 2024-09-21T02:07:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f0387fff640 1 -- 172.21.15.92:0/2759422788 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f0380004e90 con 0x7f03901034e0 2024-09-21T02:07:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 0x55f16527c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16527c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f16527d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f1651ee6f0 con 0x55f164ed41b0 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f1651ddb20 con 0x55f164eb34d0 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f039c244740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1651e1930 con 0x55f164eab2f0 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 0x55f16527c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 0x55f16527c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47606/0 (socket says 172.21.15.92:47606) 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f038f59e640 1 -- 172.21.15.92:0/2367894739 learned_addr learned my addr 172.21.15.92:0/2367894739 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.178+0000 7f038e59c640 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f16527d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038ed9d640 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16527c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038f59e640 1 -- 172.21.15.92:0/2367894739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 msgr2=0x55f16527c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038f59e640 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16527c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038f59e640 1 -- 172.21.15.92:0/2367894739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 msgr2=0x55f16527d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038f59e640 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f16527d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038f59e640 1 -- 172.21.15.92:0/2367894739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1651a3530 con 0x55f164ed41b0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038e59c640 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f16527d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038f59e640 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 0x55f16527c040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0388007f40 tx=0x7f038800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9620d2ea354d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f03857fa640 1 -- 172.21.15.92:0/2367894739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388010900 con 0x55f164ed41b0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f03857fa640 1 -- 172.21.15.92:0/2367894739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03880183f0 con 0x55f164ed41b0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f03857fa640 1 -- 172.21.15.92:0/2367894739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017600 con 0x55f164ed41b0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f0387fff640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f037c0b6510 0x7f037c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.179+0000 7f038ed9d640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f037c0b6510 0x7f037c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f0387fff640 1 -- 172.21.15.92:0/2759422788 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f03800e77c0 con 0x7f03901034e0 2024-09-21T02:07:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 -- 172.21.15.92:0/2367894739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 msgr2=0x55f16527c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f038ed9d640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f037c0b6510 0x7f037c0b8a20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03800ad630 tx=0x7f03800d1040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 0x55f16527c040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0388007f40 tx=0x7f038800f850 comp rx=0 tx=0).stop 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 -- 172.21.15.92:0/2367894739 shutdown_connections 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f16527d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16527c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 --2- 172.21.15.92:0/2367894739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164ed41b0 0x55f16527c040 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 -- 172.21.15.92:0/2367894739 >> 172.21.15.92:0/2367894739 conn(0x55f164ec8300 msgr2=0x55f165277ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 -- 172.21.15.92:0/2367894739 shutdown_connections 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.180+0000 7f039c244740 1 -- 172.21.15.92:0/2367894739 wait complete. 2024-09-21T02:07:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 -- start start 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 Processor -- start 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 0x55f165285790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165285e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164ed41b0 0x55f1652869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f165115ab0 con 0x55f164eab2f0 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f165183730 con 0x55f164eb34d0 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f16527dd20 con 0x55f164ed41b0 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f039c244740 10 client.0 _ll_register_callbacks cb 0x55f1651d76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 0x55f165285790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 0x55f165285790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47616/0 (socket says 172.21.15.92:47616) 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 -- 172.21.15.92:0/1925824382 learned_addr learned my addr 172.21.15.92:0/1925824382 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038e59c640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165285e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164ed41b0 0x55f1652869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 -- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164ed41b0 msgr2=0x55f1652869c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164ed41b0 0x55f1652869c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 -- 172.21.15.92:0/1925824382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 msgr2=0x55f165285e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165285e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.181+0000 7f038f59e640 1 -- 172.21.15.92:0/1925824382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f164f3f790 con 0x55f164eab2f0 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f038f59e640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 0x55f165285790 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f80 tx=0x7f03880072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8137f7b8fe2cc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f03767fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f03767fc640 1 -- 172.21.15.92:0/1925824382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038802a060 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f03767fc640 1 -- 172.21.15.92:0/1925824382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0388004a40 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03880183f0 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f03767fc640 1 -- 172.21.15.92:0/1925824382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0388004c40 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 10 client.16383 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f164fb8620 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f03767fc640 1 -- 172.21.15.92:0/1925824382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03880183f0 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f03767fc640 1 -- 172.21.15.92:0/1925824382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f03880041d0 con 0x55f164eab2f0 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 10 client.16383 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 20 client.16383 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164e92d80 0x55f165292c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 4 client.16383 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:07:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.182+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1651f0640 con 0x55f164e92d80 2024-09-21T02:07:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.183+0000 7f038e59c640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164e92d80 0x55f165292c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.183+0000 7f038e59c640 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164e92d80 0x55f165292c10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c8134a9bb702ad server_cookie=2f5312f830b91787 in_seq=0 out_seq=0 2024-09-21T02:07:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.183+0000 7f03767fc640 10 client.16383 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f03767fc640 1 -- 172.21.15.92:0/1925824382 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1651f0640 con 0x55f164e92d80 2024-09-21T02:07:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f03767fc640 10 client.16383 handle_command_reply: tid=0 2024-09-21T02:07:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 client.16383 shutdown 2024-09-21T02:07:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 msgr2=0x55f165285790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 0x55f165285790 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f80 tx=0x7f03880072a0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 shutdown_connections 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164e92d80 0x55f165292c10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164ed41b0 0x55f1652869c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165285e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f03767fc640 0 client.16383 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 --2- 172.21.15.92:0/1925824382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eab2f0 0x55f165285790 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 >> 172.21.15.92:0/1925824382 conn(0x55f164ec8300 msgr2=0x55f165276150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.184+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 shutdown_connections 2024-09-21T02:07:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.185+0000 7f039c244740 1 -- 172.21.15.92:0/1925824382 wait complete. 2024-09-21T02:07:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.185+0000 7f039c244740 20 client.16383 trim_cache size 0 max 16384 2024-09-21T02:07:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 Processor -- start 2024-09-21T02:07:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 -- start start 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 0x55f1652808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f165276150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165276690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03901aae00 con 0x55f164e92d80 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03901ac940 con 0x55f164eb34d0 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.199+0000 7f039c244740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03901ab4d0 con 0x55f164eab2f0 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 0x55f1652808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 0x55f1652808a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47618/0 (socket says 172.21.15.92:47618) 2024-09-21T02:07:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 -- 172.21.15.92:0/3928373713 learned_addr learned my addr 172.21.15.92:0/3928373713 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038e59c640 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f165276150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 -- 172.21.15.92:0/3928373713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 msgr2=0x55f165276150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f165276150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 -- 172.21.15.92:0/3928373713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 msgr2=0x55f165276690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165276690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 -- 172.21.15.92:0/3928373713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f165280de0 con 0x55f164e92d80 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f038f59e640 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 0x55f1652808a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f03880045a0 tx=0x7f0388005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a6283b08a1a02a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f0384ff9640 1 -- 172.21.15.92:0/3928373713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038803d070 con 0x55f164e92d80 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f0384ff9640 1 -- 172.21.15.92:0/3928373713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f038800fd50 con 0x55f164e92d80 2024-09-21T02:07:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.200+0000 7f0384ff9640 1 -- 172.21.15.92:0/3928373713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388034040 con 0x55f164e92d80 2024-09-21T02:07:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 -- 172.21.15.92:0/3928373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 msgr2=0x55f1652808a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 0x55f1652808a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f03880045a0 tx=0x7f0388005bf0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 -- 172.21.15.92:0/3928373713 shutdown_connections 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164eb34d0 0x55f165276690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f165276150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 --2- 172.21.15.92:0/3928373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164e92d80 0x55f1652808a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 -- 172.21.15.92:0/3928373713 >> 172.21.15.92:0/3928373713 conn(0x55f164ec8300 msgr2=0x55f165280190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 -- 172.21.15.92:0/3928373713 shutdown_connections 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.201+0000 7f039c244740 1 -- 172.21.15.92:0/3928373713 wait complete. 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 -- start start 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 Processor -- start 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164e92d80 0x55f16528d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16528c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 0x55f16528c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03901ac270 con 0x55f164eb34d0 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03901abba0 con 0x55f164e92d80 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03901a8de0 con 0x55f164eab2f0 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f039c244740 10 client.0 _ll_register_callbacks cb 0x7f0388007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 0x55f16528c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 0x55f16528c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47624/0 (socket says 172.21.15.92:47624) 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 -- 172.21.15.92:0/3966505430 learned_addr learned my addr 172.21.15.92:0/3966505430 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038e59c640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16528c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164e92d80 0x55f16528d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 -- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 msgr2=0x55f16528c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16528c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 -- 172.21.15.92:0/3966505430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164e92d80 msgr2=0x55f16528d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164e92d80 0x55f16528d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.202+0000 7f038f59e640 1 -- 172.21.15.92:0/3966505430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16528d070 con 0x55f164eb34d0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f038f59e640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 0x55f16528c8e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f03880189f0 tx=0x7f0388007fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bbe5fb8f00d734 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f03767fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f03767fc640 1 -- 172.21.15.92:0/3966505430 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038803d040 con 0x55f164eb34d0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f03767fc640 1 -- 172.21.15.92:0/3966505430 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0388004810 con 0x55f164eb34d0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038800fd50 con 0x55f164eb34d0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f03767fc640 1 -- 172.21.15.92:0/3966505430 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f038804a050 con 0x55f164eb34d0 2024-09-21T02:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 10 client.16389 fetch_fsmap learned FSMap version 29 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1652fd4f0 con 0x55f164eb34d0 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f03767fc640 1 -- 172.21.15.92:0/3966505430 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f038800fd50 con 0x55f164eb34d0 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f03767fc640 1 -- 172.21.15.92:0/3966505430 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0388010ca0 con 0x55f164eb34d0 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 10 client.16389 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 20 client.16389 populate_metadata read hostname 'smithi092' 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164ed41b0 0x55f1652ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 4 client.16389 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.203+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1651ec0f0 con 0x55f164ed41b0 2024-09-21T02:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.204+0000 7f038e59c640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164ed41b0 0x55f1652ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.204+0000 7f03767fc640 10 client.16389 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.204+0000 7f038e59c640 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164ed41b0 0x55f1652ffe50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a19f878b5cc742 server_cookie=41eb97f69cde5af8 in_seq=0 out_seq=0 2024-09-21T02:07:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f03767fc640 1 -- 172.21.15.92:0/3966505430 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1651ec0f0 con 0x55f164ed41b0 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f03767fc640 10 client.16389 handle_command_reply: tid=0 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 client.16389 shutdown 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 msgr2=0x55f16528c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 0x55f16528c8e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f03880189f0 tx=0x7f0388007fb0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 shutdown_connections 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f164ed41b0 0x55f1652ffe50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f164eb34d0 0x55f16528c8e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f164eab2f0 0x55f16528c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f03767fc640 0 client.16389 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 --2- 172.21.15.92:0/3966505430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f164e92d80 0x55f16528d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.218+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 >> 172.21.15.92:0/3966505430 conn(0x55f164ec8300 msgr2=0x55f1652841b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 shutdown_connections 2024-09-21T02:07:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f039c244740 1 -- 172.21.15.92:0/3966505430 wait complete. 2024-09-21T02:07:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f039c244740 20 client.16389 trim_cache size 0 max 16384 2024-09-21T02:07:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:07:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:07:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:07:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f037c0b6510 msgr2=0x7f037c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f037c0b6510 0x7f037c0b8a20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03800ad630 tx=0x7f03800d1040 comp rx=0 tx=0).stop 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 msgr2=0x7f0390199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 0x7f0390199e00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f038000e9f0 tx=0x7f038000edf0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 shutdown_connections 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f037c0b6510 0x7f037c0b8a20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0390104f00 0x7f039019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390104560 0x7f039019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 --2- 172.21.15.92:0/2759422788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03901034e0 0x7f0390199e00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 >> 172.21.15.92:0/2759422788 conn(0x7f03900fee20 msgr2=0x7f0390107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 shutdown_connections 2024-09-21T02:07:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:07:59.219+0000 7f03949fe640 1 -- 172.21.15.92:0/2759422788 wait complete. 2024-09-21T02:07:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:07:58 smithi136 ceph-mon[34754]: pgmap v450: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:59.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:07:58 smithi092 ceph-mon[31190]: pgmap v450: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:07:59.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:07:58 smithi099 ceph-mon[34404]: pgmap v450: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:00 smithi136 ceph-mon[34754]: pgmap v451: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:00 smithi092 ceph-mon[31190]: pgmap v451: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:00 smithi099 ceph-mon[34404]: pgmap v451: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:02.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:08:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:03.233 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:02 smithi092 ceph-mon[31190]: pgmap v452: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:08:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:02 smithi136 ceph-mon[34754]: pgmap v452: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:08:03.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:02 smithi099 ceph-mon[34404]: pgmap v452: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:08:03.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 -- 172.21.15.92:0/2796774466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100108cf0 msgr2=0x7fe10010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 --2- 172.21.15.92:0/2796774466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100108cf0 0x7fe10010f520 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009ca0 tx=0x7fe0f802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 -- 172.21.15.92:0/2796774466 shutdown_connections 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 --2- 172.21.15.92:0/2796774466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100108cf0 0x7fe10010f520 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 --2- 172.21.15.92:0/2796774466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1001083a0 0x7fe1001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 --2- 172.21.15.92:0/2796774466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe100102a60 0x7fe100102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 -- 172.21.15.92:0/2796774466 >> 172.21.15.92:0/2796774466 conn(0x7fe100079730 msgr2=0x7fe100079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 -- 172.21.15.92:0/2796774466 shutdown_connections 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 -- 172.21.15.92:0/2796774466 wait complete. 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.622+0000 7fe108042640 1 Processor -- start 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 -- start start 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 0x7fe10019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1001083a0 0x7fe10019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe100108cf0 0x7fe1001a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe100112960 con 0x7fe100102a60 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001127e0 con 0x7fe1001083a0 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe108042640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe100112ae0 con 0x7fe100108cf0 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 0x7fe10019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 0x7fe10019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41444/0 (socket says 172.21.15.92:41444) 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 -- 172.21.15.92:0/332198912 learned_addr learned my addr 172.21.15.92:0/332198912 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe1065df640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe100108cf0 0x7fe1001a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe1055dd640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1001083a0 0x7fe10019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 -- 172.21.15.92:0/332198912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe100108cf0 msgr2=0x7fe1001a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe100108cf0 0x7fe1001a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 -- 172.21.15.92:0/332198912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1001083a0 msgr2=0x7fe10019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1001083a0 0x7fe10019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 -- 172.21.15.92:0/332198912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1001a2550 con 0x7fe100102a60 2024-09-21T02:08:03.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe1065df640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe100108cf0 0x7fe1001a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.623+0000 7fe105dde640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 0x7fe10019c6a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c920 tx=0x7fe0fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.624+0000 7fe0f6ffd640 1 -- 172.21.15.92:0/332198912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc007c70 con 0x7fe100102a60 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.624+0000 7fe0f6ffd640 1 -- 172.21.15.92:0/332198912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0fc007e10 con 0x7fe100102a60 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.624+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1001a2840 con 0x7fe100102a60 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.624+0000 7fe0f6ffd640 1 -- 172.21.15.92:0/332198912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc018820 con 0x7fe100102a60 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.624+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1001a9060 con 0x7fe100102a60 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.626+0000 7fe0f6ffd640 1 -- 172.21.15.92:0/332198912 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe0fc0189c0 con 0x7fe100102a60 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 Processor -- start 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 -- start start 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e73650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e73b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 0x55e407e740d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e407de9e50 con 0x55e407aab390 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e407dd9280 con 0x55e407acc070 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e407ddd090 con 0x55e407aa31b0 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe1055dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 0x55e407e740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe1065df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e73650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe1065df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e73650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:40554/0 (socket says 172.21.15.92:40554) 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe1055dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 0x55e407e740d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41454/0 (socket says 172.21.15.92:41454) 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.627+0000 7fe1065df640 1 -- 172.21.15.92:0/3622175038 learned_addr learned my addr 172.21.15.92:0/3622175038 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe0f6ffd640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0ec0b6540 0x7fe0ec0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe105dde640 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e73b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 -- 172.21.15.92:0/3622175038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 msgr2=0x55e407e73b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e73b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.639 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 -- 172.21.15.92:0/3622175038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 msgr2=0x55e407e740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 0x55e407e740d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 -- 172.21.15.92:0/3622175038 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e407b37650 con 0x55e407acc070 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1055dd640 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 0x55e407e740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe0f6ffd640 1 -- 172.21.15.92:0/332198912 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe0fc0e7570 con 0x7fe100102a60 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0ec0b6540 0x7fe0ec0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e73650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005cc0 tx=0x7fe0f8031b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5335c1318f6788c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe1065df640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0ec0b6540 0x7fe0ec0b8a50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0f804c000 tx=0x7fe0f8049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe0ebfff640 1 -- 172.21.15.92:0/3622175038 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8003b30 con 0x55e407acc070 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe0ebfff640 1 -- 172.21.15.92:0/3622175038 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f8036040 con 0x55e407acc070 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.628+0000 7fe0ebfff640 1 -- 172.21.15.92:0/3622175038 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f802f860 con 0x55e407acc070 2024-09-21T02:08:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 -- 172.21.15.92:0/3622175038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 msgr2=0x55e407e73650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e73650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005cc0 tx=0x7fe0f8031b70 comp rx=0 tx=0).stop 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 -- 172.21.15.92:0/3622175038 shutdown_connections 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aab390 0x55e407e740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e73b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/3622175038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e73650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 -- 172.21.15.92:0/3622175038 >> 172.21.15.92:0/3622175038 conn(0x55e407ac0460 msgr2=0x55e407e71540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 -- 172.21.15.92:0/3622175038 shutdown_connections 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.629+0000 7fe10e5b6740 1 -- 172.21.15.92:0/3622175038 wait complete. 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.630+0000 7fe10e5b6740 1 -- start start 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.630+0000 7fe10e5b6740 1 Processor -- start 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407aab390 0x55e407e7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.641 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e407c3a700 con 0x55e407acc070 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e407d74110 con 0x55e407aab390 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e407e81cb0 con 0x55e407aa31b0 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe10e5b6740 10 client.0 _ll_register_callbacks cb 0x55e407d8d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e8afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41470/0 (socket says 172.21.15.92:41470) 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 -- 172.21.15.92:0/2793268254 learned_addr learned my addr 172.21.15.92:0/2793268254 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe105dde640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1055dd640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407aab390 0x55e407e7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 -- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 msgr2=0x55e407e7ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e7ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 -- 172.21.15.92:0/2793268254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407aab390 msgr2=0x55e407e7f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407aab390 0x55e407e7f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 -- 172.21.15.92:0/2793268254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e407da1c40 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe1065df640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e8afa0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe0f802ade0 tx=0x7fe0f8004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2023d7b1a63fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe0e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.631+0000 7fe0e97fa640 1 -- 172.21.15.92:0/2793268254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f802f350 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe0e97fa640 1 -- 172.21.15.92:0/2793268254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe0f804e050 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f8036040 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe0e97fa640 1 -- 172.21.15.92:0/2793268254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0f8004420 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 10 client.16401 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e407e8d9d0 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe0e97fa640 1 -- 172.21.15.92:0/2793268254 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f8036040 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe0e97fa640 1 -- 172.21.15.92:0/2793268254 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0f80306e0 con 0x55e407acc070 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 10 client.16401 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 20 client.16401 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407a8a9f0 0x55e407e90d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 4 client.16401 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.632+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e407debda0 con 0x55e407a8a9f0 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.633+0000 7fe105dde640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407a8a9f0 0x55e407e90d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.633+0000 7fe0e97fa640 10 client.16401 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.633+0000 7fe105dde640 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407a8a9f0 0x55e407e90d30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f70fda34886c40 server_cookie=2f419175b663e03c in_seq=0 out_seq=0 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe0e97fa640 1 -- 172.21.15.92:0/2793268254 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e407debda0 con 0x55e407a8a9f0 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe0e97fa640 10 client.16401 handle_command_reply: tid=0 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 client.16401 shutdown 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 msgr2=0x55e407e8afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e8afa0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe0f802ade0 tx=0x7fe0f8004780 comp rx=0 tx=0).stop 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 shutdown_connections 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407a8a9f0 0x55e407e90d30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e8afa0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407aab390 0x55e407e7f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe0e97fa640 0 client.16401 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/2793268254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aa31b0 0x55e407e7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 >> 172.21.15.92:0/2793268254 conn(0x55e407ac0460 msgr2=0x55e407e76830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 shutdown_connections 2024-09-21T02:08:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 1 -- 172.21.15.92:0/2793268254 wait complete. 2024-09-21T02:08:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.634+0000 7fe10e5b6740 20 client.16401 trim_cache size 0 max 16384 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 Processor -- start 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 -- start start 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 0x55e407e76e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e8af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e70e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001b1770 con 0x55e407aa31b0 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001b32b0 con 0x55e407acc070 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001b1e40 con 0x55e407aab390 2024-09-21T02:08:03.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe105dde640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 0x55e407e76e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe105dde640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 0x55e407e76e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41480/0 (socket says 172.21.15.92:41480) 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe105dde640 1 -- 172.21.15.92:0/1662177993 learned_addr learned my addr 172.21.15.92:0/1662177993 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe1055dd640 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e8af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe1065df640 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e70e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe105dde640 1 -- 172.21.15.92:0/1662177993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 msgr2=0x55e407e8af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.652+0000 7fe105dde640 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e8af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe105dde640 1 -- 172.21.15.92:0/1662177993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 msgr2=0x55e407e70e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe105dde640 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e70e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe105dde640 1 -- 172.21.15.92:0/1662177993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e407e75940 con 0x55e407aa31b0 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe1065df640 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e70e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe105dde640 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 0x55e407e76e60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c8f0 tx=0x7fe0fc0e4f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6896b601dab51475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe0eb7fe640 1 -- 172.21.15.92:0/1662177993 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0ad1f0 con 0x55e407aa31b0 2024-09-21T02:08:03.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe0eb7fe640 1 -- 172.21.15.92:0/1662177993 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0fc0adde0 con 0x55e407aa31b0 2024-09-21T02:08:03.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.653+0000 7fe0eb7fe640 1 -- 172.21.15.92:0/1662177993 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0a9950 con 0x55e407aa31b0 2024-09-21T02:08:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 -- 172.21.15.92:0/1662177993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 msgr2=0x55e407e76e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 0x55e407e76e60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c8f0 tx=0x7fe0fc0e4f20 comp rx=0 tx=0).stop 2024-09-21T02:08:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 -- 172.21.15.92:0/1662177993 shutdown_connections 2024-09-21T02:08:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407acc070 0x55e407e70e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e8af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/1662177993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407aa31b0 0x55e407e76e60 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 -- 172.21.15.92:0/1662177993 >> 172.21.15.92:0/1662177993 conn(0x55e407ac0460 msgr2=0x55e407e76750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 -- 172.21.15.92:0/1662177993 shutdown_connections 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 -- 172.21.15.92:0/1662177993 wait complete. 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.654+0000 7fe10e5b6740 1 -- start start 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 Processor -- start 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e92320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407a5fda0 0x55e407e92860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001b2be0 con 0x55e407acc070 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001b2510 con 0x55e407a5fda0 2024-09-21T02:08:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001af750 con 0x55e407aab390 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe10e5b6740 10 client.0 _ll_register_callbacks cb 0x7fe0f804b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe1055dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e92320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe1055dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e92320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41486/0 (socket says 172.21.15.92:41486) 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe1055dd640 1 -- 172.21.15.92:0/934258565 learned_addr learned my addr 172.21.15.92:0/934258565 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe105dde640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.655+0000 7fe1065df640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407a5fda0 0x55e407e92860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1055dd640 1 -- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 msgr2=0x55e407e91de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1055dd640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e91de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1055dd640 1 -- 172.21.15.92:0/934258565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407a5fda0 msgr2=0x55e407e92860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1055dd640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407a5fda0 0x55e407e92860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1055dd640 1 -- 172.21.15.92:0/934258565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e407efa530 con 0x55e407acc070 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1065df640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407a5fda0 0x55e407e92860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe1055dd640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e92320 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe10006aaa0 tx=0x7fe0f0010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45bd5078092270b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe0e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:03.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe0e97fa640 1 -- 172.21.15.92:0/934258565 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000fe30 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe0e97fa640 1 -- 172.21.15.92:0/934258565 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe0f0004020 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0fc0adde0 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe0e97fa640 1 -- 172.21.15.92:0/934258565 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0f0004220 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe10e5b6740 10 client.16407 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e407efb820 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.656+0000 7fe0e97fa640 1 -- 172.21.15.92:0/934258565 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0fc0adde0 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe0e97fa640 1 -- 172.21.15.92:0/934258565 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0f001a070 con 0x55e407acc070 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe10e5b6740 10 client.16407 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe10e5b6740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe10e5b6740 20 client.16407 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407efc000 0x55e407efe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe10e5b6740 4 client.16407 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e407de7850 con 0x55e407efc000 2024-09-21T02:08:03.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.657+0000 7fe105dde640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407efc000 0x55e407efe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe0e97fa640 10 client.16407 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe105dde640 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407efc000 0x55e407efe420 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ee1698ac6431f8 server_cookie=abba6814872750c4 in_seq=0 out_seq=0 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe0e97fa640 1 -- 172.21.15.92:0/934258565 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55e407de7850 con 0x55e407efc000 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe0e97fa640 10 client.16407 handle_command_reply: tid=0 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 client.16407 shutdown 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 msgr2=0x55e407e92320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e92320 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe10006aaa0 tx=0x7fe0f0010eb0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 shutdown_connections 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e407efc000 0x55e407efe420 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e407a5fda0 0x55e407e92860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e407acc070 0x55e407e92320 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe0e97fa640 0 client.16407 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 --2- 172.21.15.92:0/934258565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e407aab390 0x55e407e91de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 >> 172.21.15.92:0/934258565 conn(0x55e407ac0460 msgr2=0x55e407e70e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:03.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 shutdown_connections 2024-09-21T02:08:03.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 1 -- 172.21.15.92:0/934258565 wait complete. 2024-09-21T02:08:03.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.658+0000 7fe10e5b6740 20 client.16407 trim_cache size 0 max 16384 2024-09-21T02:08:03.664 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:08:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:08:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:08:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:08:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:08:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:08:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:08:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:03.669 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0ec0b6540 msgr2=0x7fe0ec0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0ec0b6540 0x7fe0ec0b8a50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0f804c000 tx=0x7fe0f8049000 comp rx=0 tx=0).stop 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 msgr2=0x7fe10019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 0x7fe10019c6a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c920 tx=0x7fe0fc00cde0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 shutdown_connections 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0ec0b6540 0x7fe0ec0b8a50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe100108cf0 0x7fe1001a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1001083a0 0x7fe10019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 --2- 172.21.15.92:0/332198912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe100102a60 0x7fe10019c6a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 >> 172.21.15.92:0/332198912 conn(0x7fe100079730 msgr2=0x7fe10006af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 shutdown_connections 2024-09-21T02:08:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:03.659+0000 7fe108042640 1 -- 172.21.15.92:0/332198912 wait complete. 2024-09-21T02:08:04.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:08:04.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:08:04.807 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:05.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:04 smithi092 ceph-mon[31190]: pgmap v453: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:08:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:04 smithi136 ceph-mon[34754]: pgmap v453: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 -- 172.21.15.92:0/300647964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f81094a0 msgr2=0x7f41f81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 --2- 172.21.15.92:0/300647964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f81094a0 0x7f41f81098b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009900 tx=0x7f41e002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 -- 172.21.15.92:0/300647964 shutdown_connections 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 --2- 172.21.15.92:0/300647964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8109df0 0x7f41f8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 --2- 172.21.15.92:0/300647964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f81094a0 0x7f41f81098b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 --2- 172.21.15.92:0/300647964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41f8103b60 0x7f41f8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 -- 172.21.15.92:0/300647964 >> 172.21.15.92:0/300647964 conn(0x7f41f807b850 msgr2=0x7f41f807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.301+0000 7f41fdfef640 1 -- 172.21.15.92:0/300647964 shutdown_connections 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.302+0000 7f41fdfef640 1 -- 172.21.15.92:0/300647964 wait complete. 2024-09-21T02:08:05.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.302+0000 7f41fdfef640 1 Processor -- start 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.302+0000 7f41fdfef640 1 -- start start 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8103b60 0x7f41f819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41f81094a0 0x7f41f819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 0x7f41f819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8113af0 con 0x7f41f8109df0 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8113970 con 0x7f41f8103b60 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8113c70 con 0x7f41f81094a0 2024-09-21T02:08:05.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 0x7f41f819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 0x7f41f819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41512/0 (socket says 172.21.15.92:41512) 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 -- 172.21.15.92:0/845769701 learned_addr learned my addr 172.21.15.92:0/845769701 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f6ffd640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41f81094a0 0x7f41f819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f77fe640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8103b60 0x7f41f819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 -- 172.21.15.92:0/845769701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41f81094a0 msgr2=0x7f41f819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41f81094a0 0x7f41f819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 -- 172.21.15.92:0/845769701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8103b60 msgr2=0x7f41f819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8103b60 0x7f41f819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 -- 172.21.15.92:0/845769701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81a0450 con 0x7f41f8109df0 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f77fe640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8103b60 0x7f41f819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:05.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f7fff640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 0x7f41f819fd20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f41f000b970 tx=0x7f41f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0019070 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81a0740 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f0014440 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.303+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00075d0 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.304+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81a6ea0 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.305+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8103f70 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.306+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f41f001c070 con 0x7f41f8109df0 2024-09-21T02:08:05.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.307+0000 7f41f4ff9640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f41e40b6510 0x7f41e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:05.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.307+0000 7f41f77fe640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f41e40b6510 0x7f41e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:05.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.307+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f41f00e67d0 con 0x7f41f8109df0 2024-09-21T02:08:05.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.307+0000 7f41f77fe640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f41e40b6510 0x7f41e40b8a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009b40 tx=0x7f41e8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:05.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.311+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41f00b1420 con 0x7f41f8109df0 2024-09-21T02:08:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:04 smithi099 ceph-mon[34404]: pgmap v453: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:08:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.526+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41f810b650 con 0x7f41f8109df0 2024-09-21T02:08:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.527+0000 7f41f4ff9640 1 -- 172.21.15.92:0/845769701 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f41f00b6360 con 0x7f41f8109df0 2024-09-21T02:08:05.528 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:08:05.528 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:08:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:08:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f41e40b6510 msgr2=0x7f41e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f41e40b6510 0x7f41e40b8a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009b40 tx=0x7f41e8005e50 comp rx=0 tx=0).stop 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 msgr2=0x7f41f819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 0x7f41f819fd20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f41f000b970 tx=0x7f41f000be30 comp rx=0 tx=0).stop 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 shutdown_connections 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f41e40b6510 0x7f41e40b8a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f8109df0 0x7f41f819fd20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41f81094a0 0x7f41f819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 --2- 172.21.15.92:0/845769701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f8103b60 0x7f41f819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 >> 172.21.15.92:0/845769701 conn(0x7f41f807b850 msgr2=0x7f41f8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 shutdown_connections 2024-09-21T02:08:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:05.529+0000 7f41fdfef640 1 -- 172.21.15.92:0/845769701 wait complete. 2024-09-21T02:08:06.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:08:06.173 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/845769701' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:08:06.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:05 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/845769701' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:08:06.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:05 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/845769701' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:08:06.395 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:06.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 -- 172.21.15.92:0/1750780695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc108cf0 msgr2=0x7fc3fc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 --2- 172.21.15.92:0/1750780695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc108cf0 0x7fc3fc10f520 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009ca0 tx=0x7fc3f402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 -- 172.21.15.92:0/1750780695 shutdown_connections 2024-09-21T02:08:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 --2- 172.21.15.92:0/1750780695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc108cf0 0x7fc3fc10f520 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 --2- 172.21.15.92:0/1750780695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 --2- 172.21.15.92:0/1750780695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3fc102a60 0x7fc3fc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 -- 172.21.15.92:0/1750780695 >> 172.21.15.92:0/1750780695 conn(0x7fc3fc079730 msgr2=0x7fc3fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 -- 172.21.15.92:0/1750780695 shutdown_connections 2024-09-21T02:08:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.874+0000 7fc402304640 1 -- 172.21.15.92:0/1750780695 wait complete. 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.875+0000 7fc402304640 1 Processor -- start 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.875+0000 7fc402304640 1 -- start start 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc402304640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3fc102a60 0x7fc3fc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc402304640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc402304640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3fc108cf0 0x7fc3fc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc402304640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc112960 con 0x7fc3fc1083a0 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc402304640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1127e0 con 0x7fc3fc108cf0 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc402304640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc112ae0 con 0x7fc3fc102a60 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41526/0 (socket says 172.21.15.92:41526) 2024-09-21T02:08:06.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 -- 172.21.15.92:0/1250406670 learned_addr learned my addr 172.21.15.92:0/1250406670 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3fc102a60 0x7fc3fc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc4008a1640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3fc108cf0 0x7fc3fc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 -- 172.21.15.92:0/1250406670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3fc102a60 msgr2=0x7fc3fc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3fc102a60 0x7fc3fc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 -- 172.21.15.92:0/1250406670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3fc108cf0 msgr2=0x7fc3fc19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3fc108cf0 0x7fc3fc19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 -- 172.21.15.92:0/1250406670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc19f2c0 con 0x7fc3fc1083a0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1998f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009830 tx=0x7fc3f0009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.876+0000 7fc3f97fa640 1 -- 172.21.15.92:0/1250406670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0005020 con 0x7fc3fc1083a0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.877+0000 7fc3f97fa640 1 -- 172.21.15.92:0/1250406670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f00047a0 con 0x7fc3fc1083a0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.877+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc19f5b0 con 0x7fc3fc1083a0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.877+0000 7fc3f97fa640 1 -- 172.21.15.92:0/1250406670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0007b20 con 0x7fc3fc1083a0 2024-09-21T02:08:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.877+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc06ae60 con 0x7fc3fc1083a0 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.879+0000 7fc408878740 1 Processor -- start 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.879+0000 7fc408878740 1 -- start start 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.879+0000 7fc3f97fa640 1 -- 172.21.15.92:0/1250406670 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc3f0004950 con 0x7fc3fc1083a0 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.879+0000 7fc408878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 0x557f178032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.879+0000 7fc408878740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f174331b0 0x557f17803800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc408878740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17803d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc408878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557f17779ea0 con 0x557f1745c070 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc408878740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557f177692d0 con 0x557f1743b390 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc408878740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557f1776d0e0 con 0x557f174331b0 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc4008a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 0x557f178032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc4008a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 0x557f178032c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41542/0 (socket says 172.21.15.92:41542) 2024-09-21T02:08:06.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc4008a1640 1 -- 172.21.15.92:0/926346884 learned_addr learned my addr 172.21.15.92:0/926346884 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f174331b0 0x557f17803800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17803d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 -- 172.21.15.92:0/926346884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 msgr2=0x557f17803d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17803d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 -- 172.21.15.92:0/926346884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 msgr2=0x557f178032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 0x557f178032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 -- 172.21.15.92:0/926346884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f174c7650 con 0x557f174331b0 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc4008a1640 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 0x557f178032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3f97fa640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3e40b6510 0x7fc3e40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc4008a1640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3e40b6510 0x7fc3e40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.880+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f174331b0 0x557f17803800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00bc20 tx=0x7fc3ec0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b7f7bb4cf659ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:06.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc3f97fa640 1 -- 172.21.15.92:0/1250406670 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc3f00e86f0 con 0x7fc3fc1083a0 2024-09-21T02:08:06.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc3e27fc640 1 -- 172.21.15.92:0/926346884 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec004030 con 0x557f174331b0 2024-09-21T02:08:06.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc3e27fc640 1 -- 172.21.15.92:0/926346884 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0035f0 con 0x557f174331b0 2024-09-21T02:08:06.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc3e27fc640 1 -- 172.21.15.92:0/926346884 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0116a0 con 0x557f174331b0 2024-09-21T02:08:06.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc408878740 1 -- 172.21.15.92:0/926346884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f174331b0 msgr2=0x557f17803800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc4008a1640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3e40b6510 0x7fc3e40b8a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc3f402f500 tx=0x7fc3f4009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:06.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.881+0000 7fc408878740 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f174331b0 0x557f17803800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00bc20 tx=0x7fc3ec0107a0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 -- 172.21.15.92:0/926346884 shutdown_connections 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17803d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f174331b0 0x557f17803800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 --2- 172.21.15.92:0/926346884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f1745c070 0x557f178032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 -- 172.21.15.92:0/926346884 >> 172.21.15.92:0/926346884 conn(0x557f17450460 msgr2=0x557f178011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 -- 172.21.15.92:0/926346884 shutdown_connections 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 -- 172.21.15.92:0/926346884 wait complete. 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 -- start start 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.882+0000 7fc408878740 1 Processor -- start 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17812cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17813230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17813790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557f175ca700 con 0x557f174331b0 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557f17704160 con 0x557f1743b390 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557f17815dd0 con 0x557f1745c070 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc408878740 10 client.0 _ll_register_callbacks cb 0x557f1771d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17812cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17812cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41548/0 (socket says 172.21.15.92:41548) 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 -- 172.21.15.92:0/747489884 learned_addr learned my addr 172.21.15.92:0/747489884 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc4008a1640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17813790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17813230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 -- 172.21.15.92:0/747489884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 msgr2=0x557f17813790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17813790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 -- 172.21.15.92:0/747489884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 msgr2=0x557f17813230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17813230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 -- 172.21.15.92:0/747489884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f17731c90 con 0x557f174331b0 2024-09-21T02:08:06.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17812cf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b4ef6d66d60d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.883+0000 7fc3db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc3db7fe640 1 -- 172.21.15.92:0/747489884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00bdd0 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc3db7fe640 1 -- 172.21.15.92:0/747489884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc3ec0037a0 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3ec0035f0 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc3db7fe640 1 -- 172.21.15.92:0/747489884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3ec0039a0 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 10 client.16425 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f17821a90 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc3db7fe640 1 -- 172.21.15.92:0/747489884 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0035f0 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc3db7fe640 1 -- 172.21.15.92:0/747489884 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc3ec0049c0 con 0x557f174331b0 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 10 client.16425 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 20 client.16425 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:06.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f1741a9f0 0x557f17824df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 4 client.16425 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f1777bdf0 con 0x557f1741a9f0 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.884+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f1741a9f0 0x557f17824df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.885+0000 7fc3db7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.885+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f1741a9f0 0x557f17824df0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f671753c4bcda7ea server_cookie=a309662cf00302c9 in_seq=0 out_seq=0 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.885+0000 7fc3db7fe640 1 -- 172.21.15.92:0/747489884 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f1777bdf0 con 0x557f1741a9f0 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.885+0000 7fc3db7fe640 10 client.16425 handle_command_reply: tid=0 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.885+0000 7fc408878740 1 client.16425 shutdown 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 msgr2=0x557f17812cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17812cf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec004230 comp rx=0 tx=0).stop 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 shutdown_connections 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f1741a9f0 0x557f17824df0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17813790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17813230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc3db7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 --2- 172.21.15.92:0/747489884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17812cf0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 >> 172.21.15.92:0/747489884 conn(0x557f17450460 msgr2=0x557f17806450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 shutdown_connections 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 1 -- 172.21.15.92:0/747489884 wait complete. 2024-09-21T02:08:06.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.886+0000 7fc408878740 20 client.16425 trim_cache size 0 max 16384 2024-09-21T02:08:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 Processor -- start 2024-09-21T02:08:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 -- start start 2024-09-21T02:08:06.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17814bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17804a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17800760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1ad260 con 0x557f174331b0 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1aeda0 con 0x557f1743b390 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc408878740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1ad930 con 0x557f1745c070 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17814bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17814bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41558/0 (socket says 172.21.15.92:41558) 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc3fbfff640 1 -- 172.21.15.92:0/651263470 learned_addr learned my addr 172.21.15.92:0/651263470 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc4008a1640 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17800760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.899+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17804a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fbfff640 1 -- 172.21.15.92:0/651263470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 msgr2=0x557f17800760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17800760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fbfff640 1 -- 172.21.15.92:0/651263470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 msgr2=0x557f17804a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17804a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fbfff640 1 -- 172.21.15.92:0/651263470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f178111f0 con 0x557f174331b0 2024-09-21T02:08:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17804a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17814bd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0107e0 tx=0x7fc3ec010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4aa2ec3cf59365 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3e1ffb640 1 -- 172.21.15.92:0/651263470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec011440 con 0x557f174331b0 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3e1ffb640 1 -- 172.21.15.92:0/651263470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0115e0 con 0x557f174331b0 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.900+0000 7fc3e1ffb640 1 -- 172.21.15.92:0/651263470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec004b60 con 0x557f174331b0 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 -- 172.21.15.92:0/651263470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 msgr2=0x557f17814bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17814bd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0107e0 tx=0x7fc3ec010ca0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 -- 172.21.15.92:0/651263470 shutdown_connections 2024-09-21T02:08:06.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1745c070 0x557f17800760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1743b390 0x557f17804a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 --2- 172.21.15.92:0/651263470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f17814bd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 -- 172.21.15.92:0/651263470 >> 172.21.15.92:0/651263470 conn(0x557f17450460 msgr2=0x557f178042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 -- 172.21.15.92:0/651263470 shutdown_connections 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.901+0000 7fc408878740 1 -- 172.21.15.92:0/651263470 wait complete. 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 -- start start 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 Processor -- start 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f1781b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1743b390 0x557f17816b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1745c070 0x557f17817050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1ae6d0 con 0x557f174331b0 2024-09-21T02:08:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1ae000 con 0x557f1745c070 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc1ab240 con 0x557f1743b390 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc408878740 10 client.0 _ll_register_callbacks cb 0x7fc3f4009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f1781b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f1781b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41560/0 (socket says 172.21.15.92:41560) 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc3fbfff640 1 -- 172.21.15.92:0/734525033 learned_addr learned my addr 172.21.15.92:0/734525033 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1743b390 0x557f17816b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.902+0000 7fc4008a1640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1745c070 0x557f17817050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3fbfff640 1 -- 172.21.15.92:0/734525033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1743b390 msgr2=0x557f17816b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1743b390 0x557f17816b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3fbfff640 1 -- 172.21.15.92:0/734525033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1745c070 msgr2=0x557f17817050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1745c070 0x557f17817050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3fbfff640 1 -- 172.21.15.92:0/734525033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f17817610 con 0x557f174331b0 2024-09-21T02:08:06.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3fbfff640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f1781b660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec004eb0 tx=0x7fc3ec004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9e3c77a8af5322 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3db7fe640 1 -- 172.21.15.92:0/734525033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec011440 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3ec0115e0 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3db7fe640 1 -- 172.21.15.92:0/734525033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc3ec003d50 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc3db7fe640 1 -- 172.21.15.92:0/734525033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3ec048050 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc408878740 10 client.16431 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.903+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f1788f420 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc3db7fe640 1 -- 172.21.15.92:0/734525033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0115e0 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc3db7fe640 1 -- 172.21.15.92:0/734525033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc3ec011790 con 0x557f174331b0 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc408878740 10 client.16431 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc408878740 10 client.16431 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc408878740 20 client.16431 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc408878740 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f173efda0 0x557f17891af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:06.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc408878740 4 client.16431 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f1780e8a0 con 0x557f173efda0 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f173efda0 0x557f17891af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc3db7fe640 10 client.16431 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.904+0000 7fc3fb7fe640 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f173efda0 0x557f17891af0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3908de7d066b04ef server_cookie=fb7cb55382d9f8bc in_seq=0 out_seq=0 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc3db7fe640 1 -- 172.21.15.92:0/734525033 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f1780e8a0 con 0x557f173efda0 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc3db7fe640 10 client.16431 handle_command_reply: tid=0 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 client.16431 shutdown 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 msgr2=0x557f1781b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f1781b660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec004eb0 tx=0x7fc3ec004b90 comp rx=0 tx=0).stop 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 shutdown_connections 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x557f173efda0 0x557f17891af0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc3db7fe640 0 client.16431 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:06.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f1745c070 0x557f17817050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f1743b390 0x557f17816b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 --2- 172.21.15.92:0/734525033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f174331b0 0x557f1781b660 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 >> 172.21.15.92:0/734525033 conn(0x557f17450460 msgr2=0x557f17804a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 shutdown_connections 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 1 -- 172.21.15.92:0/734525033 wait complete. 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.905+0000 7fc408878740 20 client.16431 trim_cache size 0 max 16384 2024-09-21T02:08:06.911 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3e40b6510 msgr2=0x7fc3e40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3e40b6510 0x7fc3e40b8a20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc3f402f500 tx=0x7fc3f4009820 comp rx=0 tx=0).stop 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 msgr2=0x7fc3fc1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1998f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009830 tx=0x7fc3f0009cf0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 shutdown_connections 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3e40b6510 0x7fc3e40b8a20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3fc108cf0 0x7fc3fc19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3fc1083a0 0x7fc3fc1998f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 --2- 172.21.15.92:0/1250406670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3fc102a60 0x7fc3fc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:06.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 >> 172.21.15.92:0/1250406670 conn(0x7fc3fc079730 msgr2=0x7fc3fc1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 shutdown_connections 2024-09-21T02:08:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:06.906+0000 7fc402304640 1 -- 172.21.15.92:0/1250406670 wait complete. 2024-09-21T02:08:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:06 smithi092 ceph-mon[31190]: pgmap v454: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:06 smithi136 ceph-mon[34754]: pgmap v454: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:07.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:06 smithi099 ceph-mon[34404]: pgmap v454: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:07.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:08:08.062 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:08.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 -- 172.21.15.92:0/54469692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0105290 msgr2=0x7f18b010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 --2- 172.21.15.92:0/54469692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0105290 0x7f18b010d8d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009ca0 tx=0x7f18a402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 -- 172.21.15.92:0/54469692 shutdown_connections 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 --2- 172.21.15.92:0/54469692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010de10 0x7f18b01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 --2- 172.21.15.92:0/54469692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0105290 0x7f18b010d8d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 --2- 172.21.15.92:0/54469692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0104860 0x7f18b0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.533+0000 7f18b5e90640 1 -- 172.21.15.92:0/54469692 >> 172.21.15.92:0/54469692 conn(0x7f18b0079730 msgr2=0x7f18b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.534+0000 7f18b5e90640 1 -- 172.21.15.92:0/54469692 shutdown_connections 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.534+0000 7f18b5e90640 1 -- 172.21.15.92:0/54469692 wait complete. 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.534+0000 7f18b5e90640 1 Processor -- start 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.534+0000 7f18b5e90640 1 -- start start 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18b5e90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 0x7f18b019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18b5e90640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0105290 0x7f18b019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18b5e90640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010de10 0x7f18b019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18b5e90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0113a60 con 0x7f18b0104860 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18b5e90640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01138e0 con 0x7f18b0105290 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18b5e90640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0113be0 con 0x7f18b010de10 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 0x7f18b019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 0x7f18b019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41564/0 (socket says 172.21.15.92:41564) 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 -- 172.21.15.92:0/3992948 learned_addr learned my addr 172.21.15.92:0/3992948 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18aeffd640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0105290 0x7f18b019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18affff640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010de10 0x7f18b019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 -- 172.21.15.92:0/3992948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010de10 msgr2=0x7f18b019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010de10 0x7f18b019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 -- 172.21.15.92:0/3992948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0105290 msgr2=0x7f18b019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0105290 0x7f18b019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 -- 172.21.15.92:0/3992948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b01a04a0 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18aeffd640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0105290 0x7f18b019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18af7fe640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 0x7f18b019a530 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f18a000d950 tx=0x7f18a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.535+0000 7f18acff9640 1 -- 172.21.15.92:0/3992948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0017070 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.536+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b01a0790 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.536+0000 7f18acff9640 1 -- 172.21.15.92:0/3992948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0009c30 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.536+0000 7f18acff9640 1 -- 172.21.15.92:0/3992948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a000f620 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.536+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a6ef0 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 Processor -- start 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18acff9640 1 -- 172.21.15.92:0/3992948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f18a0010460 con 0x7f18b0104860 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 -- start start 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 0x558951cfd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951cfd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951cfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558951c73ea0 con 0x558951956070 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558951c632d0 con 0x55895192d1b0 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.538+0000 7f18bc404740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558951c670e0 con 0x558951935390 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 0x558951cfd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 0x558951cfd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41578/0 (socket says 172.21.15.92:41578) 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 -- 172.21.15.92:0/429573998 learned_addr learned my addr 172.21.15.92:0/429573998 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18aeffd640 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951cfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18af7fe640 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951cfd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 -- 172.21.15.92:0/429573998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 msgr2=0x558951cfdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951cfdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 -- 172.21.15.92:0/429573998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 msgr2=0x558951cfd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951cfd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 -- 172.21.15.92:0/429573998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589519c1650 con 0x558951956070 2024-09-21T02:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18af7fe640 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951cfd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18acff9640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18980b6510 0x7f18980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18affff640 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 0x558951cfd2c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009a40 tx=0x7f18a8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350ab90bd1cb5b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18af7fe640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18980b6510 0x7f18980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f1895ffb640 1 -- 172.21.15.92:0/429573998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8016070 con 0x558951956070 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f18acff9640 1 -- 172.21.15.92:0/3992948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f18a00e7010 con 0x7f18b0104860 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f1895ffb640 1 -- 172.21.15.92:0/429573998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a80054d0 con 0x558951956070 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.539+0000 7f1895ffb640 1 -- 172.21.15.92:0/429573998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004ce0 con 0x558951956070 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 -- 172.21.15.92:0/429573998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 msgr2=0x558951cfd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18af7fe640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18980b6510 0x7f18980b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x558951cfbf40 tx=0x7f18a0094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 0x558951cfd2c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009a40 tx=0x7f18a8009f00 comp rx=0 tx=0).stop 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 -- 172.21.15.92:0/429573998 shutdown_connections 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951cfdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951cfd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 --2- 172.21.15.92:0/429573998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951956070 0x558951cfd2c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 -- 172.21.15.92:0/429573998 >> 172.21.15.92:0/429573998 conn(0x55895194a460 msgr2=0x558951cfb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 -- 172.21.15.92:0/429573998 shutdown_connections 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.540+0000 7f18bc404740 1 -- 172.21.15.92:0/429573998 wait complete. 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 -- start start 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 Processor -- start 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d08950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951d08e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951956070 0x558951d09410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558951ac4700 con 0x55895192d1b0 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558951bfe160 con 0x558951956070 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558951d0baa0 con 0x558951935390 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.541+0000 7f18bc404740 10 client.0 _ll_register_callbacks cb 0x558951c179e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d08950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d08950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41592/0 (socket says 172.21.15.92:41592) 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 -- 172.21.15.92:0/1386830915 learned_addr learned my addr 172.21.15.92:0/1386830915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18aeffd640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951d08e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951956070 0x558951d09410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 -- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 msgr2=0x558951d08e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951d08e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 -- 172.21.15.92:0/1386830915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951956070 msgr2=0x558951d09410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951956070 0x558951d09410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 -- 172.21.15.92:0/1386830915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558951c2bc90 con 0x55895192d1b0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18aeffd640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951d08e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18affff640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d08950 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005940 tx=0x7f18a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe0d3c9c35eff56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f188effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f188effd640 1 -- 172.21.15.92:0/1386830915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8016040 con 0x55895192d1b0 2024-09-21T02:08:08.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f188effd640 1 -- 172.21.15.92:0/1386830915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f18a8004ef0 con 0x55895192d1b0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a80054d0 con 0x55895192d1b0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f188effd640 1 -- 172.21.15.92:0/1386830915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18a802f070 con 0x55895192d1b0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18bc404740 10 client.16443 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.542+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558951d17770 con 0x55895192d1b0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f188effd640 1 -- 172.21.15.92:0/1386830915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a80054d0 con 0x55895192d1b0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f188effd640 1 -- 172.21.15.92:0/1386830915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f18a8027020 con 0x55895192d1b0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18bc404740 10 client.16443 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18bc404740 10 client.16443 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18bc404740 20 client.16443 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18bc404740 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589519149f0 0x558951d1ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18bc404740 4 client.16443 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558951c75df0 con 0x5589519149f0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.543+0000 7f18aeffd640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589519149f0 0x558951d1ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.544+0000 7f18aeffd640 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589519149f0 0x558951d1ab80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7c35fbab87978ce server_cookie=d77d891c8776a544 in_seq=0 out_seq=0 2024-09-21T02:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.544+0000 7f188effd640 10 client.16443 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.544+0000 7f188effd640 1 -- 172.21.15.92:0/1386830915 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558951c75df0 con 0x5589519149f0 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.544+0000 7f188effd640 10 client.16443 handle_command_reply: tid=0 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 client.16443 shutdown 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 msgr2=0x558951d08950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d08950 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005940 tx=0x7f18a80046c0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 shutdown_connections 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5589519149f0 0x558951d1ab80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951956070 0x558951d09410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558951935390 0x558951d08e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f188effd640 0 client.16443 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 --2- 172.21.15.92:0/1386830915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d08950 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 >> 172.21.15.92:0/1386830915 conn(0x55895194a460 msgr2=0x558951d004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 shutdown_connections 2024-09-21T02:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 1 -- 172.21.15.92:0/1386830915 wait complete. 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.545+0000 7f18bc404740 20 client.16443 trim_cache size 0 max 16384 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 Processor -- start 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 -- start start 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d08e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951d05b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 0x558951d06080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01af5e0 con 0x558951935390 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b1120 con 0x55895192d1b0 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18bc404740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01afcb0 con 0x5589519149f0 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 0x558951d06080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18affff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 0x558951d06080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41606/0 (socket says 172.21.15.92:41606) 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18affff640 1 -- 172.21.15.92:0/3967255065 learned_addr learned my addr 172.21.15.92:0/3967255065 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18aeffd640 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951d05b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.558+0000 7f18affff640 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d08e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.559+0000 7f18affff640 1 -- 172.21.15.92:0/3967255065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 msgr2=0x558951d08e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.559+0000 7f18affff640 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d08e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.559+0000 7f18affff640 1 -- 172.21.15.92:0/3967255065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 msgr2=0x558951d05b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.559+0000 7f18affff640 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951d05b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.559+0000 7f18affff640 1 -- 172.21.15.92:0/3967255065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558951d09c10 con 0x558951935390 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.559+0000 7f18affff640 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 0x558951d06080 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098f0 tx=0x7f18a8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c7801bb63b1f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.560+0000 7f18957fa640 1 -- 172.21.15.92:0/3967255065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8016070 con 0x558951935390 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.560+0000 7f18957fa640 1 -- 172.21.15.92:0/3967255065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a801e070 con 0x558951935390 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.560+0000 7f18957fa640 1 -- 172.21.15.92:0/3967255065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004ed0 con 0x558951935390 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 -- 172.21.15.92:0/3967255065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 msgr2=0x558951d06080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 0x558951d06080 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098f0 tx=0x7f18a8009f00 comp rx=0 tx=0).stop 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 -- 172.21.15.92:0/3967255065 shutdown_connections 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558951935390 0x558951d06080 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55895192d1b0 0x558951d05b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 --2- 172.21.15.92:0/3967255065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d08e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 -- 172.21.15.92:0/3967255065 >> 172.21.15.92:0/3967255065 conn(0x55895194a460 msgr2=0x558951d08700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 -- 172.21.15.92:0/3967255065 shutdown_connections 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 -- 172.21.15.92:0/3967255065 wait complete. 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 -- start start 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.561+0000 7f18bc404740 1 Processor -- start 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d1c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951935390 0x558951d84300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b0a50 con 0x55895192d1b0 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b0380 con 0x558951935390 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01ad5c0 con 0x5589519149f0 2024-09-21T02:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18bc404740 10 client.0 _ll_register_callbacks cb 0x7f18a4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d1c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d1c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41614/0 (socket says 172.21.15.92:41614) 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 -- 172.21.15.92:0/612296249 learned_addr learned my addr 172.21.15.92:0/612296249 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18affff640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18affff640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951935390 0x558951d84300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 -- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 msgr2=0x558951d1bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d1bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 -- 172.21.15.92:0/612296249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951935390 msgr2=0x558951d84300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951935390 0x558951d84300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 -- 172.21.15.92:0/612296249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558951d1cd00 con 0x55895192d1b0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f18aeffd640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d1c430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009a60 tx=0x7f18a4031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520f2f577db3962 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.562+0000 7f188effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f188effd640 1 -- 172.21.15.92:0/612296249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4003b70 con 0x55895192d1b0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f188effd640 1 -- 172.21.15.92:0/612296249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f18a4003d10 con 0x55895192d1b0 2024-09-21T02:08:08.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a801e070 con 0x55895192d1b0 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f188effd640 1 -- 172.21.15.92:0/612296249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18a4004040 con 0x55895192d1b0 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 10 client.16449 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558951d858c0 con 0x55895192d1b0 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f188effd640 1 -- 172.21.15.92:0/612296249 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a801e070 con 0x55895192d1b0 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f188effd640 1 -- 172.21.15.92:0/612296249 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f18a4045040 con 0x55895192d1b0 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 10 client.16449 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 10 client.16449 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 20 client.16449 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x558951956070 0x558951d880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 4 client.16449 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.563+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558951c718a0 con 0x558951956070 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f18affff640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x558951956070 0x558951d880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f188effd640 10 client.16449 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:08.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f18affff640 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x558951956070 0x558951d880b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67e71eeccda3a8af server_cookie=bb519eb269d0046e in_seq=0 out_seq=0 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f188effd640 1 -- 172.21.15.92:0/612296249 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558951c718a0 con 0x558951956070 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f188effd640 10 client.16449 handle_command_reply: tid=0 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f18bc404740 1 client.16449 shutdown 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 msgr2=0x558951d1c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.564+0000 7f18bc404740 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d1c430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009a60 tx=0x7f18a4031600 comp rx=0 tx=0).stop 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 shutdown_connections 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x558951956070 0x558951d880b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558951935390 0x558951d84300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55895192d1b0 0x558951d1c430 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f188effd640 0 client.16449 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 --2- 172.21.15.92:0/612296249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589519149f0 0x558951d1bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 >> 172.21.15.92:0/612296249 conn(0x55895194a460 msgr2=0x558951d003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 shutdown_connections 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 1 -- 172.21.15.92:0/612296249 wait complete. 2024-09-21T02:08:08.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18bc404740 20 client.16449 trim_cache size 0 max 16384 2024-09-21T02:08:08.581 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18980b6510 msgr2=0x7f18980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18980b6510 0x7f18980b8a20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x558951cfbf40 tx=0x7f18a0094040 comp rx=0 tx=0).stop 2024-09-21T02:08:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 msgr2=0x7f18b019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 0x7f18b019a530 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f18a000d950 tx=0x7f18a000de10 comp rx=0 tx=0).stop 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 shutdown_connections 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18980b6510 0x7f18980b8a20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010de10 0x7f18b019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b0105290 0x7f18b019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 --2- 172.21.15.92:0/3992948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18b0104860 0x7f18b019a530 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.565+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 >> 172.21.15.92:0/3992948 conn(0x7f18b0079730 msgr2=0x7f18b0100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.566+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 shutdown_connections 2024-09-21T02:08:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:08.566+0000 7f18b5e90640 1 -- 172.21.15.92:0/3992948 wait complete. 2024-09-21T02:08:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:08 smithi136 ceph-mon[34754]: pgmap v455: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:08 smithi092 ceph-mon[31190]: pgmap v455: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:08 smithi099 ceph-mon[34404]: pgmap v455: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:09.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:08:09.643 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 -- 172.21.15.92:0/3969780770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab41094f0 msgr2=0x7f1ab4109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 --2- 172.21.15.92:0/3969780770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab41094f0 0x7f1ab4109900 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009900 tx=0x7f1aa402a410 comp rx=0 tx=0).stop 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 -- 172.21.15.92:0/3969780770 shutdown_connections 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 --2- 172.21.15.92:0/3969780770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ab4109e40 0x7f1ab4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 --2- 172.21.15.92:0/3969780770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab41094f0 0x7f1ab4109900 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 --2- 172.21.15.92:0/3969780770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab4103bb0 0x7f1ab4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 -- 172.21.15.92:0/3969780770 >> 172.21.15.92:0/3969780770 conn(0x7f1ab40ff600 msgr2=0x7f1ab4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 -- 172.21.15.92:0/3969780770 shutdown_connections 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.135+0000 7f1abae9e640 1 -- 172.21.15.92:0/3969780770 wait complete. 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 Processor -- start 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 -- start start 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ab4103bb0 0x7f1ab419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab41094f0 0x7f1ab419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 0x7f1ab419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4113b00 con 0x7f1ab4109e40 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4113980 con 0x7f1ab41094f0 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1abae9e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4113c80 con 0x7f1ab4103bb0 2024-09-21T02:08:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1ab943b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 0x7f1ab419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1ab943b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 0x7f1ab419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41640/0 (socket says 172.21.15.92:41640) 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1ab943b640 1 -- 172.21.15.92:0/1009875324 learned_addr learned my addr 172.21.15.92:0/1009875324 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab41094f0 0x7f1ab419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.136+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ab4103bb0 0x7f1ab419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab943b640 1 -- 172.21.15.92:0/1009875324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ab4103bb0 msgr2=0x7f1ab419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab943b640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ab4103bb0 0x7f1ab419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab943b640 1 -- 172.21.15.92:0/1009875324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab41094f0 msgr2=0x7f1ab419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab943b640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab41094f0 0x7f1ab419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab943b640 1 -- 172.21.15.92:0/1009875324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab41a0380 con 0x7f1ab4109e40 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab943b640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 0x7f1ab419fd10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00cce0 tx=0x7f1aac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab1ffb640 1 -- 172.21.15.92:0/1009875324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00f450 con 0x7f1ab4109e40 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab1ffb640 1 -- 172.21.15.92:0/1009875324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac00faa0 con 0x7f1ab4109e40 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab41a0670 con 0x7f1ab4109e40 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1ab1ffb640 1 -- 172.21.15.92:0/1009875324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac018b70 con 0x7f1ab4109e40 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.137+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41a0b30 con 0x7f1ab4109e40 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.140+0000 7f1ac1412740 1 Processor -- start 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.140+0000 7f1ab1ffb640 1 -- 172.21.15.92:0/1009875324 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1aac018d10 con 0x7f1ab4109e40 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.140+0000 7f1ac1412740 1 -- start start 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce022070 0x55c4ce3c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 0x55c4ce3ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ac1412740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c4ce33fea0 con 0x55c4ce001390 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ac1412740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c4ce32f2d0 con 0x55c4cdff91b0 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ac1412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c4ce3330e0 con 0x55c4ce022070 2024-09-21T02:08:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 0x55c4ce3ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 0x55c4ce3ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41642/0 (socket says 172.21.15.92:41642) 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 -- 172.21.15.92:0/3776371386 learned_addr learned my addr 172.21.15.92:0/3776371386 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab1ffb640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1aa00b6540 0x7f1aa00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce022070 0x55c4ce3c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 -- 172.21.15.92:0/3776371386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce022070 msgr2=0x55c4ce3c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce022070 0x55c4ce3c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 -- 172.21.15.92:0/3776371386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 msgr2=0x55c4ce3c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab1ffb640 1 -- 172.21.15.92:0/1009875324 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1aac0e7820 con 0x7f1ab4109e40 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab943b640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1aa00b6540 0x7f1aa00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.141+0000 7f1ab3fff640 1 -- 172.21.15.92:0/3776371386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ce08d650 con 0x55c4ce001390 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.142+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 0x55c4ce3ca120 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406cbc0 tx=0x7f1aa4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aef92aa4156eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.142+0000 7f1aaaffd640 1 -- 172.21.15.92:0/3776371386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4035040 con 0x55c4ce001390 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.142+0000 7f1ab943b640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1aa00b6540 0x7f1aa00b8a50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0ad920 tx=0x7f1aac00ff50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.142+0000 7f1aaaffd640 1 -- 172.21.15.92:0/3776371386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa40316f0 con 0x55c4ce001390 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.142+0000 7f1aaaffd640 1 -- 172.21.15.92:0/3776371386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40047a0 con 0x55c4ce001390 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 -- 172.21.15.92:0/3776371386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 msgr2=0x55c4ce3ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 0x55c4ce3ca120 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406cbc0 tx=0x7f1aa4009680 comp rx=0 tx=0).stop 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 -- 172.21.15.92:0/3776371386 shutdown_connections 2024-09-21T02:08:10.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce001390 0x55c4ce3ca120 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3776371386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce022070 0x55c4ce3c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 -- 172.21.15.92:0/3776371386 >> 172.21.15.92:0/3776371386 conn(0x55c4ce016460 msgr2=0x55c4ce3c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 -- 172.21.15.92:0/3776371386 shutdown_connections 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 -- 172.21.15.92:0/3776371386 wait complete. 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 -- start start 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.143+0000 7f1ac1412740 1 Processor -- start 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce001390 0x55c4ce3d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 0x55c4ce3e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c4ce190700 con 0x55c4ce022070 2024-09-21T02:08:10.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c4ce2ca160 con 0x55c4cdff91b0 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c4ce3d7e20 con 0x55c4ce001390 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 10 client.0 _ll_register_callbacks cb 0x55c4ce2e39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 0x55c4ce3e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 0x55c4ce3e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41654/0 (socket says 172.21.15.92:41654) 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 -- 172.21.15.92:0/2771501527 learned_addr learned my addr 172.21.15.92:0/2771501527 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce001390 0x55c4ce3d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 -- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce001390 msgr2=0x55c4ce3d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce001390 0x55c4ce3d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 -- 172.21.15.92:0/2771501527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 msgr2=0x55c4ce3d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 -- 172.21.15.92:0/2771501527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ce2f7c90 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 0x55c4ce3e1140 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55c4ce3cbfc0 tx=0x7f1aa40053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37e658db0841c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1a93fff640 1 -- 172.21.15.92:0/2771501527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4035070 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1a93fff640 1 -- 172.21.15.92:0/2771501527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1aa4003800 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.144+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa40316f0 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1a93fff640 1 -- 172.21.15.92:0/2771501527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1aa405c050 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 10 client.16461 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4ce3e3b20 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1a93fff640 1 -- 172.21.15.92:0/2771501527 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa40316f0 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1a93fff640 1 -- 172.21.15.92:0/2771501527 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1aa403a040 con 0x55c4ce022070 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 10 client.16461 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:10.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 10 client.16461 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 20 client.16461 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4cdfe09f0 0x55c4ce3e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 4 client.16461 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4ce341df0 con 0x55c4cdfe09f0 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.145+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4cdfe09f0 0x55c4ce3e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.146+0000 7f1a93fff640 10 client.16461 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.146+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4cdfe09f0 0x55c4ce3e6e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5687184e247e4fe6 server_cookie=fd9c1c23318a230a in_seq=0 out_seq=0 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1a93fff640 1 -- 172.21.15.92:0/2771501527 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c4ce341df0 con 0x55c4cdfe09f0 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1a93fff640 10 client.16461 handle_command_reply: tid=0 2024-09-21T02:08:10.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 client.16461 shutdown 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 msgr2=0x55c4ce3e1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 0x55c4ce3e1140 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55c4ce3cbfc0 tx=0x7f1aa40053b0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 shutdown_connections 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4cdfe09f0 0x55c4ce3e6e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1a93fff640 0 client.16461 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4ce022070 0x55c4ce3e1140 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4ce001390 0x55c4ce3d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2771501527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4cdff91b0 0x55c4ce3d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 >> 172.21.15.92:0/2771501527 conn(0x55c4ce016460 msgr2=0x55c4ce3cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 shutdown_connections 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 1 -- 172.21.15.92:0/2771501527 wait complete. 2024-09-21T02:08:10.162 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.147+0000 7f1ac1412740 20 client.16461 trim_cache size 0 max 16384 2024-09-21T02:08:10.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.160+0000 7f1ac1412740 1 Processor -- start 2024-09-21T02:08:10.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.160+0000 7f1ac1412740 1 -- start start 2024-09-21T02:08:10.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.160+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.160+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.160+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.160+0000 7f1ac1412740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41af330 con 0x55c4cdff91b0 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ac1412740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41b0e70 con 0x55c4ce001390 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ac1412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41afa00 con 0x55c4cdfe09f0 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41666/0 (socket says 172.21.15.92:41666) 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/3783116581 learned_addr learned my addr 172.21.15.92:0/3783116581 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/3783116581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 msgr2=0x55c4ce3cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/3783116581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 msgr2=0x55c4ce3c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/3783116581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ce3cb990 con 0x55c4cdff91b0 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c6bc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00cfe0 tx=0x7f1a9c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49114e0979f0bccb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1aaa7fc640 1 -- 172.21.15.92:0/3783116581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c01c620 con 0x55c4cdff91b0 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1aaa7fc640 1 -- 172.21.15.92:0/3783116581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c01cc70 con 0x55c4cdff91b0 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.161+0000 7f1aaa7fc640 1 -- 172.21.15.92:0/3783116581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c022d50 con 0x55c4cdff91b0 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 -- 172.21.15.92:0/3783116581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 msgr2=0x55c4ce3c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c6bc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00cfe0 tx=0x7f1a9c00c430 comp rx=0 tx=0).stop 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 -- 172.21.15.92:0/3783116581 shutdown_connections 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3c6bc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 --2- 172.21.15.92:0/3783116581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 -- 172.21.15.92:0/3783116581 >> 172.21.15.92:0/3783116581 conn(0x55c4ce016460 msgr2=0x55c4ce3ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 -- 172.21.15.92:0/3783116581 shutdown_connections 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.162+0000 7f1ac1412740 1 -- 172.21.15.92:0/3783116581 wait complete. 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 -- start start 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 Processor -- start 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41b07a0 con 0x55c4cdff91b0 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41b00d0 con 0x55c4ce001390 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41ad310 con 0x55c4cdfe09f0 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ac1412740 10 client.0 _ll_register_callbacks cb 0x7f1a9c00c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ab8c3a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ab8c3a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41680/0 (socket says 172.21.15.92:41680) 2024-09-21T02:08:10.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.163+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/2296173694 learned_addr learned my addr 172.21.15.92:0/2296173694 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 msgr2=0x55c4ce3ea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3ea640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/2296173694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 msgr2=0x55c4ce3e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab8c3a640 1 -- 172.21.15.92:0/2296173694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ce3e87e0 con 0x55c4cdff91b0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ab8c3a640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3e7b60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c024040 tx=0x7f1a9c022620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6b6b344ee98b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1a93fff640 1 -- 172.21.15.92:0/2296173694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c032020 con 0x55c4cdff91b0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1a93fff640 1 -- 172.21.15.92:0/2296173694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1a9c01c420 con 0x55c4cdff91b0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a9c01cc70 con 0x55c4cdff91b0 2024-09-21T02:08:10.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1a93fff640 1 -- 172.21.15.92:0/2296173694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c03d050 con 0x55c4cdff91b0 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ac1412740 10 client.16467 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.164+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4ce451460 con 0x55c4cdff91b0 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1a93fff640 1 -- 172.21.15.92:0/2296173694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c01cc70 con 0x55c4cdff91b0 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1a93fff640 1 -- 172.21.15.92:0/2296173694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a9c02e820 con 0x55c4cdff91b0 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ac1412740 10 client.16467 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ac1412740 10 client.16467 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ac1412740 20 client.16467 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4ce022070 0x55c4ce453c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ac1412740 4 client.16467 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4ce33d8a0 con 0x55c4ce022070 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.165+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4ce022070 0x55c4ce453c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ab3fff640 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4ce022070 0x55c4ce453c00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af7b39b8482ada06 server_cookie=9d2f6b8433bc4e94 in_seq=0 out_seq=0 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1a93fff640 10 client.16467 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:10.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1a93fff640 1 -- 172.21.15.92:0/2296173694 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4ce33d8a0 con 0x55c4ce022070 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1a93fff640 10 client.16467 handle_command_reply: tid=0 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 client.16467 shutdown 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 msgr2=0x55c4ce3e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3e7b60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c024040 tx=0x7f1a9c022620 comp rx=0 tx=0).stop 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 shutdown_connections 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55c4ce022070 0x55c4ce453c00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4ce001390 0x55c4ce3e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4cdff91b0 0x55c4ce3e7b60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 --2- 172.21.15.92:0/2296173694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c4cdfe09f0 0x55c4ce3ea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 >> 172.21.15.92:0/2296173694 conn(0x55c4ce016460 msgr2=0x55c4ce3ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1a93fff640 0 client.16467 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:10.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 shutdown_connections 2024-09-21T02:08:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 1 -- 172.21.15.92:0/2296173694 wait complete. 2024-09-21T02:08:10.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.166+0000 7f1ac1412740 20 client.16467 trim_cache size 0 max 16384 2024-09-21T02:08:10.174 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1aa00b6540 msgr2=0x7f1aa00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1aa00b6540 0x7f1aa00b8a50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0ad920 tx=0x7f1aac00ff50 comp rx=0 tx=0).stop 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 msgr2=0x7f1ab419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 0x7f1ab419fd10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00cce0 tx=0x7f1aac00a620 comp rx=0 tx=0).stop 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 shutdown_connections 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1aa00b6540 0x7f1aa00b8a50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ab4109e40 0x7f1ab419fd10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab41094f0 0x7f1ab419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 --2- 172.21.15.92:0/1009875324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ab4103bb0 0x7f1ab419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:10.175 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 >> 172.21.15.92:0/1009875324 conn(0x7f1ab40ff600 msgr2=0x7f1ab4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 shutdown_connections 2024-09-21T02:08:10.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:10.167+0000 7f1abae9e640 1 -- 172.21.15.92:0/1009875324 wait complete. 2024-09-21T02:08:11.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:08:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:10 smithi136 ceph-mon[34754]: pgmap v456: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:11.294 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:10 smithi092 ceph-mon[31190]: pgmap v456: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:11.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:10 smithi099 ceph-mon[34404]: pgmap v456: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 -- 172.21.15.92:0/2033038384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c1052d0 msgr2=0x7fc68c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 --2- 172.21.15.92:0/2033038384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c1052d0 0x7fc68c10d910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc680009900 tx=0x7fc68002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 -- 172.21.15.92:0/2033038384 shutdown_connections 2024-09-21T02:08:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 --2- 172.21.15.92:0/2033038384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc68c10de50 0x7fc68c1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 --2- 172.21.15.92:0/2033038384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c1052d0 0x7fc68c10d910 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 --2- 172.21.15.92:0/2033038384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1048a0 0x7fc68c104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 -- 172.21.15.92:0/2033038384 >> 172.21.15.92:0/2033038384 conn(0x7fc68c079730 msgr2=0x7fc68c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 -- 172.21.15.92:0/2033038384 shutdown_connections 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.735+0000 7fc6923cd640 1 -- 172.21.15.92:0/2033038384 wait complete. 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 Processor -- start 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 -- start start 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1048a0 0x7fc68c1027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc68c1052d0 0x7fc68c102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 0x7fc68c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c113a60 con 0x7fc68c10de50 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1138e0 con 0x7fc68c1048a0 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc6923cd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c113be0 con 0x7fc68c1052d0 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc69096a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 0x7fc68c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc69096a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 0x7fc68c103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41702/0 (socket says 172.21.15.92:41702) 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.736+0000 7fc69096a640 1 -- 172.21.15.92:0/3580306636 learned_addr learned my addr 172.21.15.92:0/3580306636 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc68bfff640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1048a0 0x7fc68c1027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc68c1052d0 0x7fc68c102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc69096a640 1 -- 172.21.15.92:0/3580306636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc68c1052d0 msgr2=0x7fc68c102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc69096a640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc68c1052d0 0x7fc68c102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc69096a640 1 -- 172.21.15.92:0/3580306636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1048a0 msgr2=0x7fc68c1027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc69096a640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1048a0 0x7fc68c1027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc69096a640 1 -- 172.21.15.92:0/3580306636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1a8680 con 0x7fc68c10de50 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc69096a640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 0x7fc68c103230 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc68400eaa0 tx=0x7fc68400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc6897fa640 1 -- 172.21.15.92:0/3580306636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68400cdb0 con 0x7fc68c10de50 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc6897fa640 1 -- 172.21.15.92:0/3580306636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc684005020 con 0x7fc68c10de50 2024-09-21T02:08:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1a8970 con 0x7fc68c10de50 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc6897fa640 1 -- 172.21.15.92:0/3580306636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840116d0 con 0x7fc68c10de50 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.737+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a8e80 con 0x7fc68c10de50 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.740+0000 7fc698941740 1 Processor -- start 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.740+0000 7fc6897fa640 1 -- 172.21.15.92:0/3580306636 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc6840051d0 con 0x7fc68c10de50 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 -- start start 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860901f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860901f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x55860901fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558608f95ea0 con 0x558608c57390 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558608f852d0 con 0x558608c78070 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc698941740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558608f890e0 con 0x558608c4f1b0 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x55860901fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x55860901fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41704/0 (socket says 172.21.15.92:41704) 2024-09-21T02:08:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 -- 172.21.15.92:0/2199477411 learned_addr learned my addr 172.21.15.92:0/2199477411 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68bfff640 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860901f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860901f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc6897fa640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6740b6510 0x7fc6740b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 -- 172.21.15.92:0/2199477411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 msgr2=0x55860901f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860901f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 -- 172.21.15.92:0/2199477411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 msgr2=0x55860901f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860901f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.741+0000 7fc68b7fe640 1 -- 172.21.15.92:0/2199477411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558608ce3650 con 0x558608c57390 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.742+0000 7fc69096a640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6740b6510 0x7fc6740b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.742+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x55860901fd40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc680005940 tx=0x7fc6800058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d933f399deafa7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.742+0000 7fc6897fa640 1 -- 172.21.15.92:0/3580306636 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc6840e7530 con 0x7fc68c10de50 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.742+0000 7fc6727fc640 1 -- 172.21.15.92:0/2199477411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680035040 con 0x558608c57390 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.742+0000 7fc6727fc640 1 -- 172.21.15.92:0/2199477411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6800035f0 con 0x558608c57390 2024-09-21T02:08:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.742+0000 7fc6727fc640 1 -- 172.21.15.92:0/2199477411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6800044e0 con 0x558608c57390 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 -- 172.21.15.92:0/2199477411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 msgr2=0x55860901fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x55860901fd40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc680005940 tx=0x7fc6800058b0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc69096a640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6740b6510 0x7fc6740b8a20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc6840e5380 tx=0x7fc68407a9d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 -- 172.21.15.92:0/2199477411 shutdown_connections 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x55860901fd40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860901f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 --2- 172.21.15.92:0/2199477411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860901f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 -- 172.21.15.92:0/2199477411 >> 172.21.15.92:0/2199477411 conn(0x558608c6c460 msgr2=0x55860901d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 -- 172.21.15.92:0/2199477411 shutdown_connections 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 -- 172.21.15.92:0/2199477411 wait complete. 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.743+0000 7fc698941740 1 -- start start 2024-09-21T02:08:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 Processor -- start 2024-09-21T02:08:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c57390 0x55860902ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 0x558609036d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558608de6700 con 0x558608c78070 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558608f20160 con 0x558608c57390 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55860902da40 con 0x558608c4f1b0 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc698941740 10 client.0 _ll_register_callbacks cb 0x558608f399e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 0x558609036d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.744+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 0x558609036d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41706/0 (socket says 172.21.15.92:41706) 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 -- 172.21.15.92:0/522481273 learned_addr learned my addr 172.21.15.92:0/522481273 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68bfff640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c57390 0x55860902ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 -- 172.21.15.92:0/522481273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 msgr2=0x55860902a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 -- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c57390 msgr2=0x55860902ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c57390 0x55860902ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 -- 172.21.15.92:0/522481273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558608f4dc90 con 0x558608c78070 2024-09-21T02:08:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 0x558609036d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc680006010 tx=0x7fc68002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1754982f0885e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc66b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc66b7fe640 1 -- 172.21.15.92:0/522481273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680035070 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc66b7fe640 1 -- 172.21.15.92:0/522481273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc6800037a0 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6800035f0 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.745+0000 7fc66b7fe640 1 -- 172.21.15.92:0/522481273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6800039a0 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 10 client.16479 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558609039740 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc66b7fe640 1 -- 172.21.15.92:0/522481273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc680003b40 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc66b7fe640 1 -- 172.21.15.92:0/522481273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc68003a040 con 0x558608c78070 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 10 client.16479 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 10 client.16479 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 20 client.16479 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c369f0 0x55860903caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 4 client.16479 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.746+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc6800035f0 con 0x558608c369f0 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.747+0000 7fc68bfff640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c369f0 0x55860903caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.747+0000 7fc68bfff640 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c369f0 0x55860903caa0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2bbf95adaf3706d server_cookie=b74f890fb02397d9 in_seq=0 out_seq=0 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.747+0000 7fc66b7fe640 10 client.16479 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc66b7fe640 1 -- 172.21.15.92:0/522481273 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc6800035f0 con 0x558608c369f0 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc66b7fe640 10 client.16479 handle_command_reply: tid=0 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 client.16479 shutdown 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 msgr2=0x558609036d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 0x558609036d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc680006010 tx=0x7fc68002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 shutdown_connections 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c369f0 0x55860903caa0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c78070 0x558609036d60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c57390 0x55860902ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 --2- 172.21.15.92:0/522481273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 >> 172.21.15.92:0/522481273 conn(0x558608c6c460 msgr2=0x558609022450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:11.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc66b7fe640 0 client.16479 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:11.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 shutdown_connections 2024-09-21T02:08:11.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 1 -- 172.21.15.92:0/522481273 wait complete. 2024-09-21T02:08:11.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.748+0000 7fc698941740 20 client.16479 trim_cache size 0 max 16384 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 Processor -- start 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 -- start start 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x558609020c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x558609036cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b1600 con 0x558608c57390 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b3140 con 0x558608c78070 2024-09-21T02:08:11.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.761+0000 7fc698941740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b1cd0 con 0x558608c4f1b0 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x558609020c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x558609020c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41710/0 (socket says 172.21.15.92:41710) 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 -- 172.21.15.92:0/150287120 learned_addr learned my addr 172.21.15.92:0/150287120 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x558609036cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 -- 172.21.15.92:0/150287120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 msgr2=0x55860902c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 -- 172.21.15.92:0/150287120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 msgr2=0x558609036cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x558609036cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 -- 172.21.15.92:0/150287120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860902b890 con 0x558608c57390 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc68bfff640 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x558609020c60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000c00 tx=0x7fc67c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1560f50b4b6c0f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc671ffb640 1 -- 172.21.15.92:0/150287120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c014020 con 0x558608c57390 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc671ffb640 1 -- 172.21.15.92:0/150287120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc67c01c3d0 con 0x558608c57390 2024-09-21T02:08:11.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.762+0000 7fc671ffb640 1 -- 172.21.15.92:0/150287120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c010690 con 0x558608c57390 2024-09-21T02:08:11.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 -- 172.21.15.92:0/150287120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 msgr2=0x558609020c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x558609020c60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000c00 tx=0x7fc67c00cc70 comp rx=0 tx=0).stop 2024-09-21T02:08:11.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 -- 172.21.15.92:0/150287120 shutdown_connections 2024-09-21T02:08:11.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x558609036cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c57390 0x558609020c60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 --2- 172.21.15.92:0/150287120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c4f1b0 0x55860902c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 -- 172.21.15.92:0/150287120 >> 172.21.15.92:0/150287120 conn(0x558608c6c460 msgr2=0x5586090243d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 -- 172.21.15.92:0/150287120 shutdown_connections 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 -- 172.21.15.92:0/150287120 wait complete. 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.763+0000 7fc698941740 1 -- start start 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 Processor -- start 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 0x55860903db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c57390 0x55860903e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860903e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b2a70 con 0x558608c4f1b0 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b23a0 con 0x558608c78070 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1af5e0 con 0x558608c57390 2024-09-21T02:08:11.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc698941740 10 client.0 _ll_register_callbacks cb 0x7fc680032d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 0x55860903db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 0x55860903db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41716/0 (socket says 172.21.15.92:41716) 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 -- 172.21.15.92:0/468479953 learned_addr learned my addr 172.21.15.92:0/468479953 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68bfff640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c57390 0x55860903e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860903e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 -- 172.21.15.92:0/468479953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c57390 msgr2=0x55860903e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c57390 0x55860903e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 -- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 msgr2=0x55860903e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860903e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.764+0000 7fc68b7fe640 1 -- 172.21.15.92:0/468479953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586090a6250 con 0x558608c4f1b0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc68b7fe640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 0x55860903db50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc68002a930 tx=0x7fc68002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07f3041fa792fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc66b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:11.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc66b7fe640 1 -- 172.21.15.92:0/468479953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680035070 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc66b7fe640 1 -- 172.21.15.92:0/468479953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc680003b40 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67c01c3d0 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc66b7fe640 1 -- 172.21.15.92:0/468479953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc680003800 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc698941740 10 client.16485 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586090a7500 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc66b7fe640 1 -- 172.21.15.92:0/468479953 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc67c01c3d0 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc66b7fe640 1 -- 172.21.15.92:0/468479953 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc680057830 con 0x558608c4f1b0 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.765+0000 7fc698941740 10 client.16485 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc698941740 10 client.16485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc698941740 20 client.16485 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc698941740 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c0bda0 0x5586090a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc698941740 4 client.16485 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:11.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558608f97df0 con 0x558608c0bda0 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc68bfff640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c0bda0 0x5586090a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc66b7fe640 10 client.16485 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.766+0000 7fc68bfff640 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c0bda0 0x5586090a9cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5762040dea69b647 server_cookie=1a535402c0142e86 in_seq=0 out_seq=0 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc66b7fe640 1 -- 172.21.15.92:0/468479953 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558608f97df0 con 0x558608c0bda0 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc66b7fe640 10 client.16485 handle_command_reply: tid=0 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 client.16485 shutdown 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 msgr2=0x55860903db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 0x55860903db50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc68002a930 tx=0x7fc68002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 shutdown_connections 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558608c78070 0x55860903e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558608c57390 0x55860903e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x558608c0bda0 0x5586090a9cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 --2- 172.21.15.92:0/468479953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558608c4f1b0 0x55860903db50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 >> 172.21.15.92:0/468479953 conn(0x558608c6c460 msgr2=0x558609020a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:11.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc66b7fe640 0 client.16485 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:11.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 shutdown_connections 2024-09-21T02:08:11.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 1 -- 172.21.15.92:0/468479953 wait complete. 2024-09-21T02:08:11.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc698941740 20 client.16485 trim_cache size 0 max 16384 2024-09-21T02:08:11.773 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6740b6510 msgr2=0x7fc6740b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc6923cd640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6740b6510 0x7fc6740b8a20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc6840e5380 tx=0x7fc68407a9d0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 msgr2=0x7fc68c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.767+0000 7fc6923cd640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 0x7fc68c103230 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc68400eaa0 tx=0x7fc68400ef60 comp rx=0 tx=0).stop 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 shutdown_connections 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6740b6510 0x7fc6740b8a20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc68c10de50 0x7fc68c103230 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc68c1052d0 0x7fc68c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 --2- 172.21.15.92:0/3580306636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1048a0 0x7fc68c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 >> 172.21.15.92:0/3580306636 conn(0x7fc68c079730 msgr2=0x7fc68c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 shutdown_connections 2024-09-21T02:08:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:11.768+0000 7fc6923cd640 1 -- 172.21.15.92:0/3580306636 wait complete. 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 -- 172.21.15.92:0/2251857308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 msgr2=0x7faa20103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 --2- 172.21.15.92:0/2251857308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa20103a60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7faa10009900 tx=0x7faa1002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 -- 172.21.15.92:0/2251857308 shutdown_connections 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 --2- 172.21.15.92:0/2251857308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa20103ff0 0x7faa2010a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 --2- 172.21.15.92:0/2251857308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa20103a60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 --2- 172.21.15.92:0/2251857308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 0x7faa20102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 -- 172.21.15.92:0/2251857308 >> 172.21.15.92:0/2251857308 conn(0x7faa200fdf30 msgr2=0x7faa201003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 -- 172.21.15.92:0/2251857308 shutdown_connections 2024-09-21T02:08:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.318+0000 7faa26f6a640 1 -- 172.21.15.92:0/2251857308 wait complete. 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 Processor -- start 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 -- start start 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 0x7faa2006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa2019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa20103ff0 0x7faa201a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa2010e220 con 0x7faa20103650 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa2010e0a0 con 0x7faa201025d0 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa26f6a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa2010e3a0 con 0x7faa20103ff0 2024-09-21T02:08:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa2019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa2019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41744/0 (socket says 172.21.15.92:41744) 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 -- 172.21.15.92:0/365299140 learned_addr learned my addr 172.21.15.92:0/365299140 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa24d06640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 0x7faa2006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 -- 172.21.15.92:0/365299140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa20103ff0 msgr2=0x7faa201a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa25507640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa20103ff0 0x7faa201a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa20103ff0 0x7faa201a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 -- 172.21.15.92:0/365299140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 msgr2=0x7faa2006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 0x7faa2006c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 -- 172.21.15.92:0/365299140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa201a19d0 con 0x7faa20103650 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa24d06640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 0x7faa2006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.319+0000 7faa17fff640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa2019ec60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faa10006010 tx=0x7faa10004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.320+0000 7faa15ffb640 1 -- 172.21.15.92:0/365299140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1003f030 con 0x7faa20103650 2024-09-21T02:08:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.320+0000 7faa15ffb640 1 -- 172.21.15.92:0/365299140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa10003cb0 con 0x7faa20103650 2024-09-21T02:08:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.320+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa201a1c60 con 0x7faa20103650 2024-09-21T02:08:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.320+0000 7faa15ffb640 1 -- 172.21.15.92:0/365299140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10035070 con 0x7faa20103650 2024-09-21T02:08:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.320+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa201a2150 con 0x7faa20103650 2024-09-21T02:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 Processor -- start 2024-09-21T02:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 -- start start 2024-09-21T02:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa15ffb640 1 -- 172.21.15.92:0/365299140 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7faa10042030 con 0x7faa20103650 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 0x5606e34e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5606e345b6f0 con 0x5606e31411b0 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606e344ab20 con 0x5606e31204d0 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.322+0000 7faa2ded4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5606e344e930 con 0x5606e31182f0 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 0x5606e34e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 0x5606e34e8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41756/0 (socket says 172.21.15.92:41756) 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 -- 172.21.15.92:0/1366859217 learned_addr learned my addr 172.21.15.92:0/1366859217 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa24d06640 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa17fff640 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 -- 172.21.15.92:0/1366859217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 msgr2=0x5606e34e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 -- 172.21.15.92:0/1366859217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 msgr2=0x5606e34ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34ea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 -- 172.21.15.92:0/1366859217 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e3410530 con 0x5606e31411b0 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa25507640 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 0x5606e34e8e50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7faa18009df0 tx=0x7faa1800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fdbc9cfadf97f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa15ffb640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa080b6510 0x7faa080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa06ffd640 1 -- 172.21.15.92:0/1366859217 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18003970 con 0x5606e31411b0 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa24d06640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa080b6510 0x7faa080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa06ffd640 1 -- 172.21.15.92:0/1366859217 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa18004030 con 0x5606e31411b0 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa06ffd640 1 -- 172.21.15.92:0/1366859217 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18011720 con 0x5606e31411b0 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.323+0000 7faa15ffb640 1 -- 172.21.15.92:0/365299140 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faa10107a80 con 0x7faa20103650 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 -- 172.21.15.92:0/1366859217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 msgr2=0x5606e34e8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa24d06640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa080b6510 0x7faa080b8a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0058c0 tx=0x7faa1c00c330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 0x5606e34e8e50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7faa18009df0 tx=0x7faa1800f730 comp rx=0 tx=0).stop 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 -- 172.21.15.92:0/1366859217 shutdown_connections 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1366859217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31411b0 0x5606e34e8e50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 -- 172.21.15.92:0/1366859217 >> 172.21.15.92:0/1366859217 conn(0x5606e3135300 msgr2=0x5606e34e48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 -- 172.21.15.92:0/1366859217 shutdown_connections 2024-09-21T02:08:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.324+0000 7faa2ded4740 1 -- 172.21.15.92:0/1366859217 wait complete. 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 -- start start 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 Processor -- start 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5606e3382ab0 con 0x5606e31182f0 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606e33f0730 con 0x5606e31204d0 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5606e34eab30 con 0x5606e31411b0 2024-09-21T02:08:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa2ded4740 10 client.0 _ll_register_callbacks cb 0x5606e34446b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa24d06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa24d06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41766/0 (socket says 172.21.15.92:41766) 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa24d06640 1 -- 172.21.15.92:0/2697428314 learned_addr learned my addr 172.21.15.92:0/2697428314 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.325+0000 7faa17fff640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa24d06640 1 -- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 msgr2=0x5606e34f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa24d06640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa24d06640 1 -- 172.21.15.92:0/2697428314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 msgr2=0x5606e34f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa24d06640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa24d06640 1 -- 172.21.15.92:0/2697428314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e31ac790 con 0x5606e31182f0 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa17fff640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa24d06640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34f25d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7faa1c011e10 tx=0x7faa1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8842ea7221527244 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7fa9fffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7fa9fffff640 1 -- 172.21.15.92:0/2697428314 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c005020 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7fa9fffff640 1 -- 172.21.15.92:0/2697428314 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faa1c0051c0 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa18004030 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7fa9fffff640 1 -- 172.21.15.92:0/2697428314 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa1c02c070 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa2ded4740 10 client.16497 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606e3225620 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7fa9fffff640 1 -- 172.21.15.92:0/2697428314 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa18004030 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.326+0000 7fa9fffff640 1 -- 172.21.15.92:0/2697428314 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa1c01e420 con 0x5606e31182f0 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa2ded4740 10 client.16497 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa2ded4740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa2ded4740 20 client.16497 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa2ded4740 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30ffd80 0x5606e34ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa2ded4740 4 client.16497 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606e345d640 con 0x5606e30ffd80 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa17fff640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30ffd80 0x5606e34ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7faa17fff640 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30ffd80 0x5606e34ffef0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359f81629f518846 server_cookie=2a386d3d1436793d in_seq=0 out_seq=0 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.327+0000 7fa9fffff640 10 client.16497 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7fa9fffff640 1 -- 172.21.15.92:0/2697428314 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606e345d640 con 0x5606e30ffd80 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7fa9fffff640 10 client.16497 handle_command_reply: tid=0 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 client.16497 shutdown 2024-09-21T02:08:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 msgr2=0x5606e34f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34f25d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7faa1c011e10 tx=0x7faa1c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 shutdown_connections 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30ffd80 0x5606e34ffef0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 --2- 172.21.15.92:0/2697428314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34f25d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7fa9fffff640 0 client.16497 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 >> 172.21.15.92:0/2697428314 conn(0x5606e3135300 msgr2=0x5606e34e2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 shutdown_connections 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.328+0000 7faa2ded4740 1 -- 172.21.15.92:0/2697428314 wait complete. 2024-09-21T02:08:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.329+0000 7faa2ded4740 20 client.16497 trim_cache size 0 max 16384 2024-09-21T02:08:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 Processor -- start 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 -- start start 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 0x5606e34e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31411b0 0x5606e34e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa201aafe0 con 0x5606e31204d0 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa201acb20 con 0x5606e31411b0 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa2ded4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa201ab6b0 con 0x5606e31182f0 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 0x5606e34e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 0x5606e34e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41782/0 (socket says 172.21.15.92:41782) 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 -- 172.21.15.92:0/510617246 learned_addr learned my addr 172.21.15.92:0/510617246 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa24d06640 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa25507640 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31411b0 0x5606e34e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 -- 172.21.15.92:0/510617246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 msgr2=0x5606e34e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 -- 172.21.15.92:0/510617246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31411b0 msgr2=0x5606e34e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31411b0 0x5606e34e53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa17fff640 1 -- 172.21.15.92:0/510617246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e34e5010 con 0x5606e31204d0 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.343+0000 7faa25507640 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31411b0 0x5606e34e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.344+0000 7faa17fff640 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 0x5606e34e9330 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa10005e00 tx=0x7faa1003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ede9427f2290588 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.344+0000 7faa067fc640 1 -- 172.21.15.92:0/510617246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10049040 con 0x5606e31204d0 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.344+0000 7faa067fc640 1 -- 172.21.15.92:0/510617246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa10003e60 con 0x5606e31204d0 2024-09-21T02:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.344+0000 7faa067fc640 1 -- 172.21.15.92:0/510617246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa100f10e0 con 0x5606e31204d0 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- 172.21.15.92:0/510617246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 msgr2=0x5606e34e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 0x5606e34e9330 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7faa10005e00 tx=0x7faa1003b6b0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- 172.21.15.92:0/510617246 shutdown_connections 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31411b0 0x5606e34e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31204d0 0x5606e34e9330 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- 172.21.15.92:0/510617246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31182f0 0x5606e34e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- 172.21.15.92:0/510617246 >> 172.21.15.92:0/510617246 conn(0x5606e3135300 msgr2=0x5606e34df5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- 172.21.15.92:0/510617246 shutdown_connections 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- 172.21.15.92:0/510617246 wait complete. 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- start start 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 Processor -- start 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa201ac450 con 0x5606e31182f0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa201abd80 con 0x5606e31204d0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa201a8fc0 con 0x5606e31411b0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.345+0000 7faa2ded4740 10 client.0 _ll_register_callbacks cb 0x7faa1c00c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41788/0 (socket says 172.21.15.92:41788) 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 -- 172.21.15.92:0/1941634312 learned_addr learned my addr 172.21.15.92:0/1941634312 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa17fff640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa25507640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 -- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 msgr2=0x5606e34f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 -- 172.21.15.92:0/1941634312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 msgr2=0x5606e34f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 -- 172.21.15.92:0/1941634312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e34f9e30 con 0x5606e31182f0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa24d06640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34fa440 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa1c011b60 tx=0x7faa1c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a7461c5b3d3eca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7fa9fffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7fa9fffff640 1 -- 172.21.15.92:0/1941634312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c01ece0 con 0x5606e31182f0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7fa9fffff640 1 -- 172.21.15.92:0/1941634312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faa1c01e420 con 0x5606e31182f0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa10003e60 con 0x5606e31182f0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.346+0000 7fa9fffff640 1 -- 172.21.15.92:0/1941634312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa1c01e620 con 0x5606e31182f0 2024-09-21T02:08:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 10 client.16503 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606e356a8f0 con 0x5606e31182f0 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7fa9fffff640 1 -- 172.21.15.92:0/1941634312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa10003e60 con 0x5606e31182f0 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7fa9fffff640 1 -- 172.21.15.92:0/1941634312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa1c031480 con 0x5606e31182f0 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 20 client.16503 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30d51c0 0x5606e356d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 4 client.16503 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:08:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.347+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606e34590f0 con 0x5606e30d51c0 2024-09-21T02:08:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.348+0000 7faa17fff640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30d51c0 0x5606e356d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.348+0000 7faa17fff640 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30d51c0 0x5606e356d330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f55928bd8ba6243 server_cookie=fe1b2b9edb30cfbf in_seq=0 out_seq=0 2024-09-21T02:08:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.348+0000 7fa9fffff640 10 client.16503 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7fa9fffff640 1 -- 172.21.15.92:0/1941634312 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606e34590f0 con 0x5606e30d51c0 2024-09-21T02:08:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7fa9fffff640 10 client.16503 handle_command_reply: tid=0 2024-09-21T02:08:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 client.16503 shutdown 2024-09-21T02:08:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 msgr2=0x5606e34fa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34fa440 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7faa1c011b60 tx=0x7faa1c00cf10 comp rx=0 tx=0).stop 2024-09-21T02:08:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 shutdown_connections 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5606e30d51c0 0x5606e356d330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5606e31411b0 0x5606e34f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7fa9fffff640 0 client.16503 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e31204d0 0x5606e34f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 --2- 172.21.15.92:0/1941634312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5606e31182f0 0x5606e34fa440 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.362+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 >> 172.21.15.92:0/1941634312 conn(0x5606e3135300 msgr2=0x5606e34f2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 shutdown_connections 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa2ded4740 1 -- 172.21.15.92:0/1941634312 wait complete. 2024-09-21T02:08:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa2ded4740 20 client.16503 trim_cache size 0 max 16384 2024-09-21T02:08:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:08:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:08:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:08:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:08:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa080b6510 msgr2=0x7faa080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa080b6510 0x7faa080b8a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0058c0 tx=0x7faa1c00c330 comp rx=0 tx=0).stop 2024-09-21T02:08:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 msgr2=0x7faa2019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa2019ec60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faa10006010 tx=0x7faa10004340 comp rx=0 tx=0).stop 2024-09-21T02:08:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 shutdown_connections 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa080b6510 0x7faa080b8a20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa20103ff0 0x7faa201a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa20103650 0x7faa2019ec60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 --2- 172.21.15.92:0/365299140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa201025d0 0x7faa2006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.363+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 >> 172.21.15.92:0/365299140 conn(0x7faa200fdf30 msgr2=0x7faa20106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.364+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 shutdown_connections 2024-09-21T02:08:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:12.364+0000 7faa26f6a640 1 -- 172.21.15.92:0/365299140 wait complete. 2024-09-21T02:08:12.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:08:12.819 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:13.211 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:12 smithi092 ceph-mon[31190]: pgmap v457: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:08:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:12 smithi136 ceph-mon[34754]: pgmap v457: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 -- 172.21.15.92:0/3762383761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4102a60 msgr2=0x7f42d4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 --2- 172.21.15.92:0/3762383761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4102a60 0x7f42d4102e70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009900 tx=0x7f42c402a410 comp rx=0 tx=0).stop 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 -- 172.21.15.92:0/3762383761 shutdown_connections 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 --2- 172.21.15.92:0/3762383761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42d4108cf0 0x7f42d410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 --2- 172.21.15.92:0/3762383761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d41083a0 0x7f42d41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 --2- 172.21.15.92:0/3762383761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4102a60 0x7f42d4102e70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 -- 172.21.15.92:0/3762383761 >> 172.21.15.92:0/3762383761 conn(0x7f42d4079730 msgr2=0x7f42d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 -- 172.21.15.92:0/3762383761 shutdown_connections 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 -- 172.21.15.92:0/3762383761 wait complete. 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.302+0000 7f42d9538640 1 Processor -- start 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 -- start start 2024-09-21T02:08:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d4102a60 0x7f42d419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42d41083a0 0x7f42d419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 0x7f42d419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4112960 con 0x7f42d4108cf0 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41127e0 con 0x7f42d4102a60 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d9538640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4112ae0 con 0x7f42d41083a0 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 0x7f42d419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 0x7f42d419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33650/0 (socket says 172.21.15.92:33650) 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 -- 172.21.15.92:0/1950132790 learned_addr learned my addr 172.21.15.92:0/1950132790 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d4102a60 0x7f42d419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d27fc640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42d41083a0 0x7f42d419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 -- 172.21.15.92:0/1950132790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42d41083a0 msgr2=0x7f42d419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42d41083a0 0x7f42d419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 -- 172.21.15.92:0/1950132790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d4102a60 msgr2=0x7f42d419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d4102a60 0x7f42d419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.303+0000 7f42d37fe640 1 -- 172.21.15.92:0/1950132790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d41a64d0 con 0x7f42d4108cf0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.304+0000 7f42d37fe640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 0x7f42d419bc90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00eb10 tx=0x7f42cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.304+0000 7f42bbfff640 1 -- 172.21.15.92:0/1950132790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc004b20 con 0x7f42d4108cf0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.304+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d41a67c0 con 0x7f42d4108cf0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.304+0000 7f42bbfff640 1 -- 172.21.15.92:0/1950132790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42cc004cc0 con 0x7f42d4108cf0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.304+0000 7f42bbfff640 1 -- 172.21.15.92:0/1950132790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc010670 con 0x7f42d4108cf0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.304+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d41a6d00 con 0x7f42d4108cf0 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.306+0000 7f42dfaac740 1 Processor -- start 2024-09-21T02:08:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.306+0000 7f42bbfff640 1 -- 172.21.15.92:0/1950132790 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f42cc010830 con 0x7f42d4108cf0 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.306+0000 7f42dfaac740 1 -- start start 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae00d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae00e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42dfaac740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560badf83ea0 con 0x560badc45390 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42dfaac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560badf732d0 con 0x560badc3d1b0 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42dfaac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560badf770e0 con 0x560badc66070 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae00e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae00e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33656/0 (socket says 172.21.15.92:33656) 2024-09-21T02:08:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 -- 172.21.15.92:0/3749047332 learned_addr learned my addr 172.21.15.92:0/3749047332 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae00d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 -- 172.21.15.92:0/3749047332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 msgr2=0x560bae00d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae00d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 -- 172.21.15.92:0/3749047332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 msgr2=0x560bae00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 -- 172.21.15.92:0/3749047332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560badcd1650 con 0x560badc45390 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42bbfff640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f42bc0b6510 0x7f42bc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d27fc640 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae00e120 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009f60 tx=0x7f42c8009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25582ef36b84d67e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f42bc0b6510 0x7f42bc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42bbfff640 1 -- 172.21.15.92:0/1950132790 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f42cc0e7950 con 0x7f42d4108cf0 2024-09-21T02:08:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.307+0000 7f42b97fa640 1 -- 172.21.15.92:0/3749047332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8004250 con 0x560badc45390 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42b97fa640 1 -- 172.21.15.92:0/3749047332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c80043f0 con 0x560badc45390 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42b97fa640 1 -- 172.21.15.92:0/3749047332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8010930 con 0x560badc45390 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 -- 172.21.15.92:0/3749047332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 msgr2=0x560bae00e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f42bc0b6510 0x7f42bc0b8a20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000c00 tx=0x7f42c4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae00e120 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009f60 tx=0x7f42c8009790 comp rx=0 tx=0).stop 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 -- 172.21.15.92:0/3749047332 shutdown_connections 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae00e120 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3749047332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae00d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 -- 172.21.15.92:0/3749047332 >> 172.21.15.92:0/3749047332 conn(0x560badc5a460 msgr2=0x560bae00b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.308+0000 7f42dfaac740 1 -- 172.21.15.92:0/3749047332 shutdown_connections 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 -- 172.21.15.92:0/3749047332 wait complete. 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 -- start start 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 Processor -- start 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 0x560bae018d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae019290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae025170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560baddd4700 con 0x560badc3d1b0 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560badf0e160 con 0x560badc45390 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bae01be80 con 0x560badc66070 2024-09-21T02:08:13.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.309+0000 7f42dfaac740 10 client.0 _ll_register_callbacks cb 0x560badf279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 0x560bae018d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 0x560bae018d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33670/0 (socket says 172.21.15.92:33670) 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 -- 172.21.15.92:0/2572402840 learned_addr learned my addr 172.21.15.92:0/2572402840 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d37fe640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae025170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 -- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 msgr2=0x560bae025170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae025170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d27fc640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae019290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 -- 172.21.15.92:0/2572402840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 msgr2=0x560bae019290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae019290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 -- 172.21.15.92:0/2572402840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560badf3bc90 con 0x560badc3d1b0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 0x560bae018d50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f42c4033000 tx=0x7f42c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741d26357afc2b50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:13.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42b6ff5640 1 -- 172.21.15.92:0/2572402840 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c404c030 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42b6ff5640 1 -- 172.21.15.92:0/2572402840 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f42c4004250 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c80043f0 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42b6ff5640 1 -- 172.21.15.92:0/2572402840 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42c4058050 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42dfaac740 10 client.16515 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.310+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bae027b90 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42b6ff5640 1 -- 172.21.15.92:0/2572402840 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c80043f0 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42b6ff5640 1 -- 172.21.15.92:0/2572402840 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f42c4048440 con 0x560badc3d1b0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42dfaac740 10 client.16515 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42dfaac740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42dfaac740 20 client.16515 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42dfaac740 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc249f0 0x560bae02b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42dfaac740 4 client.16515 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560badf85df0 con 0x560badc249f0 2024-09-21T02:08:13.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42d27fc640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc249f0 0x560bae02b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.311+0000 7f42d27fc640 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc249f0 0x560bae02b090 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44b4cf2c627af6f server_cookie=8f3072872ea160a in_seq=0 out_seq=0 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42b6ff5640 10 client.16515 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42b6ff5640 1 -- 172.21.15.92:0/2572402840 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560badf85df0 con 0x560badc249f0 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42b6ff5640 10 client.16515 handle_command_reply: tid=0 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 client.16515 shutdown 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 msgr2=0x560bae018d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 0x560bae018d50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f42c4033000 tx=0x7f42c4009260 comp rx=0 tx=0).stop 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 shutdown_connections 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc249f0 0x560bae02b090 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc66070 0x560bae025170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42b6ff5640 0 client.16515 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae019290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 --2- 172.21.15.92:0/2572402840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc3d1b0 0x560bae018d50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.312+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 >> 172.21.15.92:0/2572402840 conn(0x560badc5a460 msgr2=0x560bae010830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.313+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 shutdown_connections 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.313+0000 7f42dfaac740 1 -- 172.21.15.92:0/2572402840 wait complete. 2024-09-21T02:08:13.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.313+0000 7f42dfaac740 20 client.16515 trim_cache size 0 max 16384 2024-09-21T02:08:13.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:12 smithi099 ceph-mon[34404]: pgmap v457: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 Processor -- start 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 -- start start 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc249f0 0x560bae01ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae025070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41af4d0 con 0x560badc45390 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41b1010 con 0x560badc3d1b0 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42dfaac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41afba0 con 0x560badc249f0 2024-09-21T02:08:13.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42d37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae025070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42d37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae025070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33680/0 (socket says 172.21.15.92:33680) 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.326+0000 7f42d37fe640 1 -- 172.21.15.92:0/3969940691 learned_addr learned my addr 172.21.15.92:0/3969940691 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d27fc640 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc249f0 0x560bae01ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d37fe640 1 -- 172.21.15.92:0/3969940691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc249f0 msgr2=0x560bae01ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d37fe640 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc249f0 0x560bae01ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d37fe640 1 -- 172.21.15.92:0/3969940691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 msgr2=0x560bae00e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d37fe640 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d37fe640 1 -- 172.21.15.92:0/3969940691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bae01b330 con 0x560badc45390 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc249f0 0x560bae01ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42d37fe640 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae025070 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0e5430 tx=0x7f42cc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bda26852ec5d150 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42b8ff9640 1 -- 172.21.15.92:0/3969940691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0b2030 con 0x560badc45390 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42b8ff9640 1 -- 172.21.15.92:0/3969940691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42cc0ad5d0 con 0x560badc45390 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.327+0000 7f42b8ff9640 1 -- 172.21.15.92:0/3969940691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0b74e0 con 0x560badc45390 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 -- 172.21.15.92:0/3969940691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 msgr2=0x560bae025070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae025070 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0e5430 tx=0x7f42cc00cee0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 -- 172.21.15.92:0/3969940691 shutdown_connections 2024-09-21T02:08:13.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc45390 0x560bae025070 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc3d1b0 0x560bae00e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 --2- 172.21.15.92:0/3969940691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc249f0 0x560bae01ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 -- 172.21.15.92:0/3969940691 >> 172.21.15.92:0/3969940691 conn(0x560badc5a460 msgr2=0x560bae0127b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 -- 172.21.15.92:0/3969940691 shutdown_connections 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 -- 172.21.15.92:0/3969940691 wait complete. 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.328+0000 7f42dfaac740 1 -- start start 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 Processor -- start 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 0x560bae01db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc3d1b0 0x560bae01e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae08eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41b0940 con 0x560badc249f0 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41b0270 con 0x560badc45390 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41ad4b0 con 0x560badc3d1b0 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42dfaac740 10 client.0 _ll_register_callbacks cb 0x7f42cc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 0x560bae01db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 0x560bae01db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33692/0 (socket says 172.21.15.92:33692) 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1923729936 learned_addr learned my addr 172.21.15.92:0/1923729936 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d37fe640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae08eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d27fc640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc3d1b0 0x560bae01e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc3d1b0 msgr2=0x560bae01e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc3d1b0 0x560bae01e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1923729936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 msgr2=0x560bae08eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae08eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.329+0000 7f42d2ffd640 1 -- 172.21.15.92:0/1923729936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bae08f170 con 0x560badc249f0 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42d2ffd640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 0x560bae01db40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f42c402a930 tx=0x7f42c4048a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c5419c139a2c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42b6ff5640 1 -- 172.21.15.92:0/1923729936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c404c040 con 0x560badc249f0 2024-09-21T02:08:13.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42b6ff5640 1 -- 172.21.15.92:0/1923729936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f42c4004450 con 0x560badc249f0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42cc0ad5d0 con 0x560badc249f0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42b6ff5640 1 -- 172.21.15.92:0/1923729936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42c4035040 con 0x560badc249f0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 10 client.16521 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bae0965c0 con 0x560badc249f0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42b6ff5640 1 -- 172.21.15.92:0/1923729936 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42cc0ad5d0 con 0x560badc249f0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42b6ff5640 1 -- 172.21.15.92:0/1923729936 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f42c40046c0 con 0x560badc249f0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 10 client.16521 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 20 client.16521 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc66070 0x560bae098db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 4 client.16521 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.330+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560badf818a0 con 0x560badc66070 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42d27fc640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc66070 0x560bae098db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:13.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42d27fc640 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc66070 0x560bae098db0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8526e038e1fc0df4 server_cookie=23e2baa8e50ed650 in_seq=0 out_seq=0 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42b6ff5640 10 client.16521 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42b6ff5640 1 -- 172.21.15.92:0/1923729936 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560badf818a0 con 0x560badc66070 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42b6ff5640 10 client.16521 handle_command_reply: tid=0 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42dfaac740 1 client.16521 shutdown 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 msgr2=0x560bae01db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42dfaac740 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 0x560bae01db40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f42c402a930 tx=0x7f42c4048a50 comp rx=0 tx=0).stop 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 shutdown_connections 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.331+0000 7f42dfaac740 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560badc66070 0x560bae098db0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560badc45390 0x560bae08eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560badc3d1b0 0x560bae01e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 1 --2- 172.21.15.92:0/1923729936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560badc249f0 0x560bae01db40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 >> 172.21.15.92:0/1923729936 conn(0x560badc5a460 msgr2=0x560bae01a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:13.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42b6ff5640 0 client.16521 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:13.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 shutdown_connections 2024-09-21T02:08:13.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 1 -- 172.21.15.92:0/1923729936 wait complete. 2024-09-21T02:08:13.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42dfaac740 20 client.16521 trim_cache size 0 max 16384 2024-09-21T02:08:13.346 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f42bc0b6510 msgr2=0x7f42bc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f42bc0b6510 0x7f42bc0b8a20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000c00 tx=0x7f42c4031400 comp rx=0 tx=0).stop 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 msgr2=0x7f42d419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 0x7f42d419bc90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00eb10 tx=0x7f42cc00c2d0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 shutdown_connections 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f42bc0b6510 0x7f42bc0b8a20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42d4108cf0 0x7f42d419bc90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42d41083a0 0x7f42d419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 --2- 172.21.15.92:0/1950132790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d4102a60 0x7f42d419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:13.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.332+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 >> 172.21.15.92:0/1950132790 conn(0x7f42d4079730 msgr2=0x7f42d4101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:13.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.333+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 shutdown_connections 2024-09-21T02:08:13.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:13.333+0000 7f42d9538640 1 -- 172.21.15.92:0/1950132790 wait complete. 2024-09-21T02:08:15.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:08:15.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:08:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:14 smithi136 ceph-mon[34754]: pgmap v458: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-09-21T02:08:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:14 smithi092 ceph-mon[31190]: pgmap v458: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-09-21T02:08:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:14 smithi099 ceph-mon[34404]: pgmap v458: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-09-21T02:08:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.794+0000 7f1844dd9640 1 -- 172.21.15.92:0/1650265133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840108d30 msgr2=0x7f184010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.794+0000 7f1844dd9640 1 --2- 172.21.15.92:0/1650265133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840108d30 0x7f184010f560 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1838009ca0 tx=0x7f183802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 -- 172.21.15.92:0/1650265133 shutdown_connections 2024-09-21T02:08:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 --2- 172.21.15.92:0/1650265133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840108d30 0x7f184010f560 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 --2- 172.21.15.92:0/1650265133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18401083e0 0x7f18401087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 --2- 172.21.15.92:0/1650265133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1840102aa0 0x7f1840102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 -- 172.21.15.92:0/1650265133 >> 172.21.15.92:0/1650265133 conn(0x7f18400797e0 msgr2=0x7f1840079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:15.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 -- 172.21.15.92:0/1650265133 shutdown_connections 2024-09-21T02:08:15.797 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 -- 172.21.15.92:0/1650265133 wait complete. 2024-09-21T02:08:15.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 Processor -- start 2024-09-21T02:08:15.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.795+0000 7f1844dd9640 1 -- start start 2024-09-21T02:08:15.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f1844dd9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 0x7f184019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f1844dd9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18401083e0 0x7f184019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f1844dd9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1840108d30 0x7f184019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f1844dd9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18401129a0 con 0x7f1840102aa0 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f1844dd9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1840112820 con 0x7f18401083e0 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f1844dd9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1840112b20 con 0x7f1840108d30 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 0x7f184019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 0x7f184019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33720/0 (socket says 172.21.15.92:33720) 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 -- 172.21.15.92:0/4234067106 learned_addr learned my addr 172.21.15.92:0/4234067106 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183dd9b640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18401083e0 0x7f184019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183ed9d640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1840108d30 0x7f184019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 -- 172.21.15.92:0/4234067106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1840108d30 msgr2=0x7f184019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1840108d30 0x7f184019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 -- 172.21.15.92:0/4234067106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18401083e0 msgr2=0x7f184019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18401083e0 0x7f184019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.796+0000 7f183e59c640 1 -- 172.21.15.92:0/4234067106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18401a03b0 con 0x7f1840102aa0 2024-09-21T02:08:15.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.797+0000 7f183e59c640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 0x7f184019a440 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f183000d950 tx=0x7f183000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:15.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.797+0000 7f182f7fe640 1 -- 172.21.15.92:0/4234067106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830017070 con 0x7f1840102aa0 2024-09-21T02:08:15.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.797+0000 7f182f7fe640 1 -- 172.21.15.92:0/4234067106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1830009c30 con 0x7f1840102aa0 2024-09-21T02:08:15.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.797+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18401a06a0 con 0x7f1840102aa0 2024-09-21T02:08:15.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.797+0000 7f182f7fe640 1 -- 172.21.15.92:0/4234067106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183000f620 con 0x7f1840102aa0 2024-09-21T02:08:15.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.797+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18401a6e00 con 0x7f1840102aa0 2024-09-21T02:08:15.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.799+0000 7f182f7fe640 1 -- 172.21.15.92:0/4234067106 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f183000f7c0 con 0x7f1840102aa0 2024-09-21T02:08:15.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 Processor -- start 2024-09-21T02:08:15.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 -- start start 2024-09-21T02:08:15.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 0x55767e13f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e13f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55767e0b5e60 con 0x55767dd6f490 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55767e0a5290 con 0x55767dd98350 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.800+0000 7f184b34d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55767e0a90a0 con 0x55767dd77670 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f182f7fe640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18280b6510 0x7f18280b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e13f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 0x55767e13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 0x55767e13f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:57836/0 (socket says 172.21.15.92:57836) 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 -- 172.21.15.92:0/30285025 learned_addr learned my addr 172.21.15.92:0/30285025 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e13f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33726/0 (socket says 172.21.15.92:33726) 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183dd9b640 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18280b6510 0x7f18280b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f182f7fe640 1 -- 172.21.15.92:0/4234067106 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f18300e6f70 con 0x7f1840102aa0 2024-09-21T02:08:15.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 -- 172.21.15.92:0/30285025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 msgr2=0x55767e13feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 -- 172.21.15.92:0/30285025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 msgr2=0x55767e13f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e13f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 -- 172.21.15.92:0/30285025 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767de03930 con 0x55767dd98350 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f183ed9d640 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 0x55767e13f430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1838004af0 tx=0x7f1838009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bacbd4764cbb1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.801+0000 7f182cff9640 1 -- 172.21.15.92:0/30285025 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838004040 con 0x55767dd98350 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.802+0000 7f182cff9640 1 -- 172.21.15.92:0/30285025 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183804e070 con 0x55767dd98350 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.802+0000 7f182cff9640 1 -- 172.21.15.92:0/30285025 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183802fb60 con 0x55767dd98350 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.802+0000 7f183ed9d640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18280b6510 0x7f18280b8a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f183804d000 tx=0x7f183804b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.802+0000 7f184b34d740 1 -- 172.21.15.92:0/30285025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 msgr2=0x55767e13f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.802+0000 7f184b34d740 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 0x55767e13f430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1838004af0 tx=0x7f1838009a20 comp rx=0 tx=0).stop 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 -- 172.21.15.92:0/30285025 shutdown_connections 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e13f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 --2- 172.21.15.92:0/30285025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd98350 0x55767e13f430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 -- 172.21.15.92:0/30285025 >> 172.21.15.92:0/30285025 conn(0x55767dd8c740 msgr2=0x55767e13d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:15.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 -- 172.21.15.92:0/30285025 shutdown_connections 2024-09-21T02:08:15.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 -- 172.21.15.92:0/30285025 wait complete. 2024-09-21T02:08:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 -- start start 2024-09-21T02:08:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 Processor -- start 2024-09-21T02:08:15.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e14f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.803+0000 7f184b34d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 0x55767e15b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f184b34d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55767df06890 con 0x55767dd98350 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f184b34d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55767e03fd40 con 0x55767dd6f490 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f184b34d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55767e151f10 con 0x55767dd77670 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f184b34d740 10 client.0 _ll_register_callbacks cb 0x55767e0a3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 0x55767e15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 0x55767e15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33730/0 (socket says 172.21.15.92:33730) 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 -- 172.21.15.92:0/3209060256 learned_addr learned my addr 172.21.15.92:0/3209060256 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183dd9b640 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e14f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 -- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 msgr2=0x55767e14f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e14f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 -- 172.21.15.92:0/3209060256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 msgr2=0x55767e14f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e14f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 -- 172.21.15.92:0/3209060256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767e15b800 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f183ed9d640 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 0x55767e15b240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1838006010 tx=0x7f1838005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a06fdc8f9e5903a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f1825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f1825ffb640 1 -- 172.21.15.92:0/3209060256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838009540 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f1825ffb640 1 -- 172.21.15.92:0/3209060256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1838003e00 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.804+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183804e070 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f1825ffb640 1 -- 172.21.15.92:0/3209060256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f183802fe80 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 10 client.16533 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55767e15e130 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f1825ffb640 1 -- 172.21.15.92:0/3209060256 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183804e070 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f1825ffb640 1 -- 172.21.15.92:0/3209060256 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1838030840 con 0x55767dd98350 2024-09-21T02:08:15.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 10 client.16533 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 20 client.16533 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd56cd0 0x55767e161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 4 client.16533 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767e0b7db0 con 0x55767dd56cd0 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.805+0000 7f183e59c640 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd56cd0 0x55767e161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.806+0000 7f1825ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.806+0000 7f183e59c640 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd56cd0 0x55767e161490 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b66bcc3de4b2a01 server_cookie=5ae226467ef687b3 in_seq=0 out_seq=0 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f1825ffb640 1 -- 172.21.15.92:0/3209060256 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55767e0b7db0 con 0x55767dd56cd0 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f1825ffb640 10 client.16533 handle_command_reply: tid=0 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 client.16533 shutdown 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 msgr2=0x55767e15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 0x55767e15b240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1838006010 tx=0x7f1838005d80 comp rx=0 tx=0).stop 2024-09-21T02:08:15.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 shutdown_connections 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd56cd0 0x55767e161490 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f1825ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd98350 0x55767e15b240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e14f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 --2- 172.21.15.92:0/3209060256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e14f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 >> 172.21.15.92:0/3209060256 conn(0x55767dd8c740 msgr2=0x55767e13b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 shutdown_connections 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 1 -- 172.21.15.92:0/3209060256 wait complete. 2024-09-21T02:08:15.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.807+0000 7f184b34d740 20 client.16533 trim_cache size 0 max 16384 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 Processor -- start 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 -- start start 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 0x55767e15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e13b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18401ad2c0 con 0x55767dd56cd0 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18401aee00 con 0x55767dd6f490 2024-09-21T02:08:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f184b34d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18401ad990 con 0x55767dd77670 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f183e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 0x55767e15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f183e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 0x55767e15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33740/0 (socket says 172.21.15.92:33740) 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f183e59c640 1 -- 172.21.15.92:0/787046888 learned_addr learned my addr 172.21.15.92:0/787046888 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.825+0000 7f183dd9b640 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183e59c640 1 -- 172.21.15.92:0/787046888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 msgr2=0x55767e13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183e59c640 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183e59c640 1 -- 172.21.15.92:0/787046888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 msgr2=0x55767e13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183e59c640 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e13b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183e59c640 1 -- 172.21.15.92:0/787046888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767e14e150 con 0x55767dd56cd0 2024-09-21T02:08:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183ed9d640 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:08:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183dd9b640 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:08:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f183e59c640 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 0x55767e15b1b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f18300ad070 tx=0x7f18300e4920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521ce41a05149857 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f1827fff640 1 -- 172.21.15.92:0/787046888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830020040 con 0x55767dd56cd0 2024-09-21T02:08:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f1827fff640 1 -- 172.21.15.92:0/787046888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18300b1bc0 con 0x55767dd56cd0 2024-09-21T02:08:15.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.826+0000 7f1827fff640 1 -- 172.21.15.92:0/787046888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300abdb0 con 0x55767dd56cd0 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 -- 172.21.15.92:0/787046888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 msgr2=0x55767e15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 0x55767e15b1b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f18300ad070 tx=0x7f18300e4920 comp rx=0 tx=0).stop 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 -- 172.21.15.92:0/787046888 shutdown_connections 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd77670 0x55767e13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd6f490 0x55767e13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 --2- 172.21.15.92:0/787046888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd56cd0 0x55767e15b1b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 -- 172.21.15.92:0/787046888 >> 172.21.15.92:0/787046888 conn(0x55767dd8c740 msgr2=0x55767e14a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 -- 172.21.15.92:0/787046888 shutdown_connections 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 -- 172.21.15.92:0/787046888 wait complete. 2024-09-21T02:08:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.827+0000 7f184b34d740 1 -- start start 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 Processor -- start 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd56cd0 0x55767e163530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e163a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd77670 0x55767e163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18401ae730 con 0x55767dd6f490 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18401ae060 con 0x55767dd77670 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18401ab2a0 con 0x55767dd56cd0 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f184b34d740 10 client.0 _ll_register_callbacks cb 0x7f183000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e163a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e163a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33750/0 (socket says 172.21.15.92:33750) 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 -- 172.21.15.92:0/3511786991 learned_addr learned my addr 172.21.15.92:0/3511786991 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183e59c640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd56cd0 0x55767e163530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183ed9d640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd77670 0x55767e163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 -- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd56cd0 msgr2=0x55767e163530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd56cd0 0x55767e163530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 -- 172.21.15.92:0/3511786991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd77670 msgr2=0x55767e163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd77670 0x55767e163fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183dd9b640 1 -- 172.21.15.92:0/3511786991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767e1cbca0 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.828+0000 7f183ed9d640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd77670 0x55767e163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f183dd9b640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e163a70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f183400bb60 tx=0x7f18340039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc556430c2ffa00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f1825ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f1825ffb640 1 -- 172.21.15.92:0/3511786991 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834021020 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f1825ffb640 1 -- 172.21.15.92:0/3511786991 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1834005220 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18300b1bc0 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f1825ffb640 1 -- 172.21.15.92:0/3511786991 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1834005420 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 10 client.16539 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55767e1ccfa0 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f1825ffb640 1 -- 172.21.15.92:0/3511786991 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18300b1bc0 con 0x55767dd6f490 2024-09-21T02:08:15.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f1825ffb640 1 -- 172.21.15.92:0/3511786991 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f183401d7c0 con 0x55767dd6f490 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 10 client.16539 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 20 client.16539 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd98350 0x55767e1cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 4 client.16539 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.829+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55767e0b3860 con 0x55767dd98350 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.830+0000 7f183e59c640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd98350 0x55767e1cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.830+0000 7f1825ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.830+0000 7f183e59c640 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd98350 0x55767e1cf810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cfaafeea89453a server_cookie=8deab0339e03a680 in_seq=0 out_seq=0 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f1825ffb640 1 -- 172.21.15.92:0/3511786991 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55767e0b3860 con 0x55767dd98350 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f1825ffb640 10 client.16539 handle_command_reply: tid=0 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 client.16539 shutdown 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 msgr2=0x55767e163a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e163a70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f183400bb60 tx=0x7f18340039c0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 shutdown_connections 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55767dd98350 0x55767e1cf810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55767dd77670 0x55767e163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55767dd6f490 0x55767e163a70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 --2- 172.21.15.92:0/3511786991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55767dd56cd0 0x55767e163530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 >> 172.21.15.92:0/3511786991 conn(0x55767dd8c740 msgr2=0x55767e14a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f1825ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 shutdown_connections 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 1 -- 172.21.15.92:0/3511786991 wait complete. 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.832+0000 7f184b34d740 20 client.16539 trim_cache size 0 max 16384 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:08:15.838 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:08:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "c7050547-640c-4032-ac18-f3db3ac2d2d9", 2024-09-21T02:08:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:08:15.839 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:08:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18280b6510 msgr2=0x7f18280b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18280b6510 0x7f18280b8a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f183804d000 tx=0x7f183804b000 comp rx=0 tx=0).stop 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 msgr2=0x7f184019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 0x7f184019a440 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f183000d950 tx=0x7f183000de10 comp rx=0 tx=0).stop 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 shutdown_connections 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f18280b6510 0x7f18280b8a20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1840108d30 0x7f184019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18401083e0 0x7f184019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 --2- 172.21.15.92:0/4234067106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1840102aa0 0x7f184019a440 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 >> 172.21.15.92:0/4234067106 conn(0x7f18400797e0 msgr2=0x7f184006bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 shutdown_connections 2024-09-21T02:08:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:15.833+0000 7f1844dd9640 1 -- 172.21.15.92:0/4234067106 wait complete. 2024-09-21T02:08:16.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:08:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:17.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:16 smithi092 ceph-mon[31190]: pgmap v459: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:08:17.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:16 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:08:17.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:08:17.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:16 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:08:17.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:08:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:16 smithi136 ceph-mon[34754]: pgmap v459: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:08:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:16 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:08:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:08:17.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:16 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:08:17.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:08:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:16 smithi099 ceph-mon[34404]: pgmap v459: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:08:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:16 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:08:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:08:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:16 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:08:17.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:08:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 -- 172.21.15.92:0/3328496726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228108420 msgr2=0x7f8228108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 --2- 172.21.15.92:0/3328496726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228108420 0x7f8228108830 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8210009900 tx=0x7f821002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:17.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 -- 172.21.15.92:0/3328496726 shutdown_connections 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 --2- 172.21.15.92:0/3328496726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108d70 0x7f822810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 --2- 172.21.15.92:0/3328496726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228108420 0x7f8228108830 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 --2- 172.21.15.92:0/3328496726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8228102ae0 0x7f8228102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.410+0000 7f822e93d640 1 -- 172.21.15.92:0/3328496726 >> 172.21.15.92:0/3328496726 conn(0x7f8228079770 msgr2=0x7f8228079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 -- 172.21.15.92:0/3328496726 shutdown_connections 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 -- 172.21.15.92:0/3328496726 wait complete. 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 Processor -- start 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 -- start start 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 0x7f82280ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108420 0x7f82280ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8228108d70 0x7f82281000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8228112a70 con 0x7f8228102ae0 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82281128f0 con 0x7f8228108420 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.411+0000 7f822e93d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8228112bf0 con 0x7f8228108d70 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 0x7f82280ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 0x7f82280ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33770/0 (socket says 172.21.15.92:33770) 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 -- 172.21.15.92:0/3785336545 learned_addr learned my addr 172.21.15.92:0/3785336545 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f821ffff640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108420 0x7f82280ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f822ceda640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8228108d70 0x7f82281000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 -- 172.21.15.92:0/3785336545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8228108d70 msgr2=0x7f82281000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8228108d70 0x7f82281000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 -- 172.21.15.92:0/3785336545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108420 msgr2=0x7f82280ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108420 0x7f82280ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 -- 172.21.15.92:0/3785336545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82281a8870 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f821ffff640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108420 0x7f82280ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8227fff640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 0x7f82280ff670 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f821800d770 tx=0x7f821800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8225ffb640 1 -- 172.21.15.92:0/3785336545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218017070 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82281a8b60 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8225ffb640 1 -- 172.21.15.92:0/3785336545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82180054d0 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f8225ffb640 1 -- 172.21.15.92:0/3785336545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218004c30 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.412+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a9070 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 Processor -- start 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8225ffb640 1 -- 172.21.15.92:0/3785336545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8218004e10 con 0x7f8228102ae0 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 -- start start 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d711f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d711fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d71200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5587d7095e50 con 0x5587d6d4f1b0 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587d7085280 con 0x5587d6d57390 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8234eb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5587d7089090 con 0x5587d6d78070 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d711fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d711fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33772/0 (socket says 172.21.15.92:33772) 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 -- 172.21.15.92:0/1117950958 learned_addr learned my addr 172.21.15.92:0/1117950958 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f822ceda640 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d711f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f821ffff640 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d71200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 -- 172.21.15.92:0/1117950958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 msgr2=0x5587d711f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d711f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 -- 172.21.15.92:0/1117950958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 msgr2=0x5587d71200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.415+0000 7f8227fff640 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d71200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f8227fff640 1 -- 172.21.15.92:0/1117950958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d6de3650 con 0x5587d6d4f1b0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f821ffff640 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d71200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f8225ffb640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82140b6510 0x7f82140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f8227fff640 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d711fb90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f82180eafd0 tx=0x7f82180ef570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82908f7137329b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f822ceda640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82140b6510 0x7f82140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f821e7fc640 1 -- 172.21.15.92:0/1117950958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180a9680 con 0x5587d6d4f1b0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f8225ffb640 1 -- 172.21.15.92:0/3785336545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f82180e6ec0 con 0x7f8228102ae0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f821e7fc640 1 -- 172.21.15.92:0/1117950958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82180acb40 con 0x5587d6d4f1b0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.416+0000 7f821e7fc640 1 -- 172.21.15.92:0/1117950958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180d02d0 con 0x5587d6d4f1b0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 -- 172.21.15.92:0/1117950958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 msgr2=0x5587d711fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f822ceda640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82140b6510 0x7f82140b8a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8220009fd0 tx=0x7f822000e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d711fb90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f82180eafd0 tx=0x7f82180ef570 comp rx=0 tx=0).stop 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 -- 172.21.15.92:0/1117950958 shutdown_connections 2024-09-21T02:08:17.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d71200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d711fb90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1117950958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d711f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 -- 172.21.15.92:0/1117950958 >> 172.21.15.92:0/1117950958 conn(0x5587d6d6c460 msgr2=0x5587d711d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 -- 172.21.15.92:0/1117950958 shutdown_connections 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 -- 172.21.15.92:0/1117950958 wait complete. 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.417+0000 7f8234eb1740 1 -- start start 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 Processor -- start 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d4f1b0 0x5587d712ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 0x5587d712b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d712b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5587d6ee6700 con 0x5587d6d57390 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587d7020110 con 0x5587d6d78070 2024-09-21T02:08:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5587d712de30 con 0x5587d6d4f1b0 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8234eb1740 10 client.0 _ll_register_callbacks cb 0x5587d7039990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f821ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 0x5587d712b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f821ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 0x5587d712b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33788/0 (socket says 172.21.15.92:33788) 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f821ffff640 1 -- 172.21.15.92:0/2709658019 learned_addr learned my addr 172.21.15.92:0/2709658019 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f822ceda640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d712b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.418+0000 7f8227fff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d4f1b0 0x5587d712ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f821ffff640 1 -- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d4f1b0 msgr2=0x5587d712ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f821ffff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d4f1b0 0x5587d712ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f821ffff640 1 -- 172.21.15.92:0/2709658019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 msgr2=0x5587d712b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f821ffff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d712b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f821ffff640 1 -- 172.21.15.92:0/2709658019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d704dc40 con 0x5587d6d57390 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f822ceda640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d712b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8227fff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d4f1b0 0x5587d712ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f821ffff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 0x5587d712b220 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8210005ab0 tx=0x7f82100317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26725ec9a3912d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f82077fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f82077fe640 1 -- 172.21.15.92:0/2709658019 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821003f070 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f82077fe640 1 -- 172.21.15.92:0/2709658019 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8210004450 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82180acb40 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f82077fe640 1 -- 172.21.15.92:0/2709658019 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8210035070 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8234eb1740 10 client.16551 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587d7139a70 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f82077fe640 1 -- 172.21.15.92:0/2709658019 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82180acb40 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f82077fe640 1 -- 172.21.15.92:0/2709658019 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8210004bf0 con 0x5587d6d57390 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8234eb1740 10 client.16551 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8234eb1740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.419+0000 7f8234eb1740 20 client.16551 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:17.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.420+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d369f0 0x5587d713cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.420+0000 7f8234eb1740 4 client.16551 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.420+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d7097da0 con 0x5587d6d369f0 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.420+0000 7f8227fff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d369f0 0x5587d713cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.420+0000 7f8227fff640 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d369f0 0x5587d713cfb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebdb264dd1f6c09d server_cookie=7ce0fbf139f730a6 in_seq=0 out_seq=0 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.420+0000 7f82077fe640 10 client.16551 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f82077fe640 1 -- 172.21.15.92:0/2709658019 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587d7097da0 con 0x5587d6d369f0 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f82077fe640 10 client.16551 handle_command_reply: tid=0 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 client.16551 shutdown 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 msgr2=0x5587d712b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 0x5587d712b220 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8210005ab0 tx=0x7f82100317c0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 shutdown_connections 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d369f0 0x5587d713cfb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d712b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d57390 0x5587d712b220 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2709658019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d4f1b0 0x5587d712ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f82077fe640 0 client.16551 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:17.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 >> 172.21.15.92:0/2709658019 conn(0x5587d6d6c460 msgr2=0x5587d71227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:17.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 shutdown_connections 2024-09-21T02:08:17.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 1 -- 172.21.15.92:0/2709658019 wait complete. 2024-09-21T02:08:17.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.421+0000 7f8234eb1740 20 client.16551 trim_cache size 0 max 16384 2024-09-21T02:08:17.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.438+0000 7f8234eb1740 1 Processor -- start 2024-09-21T02:08:17.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.438+0000 7f8234eb1740 1 -- start start 2024-09-21T02:08:17.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d7121040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d712c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d7127ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8234eb1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b1700 con 0x5587d6d4f1b0 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8234eb1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b3240 con 0x5587d6d57390 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8234eb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b1dd0 con 0x5587d6d78070 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d7121040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d7121040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33796/0 (socket says 172.21.15.92:33796) 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 -- 172.21.15.92:0/1689401263 learned_addr learned my addr 172.21.15.92:0/1689401263 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f821ffff640 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d712c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f822ceda640 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d7127ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 -- 172.21.15.92:0/1689401263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 msgr2=0x5587d7127ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d7127ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 -- 172.21.15.92:0/1689401263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 msgr2=0x5587d712c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d712c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 -- 172.21.15.92:0/1689401263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d71224b0 con 0x5587d6d4f1b0 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.439+0000 7f8227fff640 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d7121040 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8218009fd0 tx=0x7f8218094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd729dff29feddff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:17.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f821dffb640 1 -- 172.21.15.92:0/1689401263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180cd040 con 0x5587d6d4f1b0 2024-09-21T02:08:17.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f821dffb640 1 -- 172.21.15.92:0/1689401263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8218092e50 con 0x5587d6d4f1b0 2024-09-21T02:08:17.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f821dffb640 1 -- 172.21.15.92:0/1689401263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180ce000 con 0x5587d6d4f1b0 2024-09-21T02:08:17.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f8234eb1740 1 -- 172.21.15.92:0/1689401263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 msgr2=0x5587d7121040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d7121040 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8218009fd0 tx=0x7f8218094040 comp rx=0 tx=0).stop 2024-09-21T02:08:17.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f8234eb1740 1 -- 172.21.15.92:0/1689401263 shutdown_connections 2024-09-21T02:08:17.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d78070 0x5587d7127ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d57390 0x5587d712c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.440+0000 7f8234eb1740 1 --2- 172.21.15.92:0/1689401263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d7121040 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- 172.21.15.92:0/1689401263 >> 172.21.15.92:0/1689401263 conn(0x5587d6d6c460 msgr2=0x5587d712c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- 172.21.15.92:0/1689401263 shutdown_connections 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- 172.21.15.92:0/1689401263 wait complete. 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- start start 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 Processor -- start 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d712f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d57390 0x5587d712faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d71a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b2b70 con 0x5587d6d4f1b0 2024-09-21T02:08:17.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b24a0 con 0x5587d6d78070 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.441+0000 7f8234eb1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82281af6e0 con 0x5587d6d57390 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8234eb1740 10 client.0 _ll_register_callbacks cb 0x7f8210009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d712f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d712f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33806/0 (socket says 172.21.15.92:33806) 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 -- 172.21.15.92:0/2340237157 learned_addr learned my addr 172.21.15.92:0/2340237157 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f821ffff640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d57390 0x5587d712faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f822ceda640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d71a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 -- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d57390 msgr2=0x5587d712faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d57390 0x5587d712faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 -- 172.21.15.92:0/2340237157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 msgr2=0x5587d71a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d71a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 -- 172.21.15.92:0/2340237157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d71a6eb0 con 0x5587d6d4f1b0 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8227fff640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d712f4e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f82180efa70 tx=0x7f82180e71c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb336f454edc3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f82077fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f82077fe640 1 -- 172.21.15.92:0/2340237157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180cd020 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f82077fe640 1 -- 172.21.15.92:0/2340237157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f82180e4a20 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8218092e50 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f82077fe640 1 -- 172.21.15.92:0/2340237157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82180cee20 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8234eb1740 10 client.16557 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.442+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587d71a8050 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f82077fe640 1 -- 172.21.15.92:0/2340237157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8218092e50 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f82077fe640 1 -- 172.21.15.92:0/2340237157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f82180a92a0 con 0x5587d6d4f1b0 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f8234eb1740 10 client.16557 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f8234eb1740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f8234eb1740 20 client.16557 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d0bda0 0x5587d71aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f8234eb1740 4 client.16557 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:08:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587d7093850 con 0x5587d6d0bda0 2024-09-21T02:08:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.443+0000 7f821ffff640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d0bda0 0x5587d71aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f821ffff640 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d0bda0 0x5587d71aa380 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aab5f7e87690bba server_cookie=ca9764e4886777d9 in_seq=0 out_seq=0 2024-09-21T02:08:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f82077fe640 10 client.16557 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:17.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f82077fe640 1 -- 172.21.15.92:0/2340237157 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x5587d7093850 con 0x5587d6d0bda0 2024-09-21T02:08:17.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f82077fe640 10 client.16557 handle_command_reply: tid=0 2024-09-21T02:08:17.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 client.16557 shutdown 2024-09-21T02:08:17.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 msgr2=0x5587d712f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d712f4e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f82180efa70 tx=0x7f82180e71c0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 shutdown_connections 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5587d6d0bda0 0x5587d71aa380 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d6d78070 0x5587d71a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5587d6d57390 0x5587d712faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 --2- 172.21.15.92:0/2340237157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587d6d4f1b0 0x5587d712f4e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f82077fe640 0 client.16557 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 >> 172.21.15.92:0/2340237157 conn(0x5587d6d6c460 msgr2=0x5587d7136a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 shutdown_connections 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.444+0000 7f8234eb1740 1 -- 172.21.15.92:0/2340237157 wait complete. 2024-09-21T02:08:17.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f8234eb1740 20 client.16557 trim_cache size 0 max 16384 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "c7050547-640c-4032-ac18-f3db3ac2d2d9", 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:08:17.452 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:17.453 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:08:17.454 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:17.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:17.456 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82140b6510 msgr2=0x7f82140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82140b6510 0x7f82140b8a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8220009fd0 tx=0x7f822000e040 comp rx=0 tx=0).stop 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 msgr2=0x7f82280ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 0x7f82280ff670 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f821800d770 tx=0x7f821800dc30 comp rx=0 tx=0).stop 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 shutdown_connections 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f82140b6510 0x7f82140b8a20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8228108d70 0x7f82281000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8228108420 0x7f82280ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 --2- 172.21.15.92:0/3785336545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8228102ae0 0x7f82280ff670 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:17.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 >> 172.21.15.92:0/3785336545 conn(0x7f8228079770 msgr2=0x7f8228101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:17.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 shutdown_connections 2024-09-21T02:08:17.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:17.445+0000 7f822e93d640 1 -- 172.21.15.92:0/3785336545 wait complete. 2024-09-21T02:08:18.296 INFO:tasks.cephfs.filesystem:scrub status for tag:c7050547-640c-4032-ac18-f3db3ac2d2d9 - {'path': '/', 'tag': 'c7050547-640c-4032-ac18-f3db3ac2d2d9', 'options': 'recursive,force'} 2024-09-21T02:08:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:18 smithi136 ceph-mon[34754]: pgmap v460: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:18 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:08:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:18 smithi092 ceph-mon[31190]: pgmap v460: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:18 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:08:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:18 smithi099 ceph-mon[34404]: pgmap v460: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:18 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:08:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:20 smithi136 ceph-mon[34754]: pgmap v461: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:08:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:20 smithi092 ceph-mon[31190]: pgmap v461: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:08:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:20 smithi099 ceph-mon[34404]: pgmap v461: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:08:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:22 smithi136 ceph-mon[34754]: pgmap v462: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:08:23.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:22 smithi092 ceph-mon[31190]: pgmap v462: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:08:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:22 smithi099 ceph-mon[34404]: pgmap v462: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:08:25.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:24 smithi136 ceph-mon[34754]: pgmap v463: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:08:25.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:24 smithi092 ceph-mon[31190]: pgmap v463: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:08:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:24 smithi099 ceph-mon[34404]: pgmap v463: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:08:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.467+0000 7fa660358640 1 -- 172.21.15.92:0/2427218503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 msgr2=0x7fa65810a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.467+0000 7fa660358640 1 --2- 172.21.15.92:0/2427218503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65810a810 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa650009ca0 tx=0x7fa65002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 -- 172.21.15.92:0/2427218503 shutdown_connections 2024-09-21T02:08:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 --2- 172.21.15.92:0/2427218503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65810a810 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 --2- 172.21.15.92:0/2427218503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6581056e0 0x7fa658107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 --2- 172.21.15.92:0/2427218503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa658102c20 0x7fa6581050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 -- 172.21.15.92:0/2427218503 >> 172.21.15.92:0/2427218503 conn(0x7fa6580fd6d0 msgr2=0x7fa6580ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 -- 172.21.15.92:0/2427218503 shutdown_connections 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 -- 172.21.15.92:0/2427218503 wait complete. 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 Processor -- start 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.468+0000 7fa660358640 1 -- start start 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa658102c20 0x7fa658199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6581056e0 0x7fa65819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa65810f330 con 0x7fa658108300 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa65810f1b0 con 0x7fa6581056e0 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa65810f4b0 con 0x7fa658102c20 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38214/0 (socket says 172.21.15.92:38214) 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 -- 172.21.15.92:0/3190839836 learned_addr learned my addr 172.21.15.92:0/3190839836 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6581056e0 0x7fa65819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa658102c20 0x7fa658199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 -- 172.21.15.92:0/3190839836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa658102c20 msgr2=0x7fa658199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa658102c20 0x7fa658199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 -- 172.21.15.92:0/3190839836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6581056e0 msgr2=0x7fa65819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6581056e0 0x7fa65819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 -- 172.21.15.92:0/3190839836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65819b920 con 0x7fa658108300 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65819b100 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097b0 tx=0x7fa650003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa64f7fe640 1 -- 172.21.15.92:0/3190839836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650004240 con 0x7fa658108300 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65819bbb0 con 0x7fa658108300 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa64f7fe640 1 -- 172.21.15.92:0/3190839836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa65002f380 con 0x7fa658108300 2024-09-21T02:08:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa64f7fe640 1 -- 172.21.15.92:0/3190839836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65002fc30 con 0x7fa658108300 2024-09-21T02:08:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.469+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6581a2300 con 0x7fa658108300 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.472+0000 7fa6672bf740 1 Processor -- start 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.472+0000 7fa64f7fe640 1 -- 172.21.15.92:0/3190839836 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa6500305c0 con 0x7fa658108300 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 -- start start 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb23fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb241010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5592bb1b26f0 con 0x5592bae981b0 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592bb1a1b20 con 0x5592bae774d0 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa6672bf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5592bb1a5930 con 0x5592bae6f2f0 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb23fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb23fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38224/0 (socket says 172.21.15.92:38224) 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 -- 172.21.15.92:0/2737761819 learned_addr learned my addr 172.21.15.92:0/2737761819 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb241010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 -- 172.21.15.92:0/2737761819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 msgr2=0x5592bb2404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 -- 172.21.15.92:0/2737761819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 msgr2=0x5592bb241010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb241010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.473+0000 7fa65d8f3640 1 -- 172.21.15.92:0/2737761819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bb167530 con 0x5592bae981b0 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb23fe50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa648012570 tx=0x7fa64800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82271ec7a413ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa64f7fe640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6440b6510 0x7fa6440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa64cbfe640 1 -- 172.21.15.92:0/2737761819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648005020 con 0x5592bae981b0 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6440b6510 0x7fa6440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa64cbfe640 1 -- 172.21.15.92:0/2737761819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648005670 con 0x5592bae981b0 2024-09-21T02:08:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa64cbfe640 1 -- 172.21.15.92:0/2737761819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648010460 con 0x5592bae981b0 2024-09-21T02:08:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.474+0000 7fa64f7fe640 1 -- 172.21.15.92:0/3190839836 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa650108840 con 0x7fa658108300 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 -- 172.21.15.92:0/2737761819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 msgr2=0x5592bb23fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6440b6510 0x7fa6440b8a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa654005fd0 tx=0x7fa65400d300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb23fe50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa648012570 tx=0x7fa64800b7e0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 -- 172.21.15.92:0/2737761819 shutdown_connections 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb241010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 --2- 172.21.15.92:0/2737761819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb23fe50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 -- 172.21.15.92:0/2737761819 >> 172.21.15.92:0/2737761819 conn(0x5592bae8c300 msgr2=0x5592bb23b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 -- 172.21.15.92:0/2737761819 shutdown_connections 2024-09-21T02:08:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 -- 172.21.15.92:0/2737761819 wait complete. 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 -- start start 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.475+0000 7fa6672bf740 1 Processor -- start 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 0x5592bb2495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb249c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae981b0 0x5592bb24a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5592bb0d9ab0 con 0x5592bae6f2f0 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592bb147730 con 0x5592bae774d0 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5592bb241b30 con 0x5592bae981b0 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa6672bf740 10 client.0 _ll_register_callbacks cb 0x5592bb19b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 0x5592bb2495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 0x5592bb2495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38240/0 (socket says 172.21.15.92:38240) 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 -- 172.21.15.92:0/106339078 learned_addr learned my addr 172.21.15.92:0/106339078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae981b0 0x5592bb24a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb249c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 -- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae981b0 msgr2=0x5592bb24a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae981b0 0x5592bb24a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 -- 172.21.15.92:0/106339078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 msgr2=0x5592bb249c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb249c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 -- 172.21.15.92:0/106339078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592baf03790 con 0x5592bae6f2f0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb249c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 0x5592bb2495b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa6540127a0 tx=0x7fa6540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb32239ce4bb034d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.476+0000 7fa641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa641ffb640 1 -- 172.21.15.92:0/106339078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65400fa50 con 0x5592bae6f2f0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa641ffb640 1 -- 172.21.15.92:0/106339078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa65400fbf0 con 0x5592bae6f2f0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa648005670 con 0x5592bae6f2f0 2024-09-21T02:08:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa641ffb640 1 -- 172.21.15.92:0/106339078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa65400fdf0 con 0x5592bae6f2f0 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 10 client.16569 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592baf7c620 con 0x5592bae6f2f0 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa641ffb640 1 -- 172.21.15.92:0/106339078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648005670 con 0x5592bae6f2f0 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa641ffb640 1 -- 172.21.15.92:0/106339078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa65401d680 con 0x5592bae6f2f0 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 10 client.16569 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 20 client.16569 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae56d80 0x5592bb256d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 4 client.16569 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.477+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592bb1b4640 con 0x5592bae56d80 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.478+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae56d80 0x5592bb256d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.478+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae56d80 0x5592bb256d40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204e4f90e95a95b4 server_cookie=7e202d9f6ca79839 in_seq=0 out_seq=0 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.478+0000 7fa641ffb640 10 client.16569 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa641ffb640 1 -- 172.21.15.92:0/106339078 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5592bb1b4640 con 0x5592bae56d80 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa641ffb640 10 client.16569 handle_command_reply: tid=0 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 client.16569 shutdown 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 msgr2=0x5592bb2495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 0x5592bb2495b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa6540127a0 tx=0x7fa6540046c0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 shutdown_connections 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae56d80 0x5592bb256d40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae981b0 0x5592bb24a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae774d0 0x5592bb249c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa641ffb640 0 client.16569 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 --2- 172.21.15.92:0/106339078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae6f2f0 0x5592bb2495b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 >> 172.21.15.92:0/106339078 conn(0x5592bae8c300 msgr2=0x5592bb239ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 shutdown_connections 2024-09-21T02:08:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 1 -- 172.21.15.92:0/106339078 wait complete. 2024-09-21T02:08:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.479+0000 7fa6672bf740 20 client.16569 trim_cache size 0 max 16384 2024-09-21T02:08:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 Processor -- start 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 -- start start 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae6f2f0 0x5592bb23b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae774d0 0x5592bb247ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb248000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581aafe0 con 0x5592bae981b0 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581acb20 con 0x5592bae6f2f0 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa6672bf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581ab6b0 con 0x5592bae774d0 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb248000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb248000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38242/0 (socket says 172.21.15.92:38242) 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 -- 172.21.15.92:0/1536821472 learned_addr learned my addr 172.21.15.92:0/1536821472 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae774d0 0x5592bb247ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae6f2f0 0x5592bb23b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 -- 172.21.15.92:0/1536821472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae774d0 msgr2=0x5592bb247ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae774d0 0x5592bb247ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 -- 172.21.15.92:0/1536821472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae6f2f0 msgr2=0x5592bb23b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae6f2f0 0x5592bb23b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65e8f5640 1 -- 172.21.15.92:0/1536821472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bb251a80 con 0x5592bae981b0 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.494+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae774d0 0x5592bb247ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa65e8f5640 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb248000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa650106320 tx=0x7fa6501061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6fea5f3d495f66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa643fff640 1 -- 172.21.15.92:0/1536821472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500317c0 con 0x5592bae981b0 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa643fff640 1 -- 172.21.15.92:0/1536821472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa650036040 con 0x5592bae981b0 2024-09-21T02:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa643fff640 1 -- 172.21.15.92:0/1536821472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500d83d0 con 0x5592bae981b0 2024-09-21T02:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 -- 172.21.15.92:0/1536821472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 msgr2=0x5592bb248000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb248000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa650106320 tx=0x7fa6501061f0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 -- 172.21.15.92:0/1536821472 shutdown_connections 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae981b0 0x5592bb248000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae774d0 0x5592bb247ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1536821472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae6f2f0 0x5592bb23b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.495+0000 7fa6672bf740 1 -- 172.21.15.92:0/1536821472 >> 172.21.15.92:0/1536821472 conn(0x5592bae8c300 msgr2=0x5592bb2365c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 -- 172.21.15.92:0/1536821472 shutdown_connections 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 -- 172.21.15.92:0/1536821472 wait complete. 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 -- start start 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 Processor -- start 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 0x5592bb2500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae981b0 0x5592bb250800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581ac450 con 0x5592bae774d0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581abd80 con 0x5592bae981b0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581a8fc0 con 0x5592bae6f2f0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa6672bf740 10 client.0 _ll_register_callbacks cb 0x7fa65400dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa65d8f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 0x5592bb2500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa65d8f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 0x5592bb2500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38252/0 (socket says 172.21.15.92:38252) 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa65d8f3640 1 -- 172.21.15.92:0/1993575057 learned_addr learned my addr 172.21.15.92:0/1993575057 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.496+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa65d8f3640 1 -- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 msgr2=0x5592bb2513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa65d8f3640 1 -- 172.21.15.92:0/1993575057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae981b0 msgr2=0x5592bb250800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae981b0 0x5592bb250800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa65d8f3640 1 -- 172.21.15.92:0/1993575057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bb24ec70 con 0x5592bae774d0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa65d8f3640 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 0x5592bb2500f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa648012ec0 tx=0x7fa6480124a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6becf3e7752160 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa641ffb640 1 -- 172.21.15.92:0/1993575057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648004860 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa641ffb640 1 -- 172.21.15.92:0/1993575057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa648004a00 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa650036040 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa641ffb640 1 -- 172.21.15.92:0/1993575057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa648039050 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 10 client.16575 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592bb2c1920 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa641ffb640 1 -- 172.21.15.92:0/1993575057 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa650036040 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa641ffb640 1 -- 172.21.15.92:0/1993575057 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa648010580 con 0x5592bae774d0 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 10 client.16575 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 20 client.16575 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae2c1c0 0x5592bb2c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 4 client.16575 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:08:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.497+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592bb1b00f0 con 0x5592bae2c1c0 2024-09-21T02:08:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.498+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae2c1c0 0x5592bb2c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.498+0000 7fa641ffb640 10 client.16575 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.498+0000 7fa65e0f4640 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae2c1c0 0x5592bb2c4200 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bcda400e45fa21d server_cookie=860aa4da31718a0d in_seq=0 out_seq=0 2024-09-21T02:08:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.512+0000 7fa641ffb640 1 -- 172.21.15.92:0/1993575057 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592bb1b00f0 con 0x5592bae2c1c0 2024-09-21T02:08:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.512+0000 7fa641ffb640 10 client.16575 handle_command_reply: tid=0 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.512+0000 7fa6672bf740 1 client.16575 shutdown 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 msgr2=0x5592bb2500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 0x5592bb2500f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa648012ec0 tx=0x7fa6480124a0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 shutdown_connections 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5592bae2c1c0 0x5592bb2c4200 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592bae981b0 0x5592bb250800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa641ffb640 0 client.16575 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5592bae774d0 0x5592bb2500f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 --2- 172.21.15.92:0/1993575057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5592bae6f2f0 0x5592bb2513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 >> 172.21.15.92:0/1993575057 conn(0x5592bae8c300 msgr2=0x5592bb247920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 shutdown_connections 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 1 -- 172.21.15.92:0/1993575057 wait complete. 2024-09-21T02:08:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa6672bf740 20 client.16575 trim_cache size 0 max 16384 2024-09-21T02:08:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:08:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:08:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:08:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6440b6510 msgr2=0x7fa6440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6440b6510 0x7fa6440b8a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa654005fd0 tx=0x7fa65400d300 comp rx=0 tx=0).stop 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 msgr2=0x7fa65819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65819b100 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097b0 tx=0x7fa650003c20 comp rx=0 tx=0).stop 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 shutdown_connections 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa6440b6510 0x7fa6440b8a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa658108300 0x7fa65819b100 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6581056e0 0x7fa65819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 --2- 172.21.15.92:0/3190839836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa658102c20 0x7fa658199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.513+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 >> 172.21.15.92:0/3190839836 conn(0x7fa6580fd6d0 msgr2=0x7fa658103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.514+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 shutdown_connections 2024-09-21T02:08:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:25.514+0000 7fa660358640 1 -- 172.21.15.92:0/3190839836 wait complete. 2024-09-21T02:08:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:26 smithi136 ceph-mon[34754]: pgmap v464: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-09-21T02:08:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:26 smithi092 ceph-mon[31190]: pgmap v464: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-09-21T02:08:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:26 smithi099 ceph-mon[34404]: pgmap v464: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 20 op/s 2024-09-21T02:08:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:28 smithi136 ceph-mon[34754]: pgmap v465: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:08:29.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:28 smithi092 ceph-mon[31190]: pgmap v465: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:08:29.338 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:28 smithi099 ceph-mon[34404]: pgmap v465: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:08:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:30 smithi136 ceph-mon[34754]: pgmap v466: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:08:31.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:30 smithi092 ceph-mon[31190]: pgmap v466: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:08:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:30 smithi099 ceph-mon[34404]: pgmap v466: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:08:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:32 smithi136 ceph-mon[34754]: pgmap v467: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:08:33.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:32 smithi092 ceph-mon[31190]: pgmap v467: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:08:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:32 smithi099 ceph-mon[34404]: pgmap v467: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:08:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:34 smithi136 ceph-mon[34754]: pgmap v468: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:08:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:34 smithi092 ceph-mon[31190]: pgmap v468: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:08:35.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:34 smithi099 ceph-mon[34404]: pgmap v468: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:08:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:36 smithi136 ceph-mon[34754]: pgmap v469: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:08:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:36 smithi092 ceph-mon[31190]: pgmap v469: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:08:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:36 smithi099 ceph-mon[34404]: pgmap v469: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:08:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 -- 172.21.15.92:0/401590281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 msgr2=0x7fd4a8102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/401590281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a8102a30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009900 tx=0x7fd4a402a410 comp rx=0 tx=0).stop 2024-09-21T02:08:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 -- 172.21.15.92:0/401590281 shutdown_connections 2024-09-21T02:08:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/401590281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8103fe0 0x7fd4a810a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/401590281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103640 0x7fd4a8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/401590281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a8102a30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.618+0000 7fd4af2cc640 1 -- 172.21.15.92:0/401590281 >> 172.21.15.92:0/401590281 conn(0x7fd4a80fdf00 msgr2=0x7fd4a81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 -- 172.21.15.92:0/401590281 shutdown_connections 2024-09-21T02:08:38.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 -- 172.21.15.92:0/401590281 wait complete. 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 Processor -- start 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 -- start start 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a806c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8103640 0x7fd4a819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103fe0 0x7fd4a81a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810e200 con 0x7fd4a81025c0 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810e080 con 0x7fd4a8103640 2024-09-21T02:08:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.619+0000 7fd4af2cc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810e380 con 0x7fd4a8103fe0 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43130/0 (socket says 172.21.15.92:43130) 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 -- 172.21.15.92:0/1967332460 learned_addr learned my addr 172.21.15.92:0/1967332460 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad869640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103fe0 0x7fd4a81a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ac867640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8103640 0x7fd4a819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 -- 172.21.15.92:0/1967332460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103fe0 msgr2=0x7fd4a81a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103fe0 0x7fd4a81a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 -- 172.21.15.92:0/1967332460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8103640 msgr2=0x7fd4a819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8103640 0x7fd4a819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 -- 172.21.15.92:0/1967332460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a81a1a10 con 0x7fd4a81025c0 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad869640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103fe0 0x7fd4a81a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4ad068640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a806c210 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006010 tx=0x7fd4a4032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd49e7fc640 1 -- 172.21.15.92:0/1967332460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a403f030 con 0x7fd4a81025c0 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a81a1ca0 con 0x7fd4a81025c0 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd49e7fc640 1 -- 172.21.15.92:0/1967332460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a4003aa0 con 0x7fd4a81025c0 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd49e7fc640 1 -- 172.21.15.92:0/1967332460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4035070 con 0x7fd4a81025c0 2024-09-21T02:08:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.620+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a81a2190 con 0x7fd4a81025c0 2024-09-21T02:08:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 Processor -- start 2024-09-21T02:08:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 -- start start 2024-09-21T02:08:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd49e7fc640 1 -- 172.21.15.92:0/1967332460 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd4a4042030 con 0x7fd4a81025c0 2024-09-21T02:08:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 0x55d27cc94e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc96010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d27cc076f0 con 0x55d27c8ed1b0 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d27cbf6b20 con 0x55d27c8cc4d0 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4b6238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d27cbfa930 con 0x55d27c8c42f0 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.623+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 0x55d27cc94e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 0x55d27cc94e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43146/0 (socket says 172.21.15.92:43146) 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 -- 172.21.15.92:0/444494639 learned_addr learned my addr 172.21.15.92:0/444494639 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad068640 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ac867640 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc96010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 -- 172.21.15.92:0/444494639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 msgr2=0x55d27cc954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 -- 172.21.15.92:0/444494639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 msgr2=0x55d27cc96010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc96010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 -- 172.21.15.92:0/444494639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27cbbc530 con 0x55d27c8ed1b0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd49e7fc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4940b6510 0x7fd4940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad869640 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 0x55d27cc94e50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009fb0 tx=0x7fd4a0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7a1387074bd2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4ad068640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4940b6510 0x7fd4940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4937fe640 1 -- 172.21.15.92:0/444494639 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0016070 con 0x55d27c8ed1b0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4937fe640 1 -- 172.21.15.92:0/444494639 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a0012bc0 con 0x55d27c8ed1b0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd4937fe640 1 -- 172.21.15.92:0/444494639 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a001bc50 con 0x55d27c8ed1b0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.624+0000 7fd49e7fc640 1 -- 172.21.15.92:0/1967332460 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd4a41079c0 con 0x7fd4a81025c0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 -- 172.21.15.92:0/444494639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 msgr2=0x55d27cc94e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4ad068640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4940b6510 0x7fd4940b8a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4004770 tx=0x7fd4a410d550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 0x55d27cc94e50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009fb0 tx=0x7fd4a0010800 comp rx=0 tx=0).stop 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 -- 172.21.15.92:0/444494639 shutdown_connections 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc96010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 --2- 172.21.15.92:0/444494639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8ed1b0 0x55d27cc94e50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 -- 172.21.15.92:0/444494639 >> 172.21.15.92:0/444494639 conn(0x55d27c8e1300 msgr2=0x55d27cc908e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 -- 172.21.15.92:0/444494639 shutdown_connections 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.625+0000 7fd4b6238740 1 -- 172.21.15.92:0/444494639 wait complete. 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 -- start start 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 Processor -- start 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 0x55d27cc9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8ed1b0 0x55d27cc9f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d27cb2eab0 con 0x55d27c8c42f0 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d27cb9c730 con 0x55d27c8cc4d0 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d27cc96b30 con 0x55d27c8ed1b0 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4b6238740 10 client.0 _ll_register_callbacks cb 0x55d27cbf06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 0x55d27cc9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 0x55d27cc9e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43154/0 (socket says 172.21.15.92:43154) 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.626+0000 7fd4ad869640 1 -- 172.21.15.92:0/2394684972 learned_addr learned my addr 172.21.15.92:0/2394684972 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ac867640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8ed1b0 0x55d27cc9f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 -- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8ed1b0 msgr2=0x55d27cc9f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8ed1b0 0x55d27cc9f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 -- 172.21.15.92:0/2394684972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 msgr2=0x55d27cc9eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 -- 172.21.15.92:0/2394684972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27c958790 con 0x55d27c8c42f0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4ad869640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 0x55d27cc9e5e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000a010 tx=0x7fd4a001b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb1ba47fbb61658 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd490ff9640 1 -- 172.21.15.92:0/2394684972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0016070 con 0x55d27c8c42f0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd490ff9640 1 -- 172.21.15.92:0/2394684972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd4a0004c50 con 0x55d27c8c42f0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a0012bc0 con 0x55d27c8c42f0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd490ff9640 1 -- 172.21.15.92:0/2394684972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4a0004e50 con 0x55d27c8c42f0 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4b6238740 10 client.16587 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d27c9d1620 con 0x55d27c8c42f0 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.627+0000 7fd490ff9640 1 -- 172.21.15.92:0/2394684972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a0012bc0 con 0x55d27c8c42f0 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd490ff9640 1 -- 172.21.15.92:0/2394684972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd4a000f040 con 0x55d27c8c42f0 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4b6238740 10 client.16587 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4b6238740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4b6238740 20 client.16587 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4b6238740 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8abd80 0x55d27ccaba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4b6238740 4 client.16587 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27cc09640 con 0x55d27c8abd80 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.628+0000 7fd4ac867640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8abd80 0x55d27ccaba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd490ff9640 10 client.16587 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd4ac867640 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8abd80 0x55d27ccaba00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac8358eb947af4b8 server_cookie=5569659f1f791b2b in_seq=0 out_seq=0 2024-09-21T02:08:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd490ff9640 1 -- 172.21.15.92:0/2394684972 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d27cc09640 con 0x55d27c8abd80 2024-09-21T02:08:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd490ff9640 10 client.16587 handle_command_reply: tid=0 2024-09-21T02:08:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd4b6238740 1 client.16587 shutdown 2024-09-21T02:08:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 msgr2=0x55d27cc9e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd4b6238740 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 0x55d27cc9e5e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000a010 tx=0x7fd4a001b460 comp rx=0 tx=0).stop 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 shutdown_connections 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.629+0000 7fd4b6238740 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8abd80 0x55d27ccaba00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8ed1b0 0x55d27cc9f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd490ff9640 0 client.16587 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 1 --2- 172.21.15.92:0/2394684972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8c42f0 0x55d27cc9e5e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 >> 172.21.15.92:0/2394684972 conn(0x55d27c8e1300 msgr2=0x55d27cc8eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 shutdown_connections 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 1 -- 172.21.15.92:0/2394684972 wait complete. 2024-09-21T02:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.630+0000 7fd4b6238740 20 client.16587 trim_cache size 0 max 16384 2024-09-21T02:08:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.644+0000 7fd4b6238740 1 Processor -- start 2024-09-21T02:08:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.644+0000 7fd4b6238740 1 -- start start 2024-09-21T02:08:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.644+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 0x55d27cc95500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.644+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc9f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4b6238740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a81aafd0 con 0x55d27c8abd80 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4b6238740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a81acb10 con 0x55d27c8cc4d0 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4b6238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a81ab6a0 con 0x55d27c8c42f0 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 0x55d27cc95500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 0x55d27cc95500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43156/0 (socket says 172.21.15.92:43156) 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 -- 172.21.15.92:0/3247007261 learned_addr learned my addr 172.21.15.92:0/3247007261 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ac867640 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc9f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 -- 172.21.15.92:0/3247007261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 msgr2=0x55d27cc9f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc9f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 -- 172.21.15.92:0/3247007261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 msgr2=0x55d27cc9fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 -- 172.21.15.92:0/3247007261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27cca6ab0 con 0x55d27c8abd80 2024-09-21T02:08:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 0x55d27cc95500 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009fe0 tx=0x7fd4a00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c71e3a520807eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd492ffd640 1 -- 172.21.15.92:0/3247007261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0036040 con 0x55d27c8abd80 2024-09-21T02:08:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd492ffd640 1 -- 172.21.15.92:0/3247007261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a0005800 con 0x55d27c8abd80 2024-09-21T02:08:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.645+0000 7fd492ffd640 1 -- 172.21.15.92:0/3247007261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a001bb40 con 0x55d27c8abd80 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 -- 172.21.15.92:0/3247007261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 msgr2=0x55d27cc95500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 0x55d27cc95500 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009fe0 tx=0x7fd4a00021d0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 -- 172.21.15.92:0/3247007261 shutdown_connections 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8cc4d0 0x55d27cc9fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8c42f0 0x55d27cc9f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3247007261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8abd80 0x55d27cc95500 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 -- 172.21.15.92:0/3247007261 >> 172.21.15.92:0/3247007261 conn(0x55d27c8e1300 msgr2=0x55d27cc94df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 -- 172.21.15.92:0/3247007261 shutdown_connections 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 -- 172.21.15.92:0/3247007261 wait complete. 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.646+0000 7fd4b6238740 1 -- start start 2024-09-21T02:08:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 Processor -- start 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8abd80 0x55d27cca3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8c42f0 0x55d27cca3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 0x55d27cca46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a81ac440 con 0x55d27c8cc4d0 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a81abd70 con 0x55d27c8c42f0 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a81a8fb0 con 0x55d27c8abd80 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4b6238740 10 client.0 _ll_register_callbacks cb 0x7fd4a410d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 0x55d27cca46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 0x55d27cca46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43172/0 (socket says 172.21.15.92:43172) 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 -- 172.21.15.92:0/3377293359 learned_addr learned my addr 172.21.15.92:0/3377293359 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ac867640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8c42f0 0x55d27cca3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8abd80 0x55d27cca3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 -- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8abd80 msgr2=0x55d27cca3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8abd80 0x55d27cca3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 -- 172.21.15.92:0/3377293359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8c42f0 msgr2=0x55d27cca3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8c42f0 0x55d27cca3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ad869640 1 -- 172.21.15.92:0/3377293359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27cca4e20 con 0x55d27c8cc4d0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.647+0000 7fd4ac867640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8c42f0 0x55d27cca3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4ad869640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 0x55d27cca46c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0005190 tx=0x7fd4a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0da21ac1e4c787f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd490ff9640 1 -- 172.21.15.92:0/3377293359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a001b460 con 0x55d27c8cc4d0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a0005800 con 0x55d27c8cc4d0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd490ff9640 1 -- 172.21.15.92:0/3377293359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd4a001b920 con 0x55d27c8cc4d0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd490ff9640 1 -- 172.21.15.92:0/3377293359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4a00059b0 con 0x55d27c8cc4d0 2024-09-21T02:08:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 10 client.16593 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d27cca6110 con 0x55d27c8cc4d0 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd490ff9640 1 -- 172.21.15.92:0/3377293359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a0005800 con 0x55d27c8cc4d0 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd490ff9640 1 -- 172.21.15.92:0/3377293359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd4a003c020 con 0x55d27c8cc4d0 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 10 client.16593 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 20 client.16593 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8ed1b0 0x55d27cd18780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 4 client.16593 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.648+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d27cc050f0 con 0x55d27c8ed1b0 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.649+0000 7fd4ac867640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8ed1b0 0x55d27cd18780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.649+0000 7fd490ff9640 10 client.16593 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.649+0000 7fd4ac867640 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8ed1b0 0x55d27cd18780 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd19572f411a9f server_cookie=7d3fedb46b1fdc65 in_seq=0 out_seq=0 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd490ff9640 1 -- 172.21.15.92:0/3377293359 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d27cc050f0 con 0x55d27c8ed1b0 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd490ff9640 10 client.16593 handle_command_reply: tid=0 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 client.16593 shutdown 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 msgr2=0x55d27cca46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 0x55d27cca46c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0005190 tx=0x7fd4a0005f90 comp rx=0 tx=0).stop 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 shutdown_connections 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d27c8ed1b0 0x55d27cd18780 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d27c8cc4d0 0x55d27cca46c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d27c8c42f0 0x55d27cca3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd490ff9640 0 client.16593 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 --2- 172.21.15.92:0/3377293359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d27c8abd80 0x55d27cca3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 >> 172.21.15.92:0/3377293359 conn(0x55d27c8e1300 msgr2=0x55d27cc9f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 shutdown_connections 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 1 -- 172.21.15.92:0/3377293359 wait complete. 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.663+0000 7fd4b6238740 20 client.16593 trim_cache size 0 max 16384 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:08:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:08:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4940b6510 msgr2=0x7fd4940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4940b6510 0x7fd4940b8a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4004770 tx=0x7fd4a410d550 comp rx=0 tx=0).stop 2024-09-21T02:08:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 msgr2=0x7fd4a806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a806c210 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006010 tx=0x7fd4a4032f90 comp rx=0 tx=0).stop 2024-09-21T02:08:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 shutdown_connections 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd4940b6510 0x7fd4940b8a20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a8103fe0 0x7fd4a81a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8103640 0x7fd4a819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 --2- 172.21.15.92:0/1967332460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a81025c0 0x7fd4a806c210 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 >> 172.21.15.92:0/1967332460 conn(0x7fd4a80fdf00 msgr2=0x7fd4a8106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 shutdown_connections 2024-09-21T02:08:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:38.664+0000 7fd4af2cc640 1 -- 172.21.15.92:0/1967332460 wait complete. 2024-09-21T02:08:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:38 smithi136 ceph-mon[34754]: pgmap v470: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:38 smithi092 ceph-mon[31190]: pgmap v470: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:38 smithi099 ceph-mon[34404]: pgmap v470: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:41.170 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.170 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.170 INFO:tasks.workunit.client.0.smithi092.stdout:127/600: open fd = 3 2024-09-21T02:08:41.170 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.170 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.170 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:128/600: open fd = 3 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:129/600: open fd = 3 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:130/600: open fd = 3 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:131/600: open fd = 3 2024-09-21T02:08:41.171 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:132/600: open fd = 3 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:133/600: open fd = 3 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:134/600: open fd = 3 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.172 INFO:tasks.workunit.client.0.smithi092.stdout:135/600: open fd = 3 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:136/600: open fd = 3 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:137/600: open fd = 3 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:138/600: open fd = 3 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.173 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:139/600: open fd = 3 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:140/600: open fd = 3 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:141/600: open fd = 3 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:142/600: open fd = 3 2024-09-21T02:08:41.174 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:143/600: open fd = 3 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:144/600: open fd = 3 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:145/600: open fd = 3 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.175 INFO:tasks.workunit.client.0.smithi092.stdout:146/600: open fd = 3 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:147/600: open fd = 3 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:148/600: open fd = 3 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:149/600: open fd = 3 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.176 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:150/600: open fd = 3 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:151/600: open fd = 3 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:152/600: open fd = 3 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:153/600: open fd = 3 2024-09-21T02:08:41.177 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:154/600: open fd = 3 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:155/600: open fd = 3 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:156/600: open fd = 3 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.178 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:157/600: open fd = 3 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:158/600: open fd = 3 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:159/600: open fd = 3 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:160/600: open fd = 3 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.179 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:161/600: open fd = 3 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:162/600: open fd = 3 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:163/600: open fd = 3 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:164/600: open fd = 3 2024-09-21T02:08:41.180 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:165/600: open fd = 3 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:166/600: open fd = 3 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:167/600: open fd = 3 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.181 INFO:tasks.workunit.client.0.smithi092.stdout:168/600: open fd = 3 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:169/600: open fd = 3 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:170/600: open fd = 3 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:171/600: open fd = 3 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.182 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:172/600: open fd = 3 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:173/600: open fd = 3 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:174/600: open fd = 3 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.183 INFO:tasks.workunit.client.0.smithi092.stdout:175/600: open fd = 3 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:176/600: open fd = 3 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:177/600: open fd = 3 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:178/600: open fd = 3 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.184 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:179/600: open fd = 3 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:180/600: open fd = 3 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:181/600: open fd = 3 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:182/600: open fd = 3 2024-09-21T02:08:41.185 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:183/600: open fd = 3 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:184/600: open fd = 3 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:185/600: open fd = 3 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.186 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:186/600: open fd = 3 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:187/600: open fd = 3 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:188/600: open fd = 3 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.187 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:08:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:40 smithi136 ceph-mon[34754]: pgmap v471: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-21T02:08:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:40 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:08:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:40 smithi092 ceph-mon[31190]: pgmap v471: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-21T02:08:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:40 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:08:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:40 smithi099 ceph-mon[34404]: pgmap v471: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-21T02:08:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:40 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:08:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:08:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:08:42.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:41 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:08:42.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:08:42.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:08:42.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:41 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:08:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:08:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:08:42.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:41 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:08:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:42 smithi136 ceph-mon[34754]: pgmap v472: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:08:43.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:42 smithi092 ceph-mon[31190]: pgmap v472: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:08:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:42 smithi099 ceph-mon[34404]: pgmap v472: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:08:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:44 smithi136 ceph-mon[34754]: pgmap v473: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:08:45.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:44 smithi092 ceph-mon[31190]: pgmap v473: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:08:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:44 smithi099 ceph-mon[34404]: pgmap v473: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:08:47.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:46 smithi136 ceph-mon[34754]: pgmap v474: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:47.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:46 smithi092 ceph-mon[31190]: pgmap v474: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:47.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:46 smithi099 ceph-mon[34404]: pgmap v474: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:48.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:08:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 -- 172.21.15.92:0/3732365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 msgr2=0x7f7d9410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/3732365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9410f5a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009ca0 tx=0x7f7d8c02a8e0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 -- 172.21.15.92:0/3732365382 shutdown_connections 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/3732365382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9410f5a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/3732365382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94108420 0x7f7d94108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/3732365382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94102ae0 0x7f7d94102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 -- 172.21.15.92:0/3732365382 >> 172.21.15.92:0/3732365382 conn(0x7f7d94079770 msgr2=0x7f7d94079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 -- 172.21.15.92:0/3732365382 shutdown_connections 2024-09-21T02:08:49.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 -- 172.21.15.92:0/3732365382 wait complete. 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 Processor -- start 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.055+0000 7f7d9bec3640 1 -- start start 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9bec3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94102ae0 0x7f7d9419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9bec3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94108420 0x7f7d9419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9bec3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9bec3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94112a90 con 0x7f7d94108d70 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9bec3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94112910 con 0x7f7d94102ae0 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9bec3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94112c10 con 0x7f7d94108420 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58446/0 (socket says 172.21.15.92:58446) 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 -- 172.21.15.92:0/4136081674 learned_addr learned my addr 172.21.15.92:0/4136081674 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94102ae0 0x7f7d9419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9945e640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94108420 0x7f7d9419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 -- 172.21.15.92:0/4136081674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94108420 msgr2=0x7f7d9419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94108420 0x7f7d9419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 -- 172.21.15.92:0/4136081674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94102ae0 msgr2=0x7f7d9419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94102ae0 0x7f7d9419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 -- 172.21.15.92:0/4136081674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d941a6590 con 0x7f7d94108d70 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94102ae0 0x7f7d9419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.056+0000 7f7d9a460640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9419bd80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005cc0 tx=0x7f7d8c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.057+0000 7f7d8affd640 1 -- 172.21.15.92:0/4136081674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c004040 con 0x7f7d94108d70 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.057+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d941a6820 con 0x7f7d94108d70 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.057+0000 7f7d8affd640 1 -- 172.21.15.92:0/4136081674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c004690 con 0x7f7d94108d70 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.057+0000 7f7d8affd640 1 -- 172.21.15.92:0/4136081674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c030e20 con 0x7f7d94108d70 2024-09-21T02:08:49.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.057+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d941a6d60 con 0x7f7d94108d70 2024-09-21T02:08:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.059+0000 7f7da2437740 1 Processor -- start 2024-09-21T02:08:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.059+0000 7f7d8affd640 1 -- 172.21.15.92:0/4136081674 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f7d8c0305e0 con 0x7f7d94108d70 2024-09-21T02:08:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 -- start start 2024-09-21T02:08:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2da4070 0x564ba314b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d7b1b0 0x564ba314b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba314bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564ba30c1e50 con 0x564ba2d83390 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564ba30b1280 con 0x564ba2d7b1b0 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7da2437740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ba30b5090 con 0x564ba2da4070 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7d9945e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba314bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7d9945e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba314bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58456/0 (socket says 172.21.15.92:58456) 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7d9945e640 1 -- 172.21.15.92:0/2978286587 learned_addr learned my addr 172.21.15.92:0/2978286587 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7d8affd640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7d800b6540 0x7f7d800b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7d9945e640 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2da4070 0x564ba314b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.060+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d7b1b0 0x564ba314b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9945e640 1 -- 172.21.15.92:0/2978286587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2da4070 msgr2=0x564ba314b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9945e640 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2da4070 0x564ba314b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9945e640 1 -- 172.21.15.92:0/2978286587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d7b1b0 msgr2=0x564ba314b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9945e640 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d7b1b0 0x564ba314b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9945e640 1 -- 172.21.15.92:0/2978286587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba2e0f650 con 0x564ba2d83390 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9a460640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7d800b6540 0x7f7d800b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d7b1b0 0x564ba314b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d8affd640 1 -- 172.21.15.92:0/4136081674 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7d8c108760 con 0x7f7d94108d70 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d9945e640 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba314bcf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400ad20 tx=0x7f7d8400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69788ebe2de6315a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d7ffff640 1 -- 172.21.15.92:0/2978286587 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8401c070 con 0x564ba2d83390 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d7ffff640 1 -- 172.21.15.92:0/2978286587 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8400fdf0 con 0x564ba2d83390 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.061+0000 7f7d7ffff640 1 -- 172.21.15.92:0/2978286587 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84014650 con 0x564ba2d83390 2024-09-21T02:08:49.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 -- 172.21.15.92:0/2978286587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 msgr2=0x564ba314bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7d9a460640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7d800b6540 0x7f7d800b8a50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c037e90 tx=0x7f7d8c02a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba314bcf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400ad20 tx=0x7f7d8400a430 comp rx=0 tx=0).stop 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 -- 172.21.15.92:0/2978286587 shutdown_connections 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba314bcf0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d7b1b0 0x564ba314b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 --2- 172.21.15.92:0/2978286587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2da4070 0x564ba314b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 -- 172.21.15.92:0/2978286587 >> 172.21.15.92:0/2978286587 conn(0x564ba2d98460 msgr2=0x564ba3149160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 -- 172.21.15.92:0/2978286587 shutdown_connections 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.062+0000 7f7da2437740 1 -- 172.21.15.92:0/2978286587 wait complete. 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 -- start start 2024-09-21T02:08:49.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 Processor -- start 2024-09-21T02:08:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba3156f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2da4070 0x564ba3162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564ba2f12700 con 0x564ba2d83390 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564ba304c110 con 0x564ba2da4070 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ba3159a50 con 0x564ba2d7b1b0 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7da2437740 10 client.0 _ll_register_callbacks cb 0x564ba3065990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7d99c5f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba3156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7d99c5f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba3156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58470/0 (socket says 172.21.15.92:58470) 2024-09-21T02:08:49.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7d99c5f640 1 -- 172.21.15.92:0/3391749569 learned_addr learned my addr 172.21.15.92:0/3391749569 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7d9945e640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.063+0000 7f7d9945e640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2da4070 0x564ba3162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d99c5f640 1 -- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 msgr2=0x564ba3156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d99c5f640 1 -- 172.21.15.92:0/3391749569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2da4070 msgr2=0x564ba3162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2da4070 0x564ba3162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d99c5f640 1 -- 172.21.15.92:0/3391749569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba3079c40 con 0x564ba2d83390 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d9945e640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2da4070 0x564ba3162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba3156f40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7d90010840 tx=0x7f7d90010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd0c97f81783431 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/3391749569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000ed40 con 0x564ba2d83390 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/3391749569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7d90004020 con 0x564ba2d83390 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8400fdf0 con 0x564ba2d83390 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/3391749569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d90004260 con 0x564ba2d83390 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7da2437740 10 client.16605 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ba3165890 con 0x564ba2d83390 2024-09-21T02:08:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.064+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/3391749569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8400fdf0 con 0x564ba2d83390 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/3391749569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d90011440 con 0x564ba2d83390 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7da2437740 10 client.16605 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7da2437740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7da2437740 20 client.16605 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7da2437740 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d629f0 0x564ba3168de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7da2437740 4 client.16605 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:49.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ba30c3da0 con 0x564ba2d629f0 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.065+0000 7f7d9945e640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d629f0 0x564ba3168de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.066+0000 7f7d7d7fa640 10 client.16605 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.066+0000 7f7d9945e640 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d629f0 0x564ba3168de0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5a7884a7d606ee server_cookie=8b5c20ec6d7ce45b in_seq=0 out_seq=0 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.066+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/3391749569 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ba30c3da0 con 0x564ba2d629f0 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7d7d7fa640 10 client.16605 handle_command_reply: tid=0 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 client.16605 shutdown 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 msgr2=0x564ba3156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba3156f40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7d90010840 tx=0x7f7d90010d00 comp rx=0 tx=0).stop 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 shutdown_connections 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d629f0 0x564ba3168de0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2da4070 0x564ba3162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2d83390 0x564ba3156f40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 --2- 172.21.15.92:0/3391749569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7d7d7fa640 0 client.16605 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 >> 172.21.15.92:0/3391749569 conn(0x564ba2d98460 msgr2=0x564ba314e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 shutdown_connections 2024-09-21T02:08:49.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 1 -- 172.21.15.92:0/3391749569 wait complete. 2024-09-21T02:08:49.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.067+0000 7f7da2437740 20 client.16605 trim_cache size 0 max 16384 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 Processor -- start 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 -- start start 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba3162c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba3163180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941af420 con 0x564ba2da4070 2024-09-21T02:08:49.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941b0f60 con 0x564ba2d83390 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.084+0000 7f7da2437740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941afaf0 con 0x564ba2d7b1b0 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba3163180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba3163180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58486/0 (socket says 172.21.15.92:58486) 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 -- 172.21.15.92:0/1288217093 learned_addr learned my addr 172.21.15.92:0/1288217093 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba3162c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 -- 172.21.15.92:0/1288217093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 msgr2=0x564ba3156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 -- 172.21.15.92:0/1288217093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 msgr2=0x564ba3162c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba3162c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 -- 172.21.15.92:0/1288217093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba314d560 con 0x564ba2da4070 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba3162c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:49.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d9945e640 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba3163180 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400afd0 tx=0x7f7d840047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d682aca0e03c6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d7f7fe640 1 -- 172.21.15.92:0/1288217093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8401c050 con 0x564ba2da4070 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d7f7fe640 1 -- 172.21.15.92:0/1288217093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8401d040 con 0x564ba2da4070 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.085+0000 7f7d7f7fe640 1 -- 172.21.15.92:0/1288217093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8402eaa0 con 0x564ba2da4070 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 -- 172.21.15.92:0/1288217093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 msgr2=0x564ba3163180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba3163180 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400afd0 tx=0x7f7d840047d0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 -- 172.21.15.92:0/1288217093 shutdown_connections 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba3163180 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba3162c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 --2- 172.21.15.92:0/1288217093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba3156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 -- 172.21.15.92:0/1288217093 >> 172.21.15.92:0/1288217093 conn(0x564ba2d98460 msgr2=0x564ba31567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 -- 172.21.15.92:0/1288217093 shutdown_connections 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 -- 172.21.15.92:0/1288217093 wait complete. 2024-09-21T02:08:49.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.086+0000 7f7da2437740 1 -- start start 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 Processor -- start 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba315ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba315b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba315b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941b0890 con 0x564ba2da4070 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941b01c0 con 0x564ba2d83390 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d941ad400 con 0x564ba2d7b1b0 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7da2437740 10 client.0 _ll_register_callbacks cb 0x7f7d9000c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba315b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba315b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58498/0 (socket says 172.21.15.92:58498) 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 -- 172.21.15.92:0/204815856 learned_addr learned my addr 172.21.15.92:0/204815856 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba315b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba315ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 -- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 msgr2=0x564ba315ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba315ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 -- 172.21.15.92:0/204815856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 msgr2=0x564ba315b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba315b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.087+0000 7f7d9945e640 1 -- 172.21.15.92:0/204815856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba31d2840 con 0x564ba2da4070 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d9945e640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba315b7e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400ff50 tx=0x7f7d84032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761782bcaab44303 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/204815856 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8401c040 con 0x564ba2da4070 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/204815856 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7d8402e920 con 0x564ba2da4070 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8401d040 con 0x564ba2da4070 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/204815856 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d8402eb20 con 0x564ba2da4070 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 10 client.16611 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:49.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ba31d3a20 con 0x564ba2da4070 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/204815856 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8401d040 con 0x564ba2da4070 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/204815856 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d84015040 con 0x564ba2da4070 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 10 client.16611 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 20 client.16611 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d37da0 0x564ba31d5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 4 client.16611 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.088+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ba30bf850 con 0x564ba2d37da0 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.089+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d37da0 0x564ba31d5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.089+0000 7f7d7d7fa640 10 client.16611 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.089+0000 7f7d99c5f640 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d37da0 0x564ba31d5e60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5d93b353a2e465 server_cookie=fc163e04402e0f84 in_seq=0 out_seq=0 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.089+0000 7f7d7d7fa640 1 -- 172.21.15.92:0/204815856 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x564ba30bf850 con 0x564ba2d37da0 2024-09-21T02:08:49.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.089+0000 7f7d7d7fa640 10 client.16611 handle_command_reply: tid=0 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.089+0000 7f7da2437740 1 client.16611 shutdown 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 msgr2=0x564ba315b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba315b7e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400ff50 tx=0x7f7d84032040 comp rx=0 tx=0).stop 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 shutdown_connections 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564ba2d37da0 0x564ba31d5e60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ba2da4070 0x564ba315b7e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba2d83390 0x564ba315b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 --2- 172.21.15.92:0/204815856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ba2d7b1b0 0x564ba315ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7d7d7fa640 0 client.16611 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 >> 172.21.15.92:0/204815856 conn(0x564ba2d98460 msgr2=0x564ba314e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 shutdown_connections 2024-09-21T02:08:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 1 -- 172.21.15.92:0/204815856 wait complete. 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.090+0000 7f7da2437740 20 client.16611 trim_cache size 0 max 16384 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:08:49.101 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:49.102 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:49.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:08:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:08:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:08:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:08:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:08:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:08:49.104 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:08:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:08:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:49.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:08:49.105 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:08:49.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7d800b6540 msgr2=0x7f7d800b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7d800b6540 0x7f7d800b8a50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c037e90 tx=0x7f7d8c02a8e0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 msgr2=0x7f7d9419bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9419bd80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005cc0 tx=0x7f7d8c003ba0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 shutdown_connections 2024-09-21T02:08:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7d800b6540 0x7f7d800b8a50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94108d70 0x7f7d9419bd80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94108420 0x7f7d9419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 --2- 172.21.15.92:0/4136081674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d94102ae0 0x7f7d9419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 >> 172.21.15.92:0/4136081674 conn(0x7f7d94079770 msgr2=0x7f7d941043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 shutdown_connections 2024-09-21T02:08:49.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:49.091+0000 7f7d9bec3640 1 -- 172.21.15.92:0/4136081674 wait complete. 2024-09-21T02:08:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:48 smithi136 ceph-mon[34754]: pgmap v475: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:08:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:48 smithi092 ceph-mon[31190]: pgmap v475: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:08:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:48 smithi099 ceph-mon[34404]: pgmap v475: 97 pgs: 97 active+clean; 3.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:08:49.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:08:49.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:08:50.109 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 -- 172.21.15.92:0/1287942067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 msgr2=0x7f86b0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 --2- 172.21.15.92:0/1287942067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b0102ef0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009900 tx=0x7f86a002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 -- 172.21.15.92:0/1287942067 shutdown_connections 2024-09-21T02:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 --2- 172.21.15.92:0/1287942067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b0108d70 0x7f86b010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 --2- 172.21.15.92:0/1287942067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b0108420 0x7f86b0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 --2- 172.21.15.92:0/1287942067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b0102ef0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 -- 172.21.15.92:0/1287942067 >> 172.21.15.92:0/1287942067 conn(0x7f86b0079770 msgr2=0x7f86b0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 -- 172.21.15.92:0/1287942067 shutdown_connections 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.600+0000 7f86b49f2640 1 -- 172.21.15.92:0/1287942067 wait complete. 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 Processor -- start 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 -- start start 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b0108420 0x7f86b019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b0108d70 0x7f86b019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0113b50 con 0x7f86b0102ae0 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01139d0 con 0x7f86b0108420 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86b49f2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0113cd0 con 0x7f86b0108d70 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86ae59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.601+0000 7f86ae59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58508/0 (socket says 172.21.15.92:58508) 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 -- 172.21.15.92:0/262141936 learned_addr learned my addr 172.21.15.92:0/262141936 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86a7fff640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b0108420 0x7f86b019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86aed9d640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b0108d70 0x7f86b019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 -- 172.21.15.92:0/262141936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b0108d70 msgr2=0x7f86b019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b0108d70 0x7f86b019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 -- 172.21.15.92:0/262141936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b0108420 msgr2=0x7f86b019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b0108420 0x7f86b019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 -- 172.21.15.92:0/262141936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b01a0510 con 0x7f86b0102ae0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86ae59c640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b019a5a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005940 tx=0x7f86a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a003f030 con 0x7f86b0102ae0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a0003cb0 con 0x7f86b0102ae0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b01a07a0 con 0x7f86b0102ae0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0035070 con 0x7f86b0102ae0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.602+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01a6f60 con 0x7f86b0102ae0 2024-09-21T02:08:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.603+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b0103d60 con 0x7f86b0102ae0 2024-09-21T02:08:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.605+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f86a0042030 con 0x7f86b0102ae0 2024-09-21T02:08:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.606+0000 7f86a77fe640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f869c0b6510 0x7f869c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.606+0000 7f86a7fff640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f869c0b6510 0x7f869c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.606+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86a0107d60 con 0x7f86b0102ae0 2024-09-21T02:08:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.606+0000 7f86a7fff640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f869c0b6510 0x7f869c0b8a20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f86b0105440 tx=0x7f869800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:50.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.609+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86a00d29b0 con 0x7f86b0102ae0 2024-09-21T02:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.832+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86b0102ef0 con 0x7f86b0102ae0 2024-09-21T02:08:50.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.832+0000 7f86a77fe640 1 -- 172.21.15.92:0/262141936 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f86a00d78f0 con 0x7f86b0102ae0 2024-09-21T02:08:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:08:50.835 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:08:50.835 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:08:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f869c0b6510 msgr2=0x7f869c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f869c0b6510 0x7f869c0b8a20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f86b0105440 tx=0x7f869800b390 comp rx=0 tx=0).stop 2024-09-21T02:08:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 msgr2=0x7f86b019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:50.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b019a5a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005940 tx=0x7f86a00042f0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 shutdown_connections 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f869c0b6510 0x7f869c0b8a20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b0108d70 0x7f86b019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b0108420 0x7f86b019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 --2- 172.21.15.92:0/262141936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b0102ae0 0x7f86b019a5a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.834+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 >> 172.21.15.92:0/262141936 conn(0x7f86b0079770 msgr2=0x7f86b0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.835+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 shutdown_connections 2024-09-21T02:08:50.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:50.835+0000 7f86b49f2640 1 -- 172.21.15.92:0/262141936 wait complete. 2024-09-21T02:08:51.175 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:51 smithi092 ceph-mon[31190]: pgmap v476: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:51.175 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:51 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/262141936' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:08:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:51 smithi136 ceph-mon[34754]: pgmap v476: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:51.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:51 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/262141936' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:08:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:51 smithi099 ceph-mon[34404]: pgmap v476: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:08:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:51 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/262141936' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:08:51.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:08:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 -- 172.21.15.92:0/1906982027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e01025a0 msgr2=0x7f53e0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 --2- 172.21.15.92:0/1906982027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e01025a0 0x7f53e0102a10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009900 tx=0x7f53d002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 -- 172.21.15.92:0/1906982027 shutdown_connections 2024-09-21T02:08:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 --2- 172.21.15.92:0/1906982027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e0103fc0 0x7f53e010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 --2- 172.21.15.92:0/1906982027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53e0103620 0x7f53e0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 --2- 172.21.15.92:0/1906982027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e01025a0 0x7f53e0102a10 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 -- 172.21.15.92:0/1906982027 >> 172.21.15.92:0/1906982027 conn(0x7f53e00fdee0 msgr2=0x7f53e0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 -- 172.21.15.92:0/1906982027 shutdown_connections 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.760+0000 7f53e757b640 1 -- 172.21.15.92:0/1906982027 wait complete. 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 Processor -- start 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 -- start start 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e01025a0 0x7f53e006b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 0x7f53e019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53e0103fc0 0x7f53e01a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53e010e1f0 con 0x7f53e0103620 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53e010e070 con 0x7f53e01025a0 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e757b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53e010e370 con 0x7f53e0103fc0 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e4b16640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 0x7f53e019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e4b16640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 0x7f53e019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58518/0 (socket says 172.21.15.92:58518) 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e4b16640 1 -- 172.21.15.92:0/3414666576 learned_addr learned my addr 172.21.15.92:0/3414666576 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.761+0000 7f53e5317640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e01025a0 0x7f53e006b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e5b18640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53e0103fc0 0x7f53e01a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e4b16640 1 -- 172.21.15.92:0/3414666576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53e0103fc0 msgr2=0x7f53e01a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e4b16640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53e0103fc0 0x7f53e01a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e4b16640 1 -- 172.21.15.92:0/3414666576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e01025a0 msgr2=0x7f53e006b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e4b16640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e01025a0 0x7f53e006b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e4b16640 1 -- 172.21.15.92:0/3414666576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e01a19e0 con 0x7f53e0103620 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e4b16640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 0x7f53e019ed20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00d770 tx=0x7f53dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53d67fc640 1 -- 172.21.15.92:0/3414666576 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc017070 con 0x7f53e0103620 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53d67fc640 1 -- 172.21.15.92:0/3414666576 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53dc0054d0 con 0x7f53e0103620 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53d67fc640 1 -- 172.21.15.92:0/3414666576 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004c30 con 0x7f53e0103620 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e01a1cd0 con 0x7f53e0103620 2024-09-21T02:08:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.762+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e01a21d0 con 0x7f53e0103620 2024-09-21T02:08:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 Processor -- start 2024-09-21T02:08:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 -- start start 2024-09-21T02:08:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53d67fc640 1 -- 172.21.15.92:0/3414666576 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f53dc004e10 con 0x7f53e0103620 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c14e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9b876f0 con 0x55eaf98442f0 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9b76b20 con 0x55eaf986d1b0 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9b7a930 con 0x55eaf984c4d0 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53e5317640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53e5317640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58532/0 (socket says 172.21.15.92:58532) 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.765+0000 7f53e5317640 1 -- 172.21.15.92:0/1939717380 learned_addr learned my addr 172.21.15.92:0/1939717380 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e4b16640 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5b18640 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c14e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5317640 1 -- 172.21.15.92:0/1939717380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 msgr2=0x55eaf9c16010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5317640 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c16010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5317640 1 -- 172.21.15.92:0/1939717380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 msgr2=0x55eaf9c14e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5317640 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c14e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5317640 1 -- 172.21.15.92:0/1939717380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf9b3c530 con 0x55eaf98442f0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53d67fc640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53cc0b6510 0x7f53cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5317640 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c154c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f53d002a930 tx=0x7f53d00316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb837aba5bd2ad20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53e5b18640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53cc0b6510 0x7f53cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53d67fc640 1 -- 172.21.15.92:0/3414666576 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f53dc0e6d60 con 0x7f53e0103620 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53cb7fe640 1 -- 172.21.15.92:0/1939717380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0035030 con 0x55eaf98442f0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53cb7fe640 1 -- 172.21.15.92:0/1939717380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d0040070 con 0x55eaf98442f0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.766+0000 7f53cb7fe640 1 -- 172.21.15.92:0/1939717380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0003a80 con 0x55eaf98442f0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 -- 172.21.15.92:0/1939717380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 msgr2=0x55eaf9c154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c154c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f53d002a930 tx=0x7f53d00316e0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53e5b18640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53cc0b6510 0x7f53cc0b8a20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f53d8005fd0 tx=0x7f53d8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 -- 172.21.15.92:0/1939717380 shutdown_connections 2024-09-21T02:08:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c16010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c154c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/1939717380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 -- 172.21.15.92:0/1939717380 >> 172.21.15.92:0/1939717380 conn(0x55eaf9861300 msgr2=0x55eaf9c108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.767+0000 7f53ee5c4740 1 -- 172.21.15.92:0/1939717380 shutdown_connections 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 -- 172.21.15.92:0/1939717380 wait complete. 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 -- start start 2024-09-21T02:08:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 Processor -- start 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9aaeab0 con 0x55eaf98442f0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9b1c730 con 0x55eaf986d1b0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eaf9c16ad0 con 0x55eaf984c4d0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53ee5c4740 10 client.0 _ll_register_callbacks cb 0x55eaf9b706b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53e5317640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.768+0000 7f53e5317640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58540/0 (socket says 172.21.15.92:58540) 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 -- 172.21.15.92:0/2799741422 learned_addr learned my addr 172.21.15.92:0/2799741422 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e4b16640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5b18640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 -- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 msgr2=0x55eaf9c1ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 -- 172.21.15.92:0/2799741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 msgr2=0x55eaf9c1f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 -- 172.21.15.92:0/2799741422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf98d8790 con 0x55eaf98442f0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e4b16640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5b18640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53e5317640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1e560 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f53d002a450 tx=0x7f53d0031d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79664c92f0c32043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53c8ff9640 1 -- 172.21.15.92:0/2799741422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0035070 con 0x55eaf98442f0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d0040070 con 0x55eaf98442f0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53c8ff9640 1 -- 172.21.15.92:0/2799741422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f53d0004450 con 0x55eaf98442f0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53c8ff9640 1 -- 172.21.15.92:0/2799741422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53d00318d0 con 0x55eaf98442f0 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53ee5c4740 10 client.16629 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.769+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eaf9951620 con 0x55eaf98442f0 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53c8ff9640 1 -- 172.21.15.92:0/2799741422 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d0040070 con 0x55eaf98442f0 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53c8ff9640 1 -- 172.21.15.92:0/2799741422 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f53d0045970 con 0x55eaf98442f0 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53ee5c4740 10 client.16629 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53ee5c4740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53ee5c4740 20 client.16629 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf982bd80 0x55eaf9c2b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53ee5c4740 4 client.16629 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaf9b89640 con 0x55eaf982bd80 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.770+0000 7f53e4b16640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf982bd80 0x55eaf9c2b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.771+0000 7f53c8ff9640 10 client.16629 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.771+0000 7f53e4b16640 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf982bd80 0x55eaf9c2b870 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f0d218d51febb8 server_cookie=873be184bbb8540f in_seq=0 out_seq=0 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.771+0000 7f53c8ff9640 1 -- 172.21.15.92:0/2799741422 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eaf9b89640 con 0x55eaf982bd80 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.771+0000 7f53c8ff9640 10 client.16629 handle_command_reply: tid=0 2024-09-21T02:08:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.771+0000 7f53ee5c4740 1 client.16629 shutdown 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 msgr2=0x55eaf9c1e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1e560 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f53d002a450 tx=0x7f53d0031d00 comp rx=0 tx=0).stop 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 shutdown_connections 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf982bd80 0x55eaf9c2b870 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53c8ff9640 0 client.16629 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/2799741422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1e560 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 >> 172.21.15.92:0/2799741422 conn(0x55eaf9861300 msgr2=0x55eaf9c0eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 shutdown_connections 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 1 -- 172.21.15.92:0/2799741422 wait complete. 2024-09-21T02:08:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.772+0000 7f53ee5c4740 20 client.16629 trim_cache size 0 max 16384 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 Processor -- start 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 -- start start 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01aaef0 con 0x55eaf98442f0 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01aca30 con 0x55eaf984c4d0 2024-09-21T02:08:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01ab5c0 con 0x55eaf986d1b0 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53e5317640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53e5317640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58542/0 (socket says 172.21.15.92:58542) 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.786+0000 7f53e5317640 1 -- 172.21.15.92:0/356109965 learned_addr learned my addr 172.21.15.92:0/356109965 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e4b16640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5b18640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5317640 1 -- 172.21.15.92:0/356109965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 msgr2=0x55eaf9c1cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5317640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5317640 1 -- 172.21.15.92:0/356109965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf984c4d0 msgr2=0x55eaf9c1cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5317640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5317640 1 -- 172.21.15.92:0/356109965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf9c1d530 con 0x55eaf98442f0 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e4b16640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5b18640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53e5317640 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1d760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f53d0031500 tx=0x7f53d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53904d30ff62512 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53caffd640 1 -- 172.21.15.92:0/356109965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0004450 con 0x55eaf98442f0 2024-09-21T02:08:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53caffd640 1 -- 172.21.15.92:0/356109965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d005d070 con 0x55eaf98442f0 2024-09-21T02:08:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.787+0000 7f53caffd640 1 -- 172.21.15.92:0/356109965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0003e10 con 0x55eaf98442f0 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 -- 172.21.15.92:0/356109965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 msgr2=0x55eaf9c1d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1d760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f53d0031500 tx=0x7f53d0005fd0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 -- 172.21.15.92:0/356109965 shutdown_connections 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c1cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf984c4d0 0x55eaf9c1cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/356109965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf98442f0 0x55eaf9c1d760 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 -- 172.21.15.92:0/356109965 >> 172.21.15.92:0/356109965 conn(0x55eaf9861300 msgr2=0x55eaf9c14df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 -- 172.21.15.92:0/356109965 shutdown_connections 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 -- 172.21.15.92:0/356109965 wait complete. 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.788+0000 7f53ee5c4740 1 -- start start 2024-09-21T02:08:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 Processor -- start 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf98442f0 0x55eaf9c259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 0x55eaf9c26070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c24820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01ac360 con 0x55eaf984c4d0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01abc90 con 0x55eaf98442f0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53e01a8ed0 con 0x55eaf986d1b0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53ee5c4740 10 client.0 _ll_register_callbacks cb 0x7f53d8007960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 0x55eaf9c26070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 0x55eaf9c26070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58546/0 (socket says 172.21.15.92:58546) 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 -- 172.21.15.92:0/134562175 learned_addr learned my addr 172.21.15.92:0/134562175 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e5317640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf98442f0 0x55eaf9c259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e5b18640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c24820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 -- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 msgr2=0x55eaf9c24820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c24820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 -- 172.21.15.92:0/134562175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf98442f0 msgr2=0x55eaf9c259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf98442f0 0x55eaf9c259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e4b16640 1 -- 172.21.15.92:0/134562175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf9c24f90 con 0x55eaf984c4d0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e5b18640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c24820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.789+0000 7f53e5317640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf98442f0 0x55eaf9c259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53e4b16640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 0x55eaf9c26070 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0e4840 tx=0x7f53dc0ea840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdb2d86f5a7feb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53c8ff9640 1 -- 172.21.15.92:0/134562175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc020040 con 0x55eaf984c4d0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53c8ff9640 1 -- 172.21.15.92:0/134562175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f53dc0a91f0 con 0x55eaf984c4d0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d005d070 con 0x55eaf984c4d0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53c8ff9640 1 -- 172.21.15.92:0/134562175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53dc0ac8d0 con 0x55eaf984c4d0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 10 client.16635 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eaf9c23d80 con 0x55eaf984c4d0 2024-09-21T02:08:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53c8ff9640 1 -- 172.21.15.92:0/134562175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d005d070 con 0x55eaf984c4d0 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53c8ff9640 1 -- 172.21.15.92:0/134562175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f53dc0b1060 con 0x55eaf984c4d0 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 10 client.16635 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 20 client.16635 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf98011c0 0x55eaf9c98720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 4 client.16635 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.790+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eaf9b850f0 con 0x55eaf98011c0 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.791+0000 7f53e5317640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf98011c0 0x55eaf9c98720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.791+0000 7f53c8ff9640 10 client.16635 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.791+0000 7f53e5317640 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf98011c0 0x55eaf9c98720 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b355e1b8c969a31 server_cookie=db515ada0705c5d in_seq=0 out_seq=0 2024-09-21T02:08:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53c8ff9640 1 -- 172.21.15.92:0/134562175 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eaf9b850f0 con 0x55eaf98011c0 2024-09-21T02:08:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53c8ff9640 10 client.16635 handle_command_reply: tid=0 2024-09-21T02:08:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 client.16635 shutdown 2024-09-21T02:08:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 msgr2=0x55eaf9c26070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 0x55eaf9c26070 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0e4840 tx=0x7f53dc0ea840 comp rx=0 tx=0).stop 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 shutdown_connections 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eaf98011c0 0x55eaf9c98720 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eaf986d1b0 0x55eaf9c24820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eaf984c4d0 0x55eaf9c26070 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53c8ff9640 0 client.16635 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 --2- 172.21.15.92:0/134562175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eaf98442f0 0x55eaf9c259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 >> 172.21.15.92:0/134562175 conn(0x55eaf9861300 msgr2=0x55eaf9c1c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.806+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 shutdown_connections 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53ee5c4740 1 -- 172.21.15.92:0/134562175 wait complete. 2024-09-21T02:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53ee5c4740 20 client.16635 trim_cache size 0 max 16384 2024-09-21T02:08:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:08:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:08:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:08:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:08:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53cc0b6510 msgr2=0x7f53cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53cc0b6510 0x7f53cc0b8a20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f53d8005fd0 tx=0x7f53d8007520 comp rx=0 tx=0).stop 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 msgr2=0x7f53e019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 0x7f53e019ed20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00d770 tx=0x7f53dc00dc30 comp rx=0 tx=0).stop 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 shutdown_connections 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f53cc0b6510 0x7f53cc0b8a20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53e0103fc0 0x7f53e01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53e0103620 0x7f53e019ed20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 --2- 172.21.15.92:0/3414666576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e01025a0 0x7f53e006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 >> 172.21.15.92:0/3414666576 conn(0x7f53e00fdee0 msgr2=0x7f53e0106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 shutdown_connections 2024-09-21T02:08:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:08:51.807+0000 7f53e757b640 1 -- 172.21.15.92:0/3414666576 wait complete. 2024-09-21T02:08:51.880 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 -- 172.21.15.92:0/1053457411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f4102a80 msgr2=0x7f86f4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/1053457411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f4102a80 0x7f86f4102e90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009900 tx=0x7f86e802a410 comp rx=0 tx=0).stop 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 -- 172.21.15.92:0/1053457411 shutdown_connections 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/1053457411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f4108d10 0x7f86f410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/1053457411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f41083c0 0x7f86f41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/1053457411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f4102a80 0x7f86f4102e90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 -- 172.21.15.92:0/1053457411 >> 172.21.15.92:0/1053457411 conn(0x7f86f4079740 msgr2=0x7f86f4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 -- 172.21.15.92:0/1053457411 shutdown_connections 2024-09-21T02:08:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 -- 172.21.15.92:0/1053457411 wait complete. 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 Processor -- start 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.344+0000 7f86fb4a1640 1 -- start start 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86fb4a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f4102a80 0x7f86f419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86fb4a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 0x7f86f419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86fb4a1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f4108d10 0x7f86f419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86fb4a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41129e0 con 0x7f86f41083c0 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86fb4a1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86f4112860 con 0x7f86f4102a80 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86fb4a1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f86f4112b60 con 0x7f86f4108d10 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 0x7f86f419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 0x7f86f419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58572/0 (socket says 172.21.15.92:58572) 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 -- 172.21.15.92:0/4136074861 learned_addr learned my addr 172.21.15.92:0/4136074861 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f923d640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f4102a80 0x7f86f419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f4108d10 0x7f86f419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 -- 172.21.15.92:0/4136074861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f4108d10 msgr2=0x7f86f419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f4108d10 0x7f86f419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 -- 172.21.15.92:0/4136074861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f4102a80 msgr2=0x7f86f419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f4102a80 0x7f86f419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 -- 172.21.15.92:0/4136074861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f41a0450 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 0x7f86f419aa80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c970 tx=0x7f86f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.345+0000 7f86e67fc640 1 -- 172.21.15.92:0/4136074861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0007bb0 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.346+0000 7f86e67fc640 1 -- 172.21.15.92:0/4136074861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f0007d50 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.346+0000 7f86e67fc640 1 -- 172.21.15.92:0/4136074861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000f450 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.346+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f41a0740 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.346+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f41a6f20 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.348+0000 7f8701a15740 1 Processor -- start 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 -- start start 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86e67fc640 1 -- 172.21.15.92:0/4136074861 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f86f0005020 con 0x7f86f41083c0 2024-09-21T02:08:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 0x56227aa076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa07be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56227a97dea0 con 0x56227a660070 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56227a96d2d0 con 0x56227a6371b0 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f8701a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56227a9710e0 con 0x56227a63f390 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 0x56227aa076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 0x56227aa076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58582/0 (socket says 172.21.15.92:58582) 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 -- 172.21.15.92:0/3292367791 learned_addr learned my addr 172.21.15.92:0/3292367791 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f923d640 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa07be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 -- 172.21.15.92:0/3292367791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 msgr2=0x56227aa08120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa08120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 -- 172.21.15.92:0/3292367791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 msgr2=0x56227aa07be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa07be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.349+0000 7f86f9a3e640 1 -- 172.21.15.92:0/3292367791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227a6cb650 con 0x56227a660070 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 0x56227aa076a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009f20 tx=0x7f86ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da88d3e9b001704b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86e67fc640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86e00b6590 0x7f86e00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86f923d640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86e00b6590 0x7f86e00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86df7fe640 1 -- 172.21.15.92:0/3292367791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec01a070 con 0x56227a660070 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86df7fe640 1 -- 172.21.15.92:0/3292367791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0158c0 con 0x56227a660070 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86df7fe640 1 -- 172.21.15.92:0/3292367791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00a7c0 con 0x56227a660070 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.350+0000 7f86e67fc640 1 -- 172.21.15.92:0/4136074861 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86f00e7280 con 0x7f86f41083c0 2024-09-21T02:08:52.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 -- 172.21.15.92:0/3292367791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 msgr2=0x56227aa076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f86f923d640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86e00b6590 0x7f86e00b8aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f86e8031440 tx=0x7f86e8009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 0x56227aa076a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009f20 tx=0x7f86ec00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 -- 172.21.15.92:0/3292367791 shutdown_connections 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa07be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 --2- 172.21.15.92:0/3292367791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a660070 0x56227aa076a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 -- 172.21.15.92:0/3292367791 >> 172.21.15.92:0/3292367791 conn(0x56227a654460 msgr2=0x56227aa05590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 -- 172.21.15.92:0/3292367791 shutdown_connections 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 -- 172.21.15.92:0/3292367791 wait complete. 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 -- start start 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.351+0000 7f8701a15740 1 Processor -- start 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa12e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 0x56227aa13360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56227a7ce700 con 0x56227a63f390 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56227a908160 con 0x56227a6371b0 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56227aa15e80 con 0x56227a660070 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f8701a15740 10 client.0 _ll_register_callbacks cb 0x56227a9219e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 0x56227aa13360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 0x56227aa13360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58596/0 (socket says 172.21.15.92:58596) 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 -- 172.21.15.92:0/935961330 learned_addr learned my addr 172.21.15.92:0/935961330 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f923d640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa12e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 -- 172.21.15.92:0/935961330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 msgr2=0x56227aa1f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 -- 172.21.15.92:0/935961330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 msgr2=0x56227aa12e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa12e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f8a3c640 1 -- 172.21.15.92:0/935961330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227a935c90 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f923d640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa12e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.352+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 0x56227aa13360 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f86f00a99c0 tx=0x7f86f00a9290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd3d8cd1b304364 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86dcff9640 1 -- 172.21.15.92:0/935961330 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f001d040 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86dcff9640 1 -- 172.21.15.92:0/935961330 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86f0094b80 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec0158c0 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86dcff9640 1 -- 172.21.15.92:0/935961330 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86f00acb60 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 10 client.16647 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56227aa21c90 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86dcff9640 1 -- 172.21.15.92:0/935961330 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0158c0 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f86dcff9640 1 -- 172.21.15.92:0/935961330 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86f00a7af0 con 0x56227a63f390 2024-09-21T02:08:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 10 client.16647 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 10 client.16647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 20 client.16647 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.353+0000 7f8701a15740 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a61e9f0 0x56227aa25190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.354+0000 7f8701a15740 4 client.16647 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.354+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56227a97fdf0 con 0x56227a61e9f0 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.354+0000 7f86f923d640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a61e9f0 0x56227aa25190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.354+0000 7f86dcff9640 10 client.16647 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.354+0000 7f86f923d640 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a61e9f0 0x56227aa25190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5164dbf523b74de8 server_cookie=3732ec5903aa98a5 in_seq=0 out_seq=0 2024-09-21T02:08:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f86dcff9640 1 -- 172.21.15.92:0/935961330 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56227a97fdf0 con 0x56227a61e9f0 2024-09-21T02:08:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f86dcff9640 10 client.16647 handle_command_reply: tid=0 2024-09-21T02:08:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 client.16647 shutdown 2024-09-21T02:08:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 msgr2=0x56227aa13360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 0x56227aa13360 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f86f00a99c0 tx=0x7f86f00a9290 comp rx=0 tx=0).stop 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 shutdown_connections 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a61e9f0 0x56227aa25190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a63f390 0x56227aa13360 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 --2- 172.21.15.92:0/935961330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a6371b0 0x56227aa12e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f86dcff9640 0 client.16647 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 >> 172.21.15.92:0/935961330 conn(0x56227a654460 msgr2=0x56227aa0a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 shutdown_connections 2024-09-21T02:08:52.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 1 -- 172.21.15.92:0/935961330 wait complete. 2024-09-21T02:08:52.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.355+0000 7f8701a15740 20 client.16647 trim_cache size 0 max 16384 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.368+0000 7f8701a15740 1 Processor -- start 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.368+0000 7f8701a15740 1 -- start start 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f8701a15740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f8701a15740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a63f390 0x56227aa14a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f8701a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f8701a15740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41af4c0 con 0x56227a6371b0 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f8701a15740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41b1000 con 0x56227a63f390 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f8701a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41afb90 con 0x56227a660070 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa09090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58604/0 (socket says 172.21.15.92:58604) 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 -- 172.21.15.92:0/2110250547 learned_addr learned my addr 172.21.15.92:0/2110250547 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:52.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a63f390 0x56227aa14a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 -- 172.21.15.92:0/2110250547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 msgr2=0x56227aa1f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 -- 172.21.15.92:0/2110250547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a63f390 msgr2=0x56227aa14a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a63f390 0x56227aa14a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 -- 172.21.15.92:0/2110250547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227aa140e0 con 0x56227a6371b0 2024-09-21T02:08:52.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.369+0000 7f86f923d640 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa09090 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f86e8032fd0 tx=0x7f86e803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b89aec12a54f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f86deffd640 1 -- 172.21.15.92:0/2110250547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8035040 con 0x56227a6371b0 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f86deffd640 1 -- 172.21.15.92:0/2110250547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e80035f0 con 0x56227a6371b0 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f86deffd640 1 -- 172.21.15.92:0/2110250547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e803c730 con 0x56227a6371b0 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 -- 172.21.15.92:0/2110250547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 msgr2=0x56227aa09090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa09090 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f86e8032fd0 tx=0x7f86e803e000 comp rx=0 tx=0).stop 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 -- 172.21.15.92:0/2110250547 shutdown_connections 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a660070 0x56227aa1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a63f390 0x56227aa14a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 --2- 172.21.15.92:0/2110250547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa09090 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 -- 172.21.15.92:0/2110250547 >> 172.21.15.92:0/2110250547 conn(0x56227a654460 msgr2=0x56227aa14340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:52.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.370+0000 7f8701a15740 1 -- 172.21.15.92:0/2110250547 shutdown_connections 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 -- 172.21.15.92:0/2110250547 wait complete. 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 -- start start 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 Processor -- start 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a660070 0x56227aa19dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41b0930 con 0x56227a6371b0 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41b0260 con 0x56227a660070 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.371+0000 7f8701a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f86f41ad4a0 con 0x56227a63f390 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f8701a15740 10 client.0 _ll_register_callbacks cb 0x7f86f00058a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa19300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58614/0 (socket says 172.21.15.92:58614) 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 -- 172.21.15.92:0/4224034770 learned_addr learned my addr 172.21.15.92:0/4224034770 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:52.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f9a3e640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a660070 0x56227aa19dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 -- 172.21.15.92:0/4224034770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 msgr2=0x56227aa19840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa19840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 -- 172.21.15.92:0/4224034770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a660070 msgr2=0x56227aa19dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a660070 0x56227aa19dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 -- 172.21.15.92:0/4224034770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227aa8edb0 con 0x56227a6371b0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa19840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86f923d640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa19300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006010 tx=0x7f86e802abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5ee806223029e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86dcff9640 1 -- 172.21.15.92:0/4224034770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e805b070 con 0x56227a6371b0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86dcff9640 1 -- 172.21.15.92:0/4224034770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f86e80037a0 con 0x56227a6371b0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e80035f0 con 0x56227a6371b0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f86dcff9640 1 -- 172.21.15.92:0/4224034770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86e8066070 con 0x56227a6371b0 2024-09-21T02:08:52.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f8701a15740 10 client.16653 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.372+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56227aa90030 con 0x56227a6371b0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f86dcff9640 1 -- 172.21.15.92:0/4224034770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e80035f0 con 0x56227a6371b0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f86dcff9640 1 -- 172.21.15.92:0/4224034770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86e8060c90 con 0x56227a6371b0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f8701a15740 10 client.16653 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f8701a15740 10 client.16653 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f8701a15740 20 client.16653 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f8701a15740 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a5f3da0 0x56227aa92850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f8701a15740 4 client.16653 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56227a97b8a0 con 0x56227a5f3da0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a5f3da0 0x56227aa92850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.373+0000 7f86dcff9640 10 client.16653 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f86f8a3c640 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a5f3da0 0x56227aa92850 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=666a7077125318b2 server_cookie=3172d9e410b35ded in_seq=0 out_seq=0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f86dcff9640 1 -- 172.21.15.92:0/4224034770 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56227a97b8a0 con 0x56227a5f3da0 2024-09-21T02:08:52.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f86dcff9640 10 client.16653 handle_command_reply: tid=0 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 client.16653 shutdown 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 msgr2=0x56227aa19300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa19300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006010 tx=0x7f86e802abc0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 shutdown_connections 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x56227a5f3da0 0x56227aa92850 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56227a660070 0x56227aa19dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56227a63f390 0x56227aa19840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f86dcff9640 0 client.16653 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 --2- 172.21.15.92:0/4224034770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56227a6371b0 0x56227aa19300 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 >> 172.21.15.92:0/4224034770 conn(0x56227a654460 msgr2=0x56227aa08e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 shutdown_connections 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 1 -- 172.21.15.92:0/4224034770 wait complete. 2024-09-21T02:08:52.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.374+0000 7f8701a15740 20 client.16653 trim_cache size 0 max 16384 2024-09-21T02:08:52.381 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86e00b6590 msgr2=0x7f86e00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86e00b6590 0x7f86e00b8aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f86e8031440 tx=0x7f86e8009680 comp rx=0 tx=0).stop 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 msgr2=0x7f86f419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 0x7f86f419aa80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c970 tx=0x7f86f000ce30 comp rx=0 tx=0).stop 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 shutdown_connections 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86e00b6590 0x7f86e00b8aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f4108d10 0x7f86f419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f41083c0 0x7f86f419aa80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 --2- 172.21.15.92:0/4136074861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f4102a80 0x7f86f419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 >> 172.21.15.92:0/4136074861 conn(0x7f86f4079740 msgr2=0x7f86f4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 shutdown_connections 2024-09-21T02:08:52.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:52.375+0000 7f86fb4a1640 1 -- 172.21.15.92:0/4136074861 wait complete. 2024-09-21T02:08:53.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:08:53.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:53 smithi136 ceph-mon[34754]: pgmap v477: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:08:53.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:53 smithi092 ceph-mon[31190]: pgmap v477: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:08:53.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:53 smithi099 ceph-mon[34404]: pgmap v477: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:08:53.436 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:53.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.893+0000 7f44aeeb3640 1 -- 172.21.15.92:0/3005469386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a81094a0 msgr2=0x7f44a81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.893+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/3005469386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a81094a0 0x7f44a81098b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4494009900 tx=0x7f449402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 -- 172.21.15.92:0/3005469386 shutdown_connections 2024-09-21T02:08:53.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/3005469386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44a8109df0 0x7f44a8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/3005469386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a81094a0 0x7f44a81098b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/3005469386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 0x7f44a8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 -- 172.21.15.92:0/3005469386 >> 172.21.15.92:0/3005469386 conn(0x7f44a807b850 msgr2=0x7f44a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 -- 172.21.15.92:0/3005469386 shutdown_connections 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 -- 172.21.15.92:0/3005469386 wait complete. 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 Processor -- start 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.894+0000 7f44aeeb3640 1 -- start start 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 0x7f44a819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44a81094a0 0x7f44a819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 0x7f44a819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8113ab0 con 0x7f44a8109df0 2024-09-21T02:08:53.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8113930 con 0x7f44a8103b60 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8113c30 con 0x7f44a81094a0 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 0x7f44a819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 0x7f44a819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49024/0 (socket says 172.21.15.92:49024) 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 -- 172.21.15.92:0/1050168298 learned_addr learned my addr 172.21.15.92:0/1050168298 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 0x7f44a819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44a81094a0 0x7f44a819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 -- 172.21.15.92:0/1050168298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44a81094a0 msgr2=0x7f44a819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44a81094a0 0x7f44a819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 -- 172.21.15.92:0/1050168298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 msgr2=0x7f44a819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 0x7f44a819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 -- 172.21.15.92:0/1050168298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a81a0370 con 0x7f44a8109df0 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 0x7f44a819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:53.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44ad450640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 0x7f44a819fd00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f44a000cce0 tx=0x7f44a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44a5ffb640 1 -- 172.21.15.92:0/1050168298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a000f450 con 0x7f44a8109df0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a81a0660 con 0x7f44a8109df0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44a5ffb640 1 -- 172.21.15.92:0/1050168298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a000faa0 con 0x7f44a8109df0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.895+0000 7f44a5ffb640 1 -- 172.21.15.92:0/1050168298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0018b70 con 0x7f44a8109df0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.896+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a81a6e80 con 0x7f44a8109df0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.898+0000 7f44a5ffb640 1 -- 172.21.15.92:0/1050168298 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f44a0018d10 con 0x7f44a8109df0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.898+0000 7f44b5427740 1 Processor -- start 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.898+0000 7f44b5427740 1 -- start start 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44b5427740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44b5427740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 0x560b335d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44b5427740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44b5427740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560b33547ea0 con 0x560b332011b0 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44b5427740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560b335372d0 con 0x560b3322a070 2024-09-21T02:08:53.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44b5427740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b3353b0e0 con 0x560b33209390 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 0x560b335d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 0x560b335d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49038/0 (socket says 172.21.15.92:49038) 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 -- 172.21.15.92:0/1278322010 learned_addr learned my addr 172.21.15.92:0/1278322010 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44a5ffb640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f44980b6540 0x7f44980b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44ad450640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f44980b6540 0x7f44980b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 -- 172.21.15.92:0/1278322010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 msgr2=0x560b335d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 -- 172.21.15.92:0/1278322010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 msgr2=0x560b335d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 -- 172.21.15.92:0/1278322010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b33295650 con 0x560b332011b0 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:53.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44a5ffb640 1 -- 172.21.15.92:0/1050168298 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f44a00e77f0 con 0x7f44a8109df0 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.899+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 0x560b335d1be0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f449c008c60 tx=0x7f449c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1b0fb1c4dcb703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.900+0000 7f4492ffd640 1 -- 172.21.15.92:0/1278322010 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c014070 con 0x560b332011b0 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.900+0000 7f44ad450640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f44980b6540 0x7f44980b8a50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f44a00a9f30 tx=0x7f44a007ac90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.900+0000 7f4492ffd640 1 -- 172.21.15.92:0/1278322010 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c0047a0 con 0x560b332011b0 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.900+0000 7f4492ffd640 1 -- 172.21.15.92:0/1278322010 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c010690 con 0x560b332011b0 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 -- 172.21.15.92:0/1278322010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 msgr2=0x560b335d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 0x560b335d1be0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f449c008c60 tx=0x7f449c00e720 comp rx=0 tx=0).stop 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 -- 172.21.15.92:0/1278322010 shutdown_connections 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b332011b0 0x560b335d1be0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 --2- 172.21.15.92:0/1278322010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 -- 172.21.15.92:0/1278322010 >> 172.21.15.92:0/1278322010 conn(0x560b3321e460 msgr2=0x560b335cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 -- 172.21.15.92:0/1278322010 shutdown_connections 2024-09-21T02:08:53.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 -- 172.21.15.92:0/1278322010 wait complete. 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 -- start start 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.901+0000 7f44b5427740 1 Processor -- start 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335dcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335dd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b3322a070 0x560b335e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560b33398700 con 0x560b33209390 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560b334d2160 con 0x560b332011b0 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b335dfe80 con 0x560b3322a070 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 10 client.0 _ll_register_callbacks cb 0x560b334eb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335dd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335dd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49054/0 (socket says 172.21.15.92:49054) 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 -- 172.21.15.92:0/1409990473 learned_addr learned my addr 172.21.15.92:0/1409990473 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b3322a070 0x560b335e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335dcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 -- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b3322a070 msgr2=0x560b335e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b3322a070 0x560b335e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 -- 172.21.15.92:0/1409990473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 msgr2=0x560b335dcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335dcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 -- 172.21.15.92:0/1409990473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b334ffc90 con 0x560b33209390 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b3322a070 0x560b335e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44acc4f640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335dd320 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f449c010bb0 tx=0x7f449c010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ffb826d528f4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f4487fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f4487fff640 1 -- 172.21.15.92:0/1409990473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c027060 con 0x560b33209390 2024-09-21T02:08:53.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f4487fff640 1 -- 172.21.15.92:0/1409990473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f449c022410 con 0x560b33209390 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.902+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c0047a0 con 0x560b33209390 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f4487fff640 1 -- 172.21.15.92:0/1409990473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f449c014070 con 0x560b33209390 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 10 client.16665 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b335ebbf0 con 0x560b33209390 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f4487fff640 1 -- 172.21.15.92:0/1409990473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c0047a0 con 0x560b33209390 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f4487fff640 1 -- 172.21.15.92:0/1409990473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f449c018450 con 0x560b33209390 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 10 client.16665 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 10 client.16665 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 20 client.16665 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331e89f0 0x560b335ef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 4 client.16665 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.903+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b33549df0 con 0x560b331e89f0 2024-09-21T02:08:53.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.904+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331e89f0 0x560b335ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.904+0000 7f44a7fff640 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331e89f0 0x560b335ef000 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f08bd8611830e47 server_cookie=10d1a960264768a7 in_seq=0 out_seq=0 2024-09-21T02:08:53.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.904+0000 7f4487fff640 10 client.16665 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:53.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f4487fff640 1 -- 172.21.15.92:0/1409990473 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560b33549df0 con 0x560b331e89f0 2024-09-21T02:08:53.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f4487fff640 10 client.16665 handle_command_reply: tid=0 2024-09-21T02:08:53.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 client.16665 shutdown 2024-09-21T02:08:53.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 msgr2=0x560b335dd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335dd320 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f449c010bb0 tx=0x7f449c010be0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 shutdown_connections 2024-09-21T02:08:53.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b3322a070 0x560b335e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335dd320 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331e89f0 0x560b335ef000 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 --2- 172.21.15.92:0/1409990473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335dcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f4487fff640 0 client.16665 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 >> 172.21.15.92:0/1409990473 conn(0x560b3321e460 msgr2=0x560b335d4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 shutdown_connections 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 1 -- 172.21.15.92:0/1409990473 wait complete. 2024-09-21T02:08:53.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.905+0000 7f44b5427740 20 client.16665 trim_cache size 0 max 16384 2024-09-21T02:08:53.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 Processor -- start 2024-09-21T02:08:53.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 -- start start 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b332011b0 0x560b335dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81af3d0 con 0x560b33209390 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81b0f10 con 0x560b3322a070 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44b5427740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81afaa0 con 0x560b332011b0 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335e9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49058/0 (socket says 172.21.15.92:49058) 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 -- 172.21.15.92:0/2382547993 learned_addr learned my addr 172.21.15.92:0/2382547993 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b332011b0 0x560b335dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.922 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 -- 172.21.15.92:0/2382547993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b332011b0 msgr2=0x560b335dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b332011b0 0x560b335dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 -- 172.21.15.92:0/2382547993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 msgr2=0x560b335d9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 -- 172.21.15.92:0/2382547993 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b335de0b0 con 0x560b33209390 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.919+0000 7f44acc4f640 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335e9110 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f449c0187f0 tx=0x7f449c005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a2820e8bc14e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44927fc640 1 -- 172.21.15.92:0/2382547993 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c027040 con 0x560b33209390 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44927fc640 1 -- 172.21.15.92:0/2382547993 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c00ec20 con 0x560b33209390 2024-09-21T02:08:53.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44927fc640 1 -- 172.21.15.92:0/2382547993 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c018cc0 con 0x560b33209390 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 -- 172.21.15.92:0/2382547993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 msgr2=0x560b335e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335e9110 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f449c0187f0 tx=0x7f449c005f40 comp rx=0 tx=0).stop 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 -- 172.21.15.92:0/2382547993 shutdown_connections 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b3322a070 0x560b335d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b33209390 0x560b335e9110 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 --2- 172.21.15.92:0/2382547993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b332011b0 0x560b335dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.920+0000 7f44b5427740 1 -- 172.21.15.92:0/2382547993 >> 172.21.15.92:0/2382547993 conn(0x560b3321e460 msgr2=0x560b335dcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 -- 172.21.15.92:0/2382547993 shutdown_connections 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 -- 172.21.15.92:0/2382547993 wait complete. 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 -- start start 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 Processor -- start 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 0x560b335f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81b0840 con 0x560b3322a070 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81b0170 con 0x560b332011b0 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ad3b0 con 0x560b33209390 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.921+0000 7f44b5427740 10 client.0 _ll_register_callbacks cb 0x7f4494009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 0x560b335f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 0x560b335f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49064/0 (socket says 172.21.15.92:49064) 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 -- 172.21.15.92:0/2973586528 learned_addr learned my addr 172.21.15.92:0/2973586528 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44acc4f640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 -- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 msgr2=0x560b335efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 -- 172.21.15.92:0/2973586528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 msgr2=0x560b335f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:53.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 -- 172.21.15.92:0/2973586528 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b335f0620 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44a7fff640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 0x560b335f0060 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4494000c00 tx=0x7f4494040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905c5643fad81b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f4487fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f4487fff640 1 -- 172.21.15.92:0/2973586528 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494035040 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c00ec20 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f4487fff640 1 -- 172.21.15.92:0/2973586528 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4494005220 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.922+0000 7f4487fff640 1 -- 172.21.15.92:0/2973586528 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4494005420 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 10 client.16671 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b33659460 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f4487fff640 1 -- 172.21.15.92:0/2973586528 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449c00ec20 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f4487fff640 1 -- 172.21.15.92:0/2973586528 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f449403a040 con 0x560b3322a070 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 10 client.16671 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 10 client.16671 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:08:53.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 20 client.16671 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331bdda0 0x560b3365bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 4 client.16671 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b335458a0 con 0x560b331bdda0 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.923+0000 7f44acc4f640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331bdda0 0x560b3365bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f4487fff640 10 client.16671 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44acc4f640 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331bdda0 0x560b3365bc00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82052b7bbac1151c server_cookie=21fe5dca7d866234 in_seq=0 out_seq=0 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f4487fff640 1 -- 172.21.15.92:0/2973586528 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b335458a0 con 0x560b331bdda0 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f4487fff640 10 client.16671 handle_command_reply: tid=0 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 client.16671 shutdown 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 msgr2=0x560b335f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 0x560b335f0060 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4494000c00 tx=0x7f4494040040 comp rx=0 tx=0).stop 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 shutdown_connections 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560b3322a070 0x560b335f0060 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b33209390 0x560b335efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x560b331bdda0 0x560b3365bc00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 --2- 172.21.15.92:0/2973586528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b332011b0 0x560b335f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f4487fff640 0 client.16671 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 >> 172.21.15.92:0/2973586528 conn(0x560b3321e460 msgr2=0x560b335d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 shutdown_connections 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 1 -- 172.21.15.92:0/2973586528 wait complete. 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.924+0000 7f44b5427740 20 client.16671 trim_cache size 0 max 16384 2024-09-21T02:08:53.931 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f44980b6540 msgr2=0x7f44980b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f44980b6540 0x7f44980b8a50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f44a00a9f30 tx=0x7f44a007ac90 comp rx=0 tx=0).stop 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 msgr2=0x7f44a819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 0x7f44a819fd00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f44a000cce0 tx=0x7f44a000a620 comp rx=0 tx=0).stop 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 shutdown_connections 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f44980b6540 0x7f44980b8a50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44a8109df0 0x7f44a819fd00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44a81094a0 0x7f44a819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 --2- 172.21.15.92:0/1050168298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a8103b60 0x7f44a819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:53.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 >> 172.21.15.92:0/1050168298 conn(0x7f44a807b850 msgr2=0x7f44a8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:53.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 shutdown_connections 2024-09-21T02:08:53.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:53.925+0000 7f44aeeb3640 1 -- 172.21.15.92:0/1050168298 wait complete. 2024-09-21T02:08:54.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:08:55.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:55 smithi092 ceph-mon[31190]: pgmap v478: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:08:55.059 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:55.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:55 smithi136 ceph-mon[34754]: pgmap v478: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:08:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:55 smithi099 ceph-mon[34404]: pgmap v478: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 -- 172.21.15.92:0/2962678994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584107b80 msgr2=0x7fe584107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 --2- 172.21.15.92:0/2962678994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584107b80 0x7fe584107f90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009900 tx=0x7fe56c02a410 comp rx=0 tx=0).stop 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 -- 172.21.15.92:0/2962678994 shutdown_connections 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 --2- 172.21.15.92:0/2962678994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5841084d0 0x7fe58410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 --2- 172.21.15.92:0/2962678994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584107b80 0x7fe584107f90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 --2- 172.21.15.92:0/2962678994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe584102240 0x7fe584102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 -- 172.21.15.92:0/2962678994 >> 172.21.15.92:0/2962678994 conn(0x7fe584079f90 msgr2=0x7fe58407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 -- 172.21.15.92:0/2962678994 shutdown_connections 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.527+0000 7fe589934640 1 -- 172.21.15.92:0/2962678994 wait complete. 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.528+0000 7fe589934640 1 Processor -- start 2024-09-21T02:08:55.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.528+0000 7fe589934640 1 -- start start 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 0x7fe58419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe584107b80 0x7fe58419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5841084d0 0x7fe58419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe584113b00 con 0x7fe584102240 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe584113980 con 0x7fe584107b80 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe584113c80 con 0x7fe5841084d0 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 0x7fe58419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 0x7fe58419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49090/0 (socket says 172.21.15.92:49090) 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 -- 172.21.15.92:0/3767895462 learned_addr learned my addr 172.21.15.92:0/3767895462 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe584107b80 0x7fe58419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5841084d0 0x7fe58419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 -- 172.21.15.92:0/3767895462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5841084d0 msgr2=0x7fe58419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5841084d0 0x7fe58419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 -- 172.21.15.92:0/3767895462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe584107b80 msgr2=0x7fe58419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe584107b80 0x7fe58419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 -- 172.21.15.92:0/3767895462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5841a0400 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe584107b80 0x7fe58419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5841084d0 0x7fe58419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe582ffd640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 0x7fe58419a550 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c920 tx=0x7fe57400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe588932640 1 -- 172.21.15.92:0/3767895462 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574007c70 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5841a06f0 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe588932640 1 -- 172.21.15.92:0/3767895462 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe574007e10 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe588932640 1 -- 172.21.15.92:0/3767895462 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574018820 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.529+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5841a6f10 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe588932640 1 -- 172.21.15.92:0/3767895462 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe57400f450 con 0x7fe584102240 2024-09-21T02:08:55.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 Processor -- start 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 -- start start 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x5609642536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x560964253be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 0x560964254120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5609641c9ea0 con 0x560963e8b390 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609641b92d0 con 0x560963e831b0 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.532+0000 7fe58fea8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5609641bd0e0 con 0x560963eac070 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 0x560964254120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 0x560964254120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49092/0 (socket says 172.21.15.92:49092) 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 -- 172.21.15.92:0/2714210353 learned_addr learned my addr 172.21.15.92:0/2714210353 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe5837fe640 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x5609642536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x560964253be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 -- 172.21.15.92:0/2714210353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 msgr2=0x5609642536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x5609642536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 -- 172.21.15.92:0/2714210353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 msgr2=0x560964253be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x560964253be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 -- 172.21.15.92:0/2714210353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560963f17650 con 0x560963e8b390 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe588932640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe5700b6540 0x7fe5700b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe5700b6540 0x7fe5700b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 0x560964254120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0096c0 tx=0x7fe56c031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99f8399a87758a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe588932640 1 -- 172.21.15.92:0/3767895462 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe5740e7510 con 0x7fe584102240 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.533+0000 7fe579ffb640 1 -- 172.21.15.92:0/2714210353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c03f070 con 0x560963e8b390 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe579ffb640 1 -- 172.21.15.92:0/2714210353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56c031760 con 0x560963e8b390 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe579ffb640 1 -- 172.21.15.92:0/2714210353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c035070 con 0x560963e8b390 2024-09-21T02:08:55.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 -- 172.21.15.92:0/2714210353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 msgr2=0x560964254120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe5700b6540 0x7fe5700b8a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00af00 tx=0x7fe57c00a6b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 0x560964254120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0096c0 tx=0x7fe56c031e80 comp rx=0 tx=0).stop 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 -- 172.21.15.92:0/2714210353 shutdown_connections 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e8b390 0x560964254120 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x560964253be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2714210353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x5609642536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 -- 172.21.15.92:0/2714210353 >> 172.21.15.92:0/2714210353 conn(0x560963ea0460 msgr2=0x560964251590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.534+0000 7fe58fea8740 1 -- 172.21.15.92:0/2714210353 shutdown_connections 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 -- 172.21.15.92:0/2714210353 wait complete. 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 -- start start 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 Processor -- start 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x56096425ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e8b390 0x56096425f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x56096426b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56096401a700 con 0x560963eac070 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560964154160 con 0x560963e831b0 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.535+0000 7fe58fea8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560964261e20 con 0x560963e8b390 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe58fea8740 10 client.0 _ll_register_callbacks cb 0x56096416d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x56096426b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x56096426b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49104/0 (socket says 172.21.15.92:49104) 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 -- 172.21.15.92:0/2989833425 learned_addr learned my addr 172.21.15.92:0/2989833425 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x56096425ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e8b390 0x56096425f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 -- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e8b390 msgr2=0x56096425f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e8b390 0x56096425f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 -- 172.21.15.92:0/2989833425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 msgr2=0x56096425ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x56096425ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 -- 172.21.15.92:0/2989833425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560964181c90 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x56096425ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe5837fe640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x56096426b140 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00a4a0 tx=0x7fe57c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22711a9dd578b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe562ffd640 1 -- 172.21.15.92:0/2989833425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c01cd30 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe562ffd640 1 -- 172.21.15.92:0/2989833425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe57c027430 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56c031760 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe562ffd640 1 -- 172.21.15.92:0/2989833425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe57c027630 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe58fea8740 10 client.16683 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.536+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56096426dc10 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe562ffd640 1 -- 172.21.15.92:0/2989833425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56c031760 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe562ffd640 1 -- 172.21.15.92:0/2989833425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe57c01a420 con 0x560963eac070 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe58fea8740 10 client.16683 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe58fea8740 10 client.16683 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe58fea8740 20 client.16683 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:55.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e6a9f0 0x560964271170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe58fea8740 4 client.16683 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609641cbdf0 con 0x560963e6a9f0 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.537+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e6a9f0 0x560964271170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.538+0000 7fe562ffd640 10 client.16683 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.538+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e6a9f0 0x560964271170 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=666efea968680ad0 server_cookie=ee6ea4f58a69162c in_seq=0 out_seq=0 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.538+0000 7fe562ffd640 1 -- 172.21.15.92:0/2989833425 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609641cbdf0 con 0x560963e6a9f0 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.538+0000 7fe562ffd640 10 client.16683 handle_command_reply: tid=0 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.538+0000 7fe58fea8740 1 client.16683 shutdown 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 msgr2=0x56096426b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x56096426b140 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00a4a0 tx=0x7fe57c004a40 comp rx=0 tx=0).stop 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 shutdown_connections 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e6a9f0 0x560964271170 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x56096426b140 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e8b390 0x56096425f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe562ffd640 0 client.16683 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:55.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2989833425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e831b0 0x56096425ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 >> 172.21.15.92:0/2989833425 conn(0x560963ea0460 msgr2=0x560964256830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:55.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 shutdown_connections 2024-09-21T02:08:55.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 1 -- 172.21.15.92:0/2989833425 wait complete. 2024-09-21T02:08:55.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.539+0000 7fe58fea8740 20 client.16683 trim_cache size 0 max 16384 2024-09-21T02:08:55.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 Processor -- start 2024-09-21T02:08:55.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 -- start start 2024-09-21T02:08:55.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 0x56096425dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964260990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x560964250bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841af410 con 0x560963e831b0 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841b0f50 con 0x560963e8b390 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe58fea8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841afae0 con 0x560963eac070 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe582ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 0x56096425dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe582ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 0x56096425dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49118/0 (socket says 172.21.15.92:49118) 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.552+0000 7fe582ffd640 1 -- 172.21.15.92:0/2351378991 learned_addr learned my addr 172.21.15.92:0/2351378991 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe5837fe640 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x560964250bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964260990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe582ffd640 1 -- 172.21.15.92:0/2351378991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 msgr2=0x560964250bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x560964250bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe582ffd640 1 -- 172.21.15.92:0/2351378991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 msgr2=0x560964260990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964260990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe582ffd640 1 -- 172.21.15.92:0/2351378991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609642544b0 con 0x560963e831b0 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964260990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe582ffd640 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 0x56096425dc50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe57401f000 tx=0x7fe5740ed060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0b325d620315f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe5797fa640 1 -- 172.21.15.92:0/2351378991 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740acdf0 con 0x560963e831b0 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe5797fa640 1 -- 172.21.15.92:0/2351378991 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5740ad190 con 0x560963e831b0 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.553+0000 7fe5797fa640 1 -- 172.21.15.92:0/2351378991 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740a98f0 con 0x560963e831b0 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 -- 172.21.15.92:0/2351378991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 msgr2=0x56096425dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 0x56096425dc50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe57401f000 tx=0x7fe5740ed060 comp rx=0 tx=0).stop 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 -- 172.21.15.92:0/2351378991 shutdown_connections 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963eac070 0x560964250bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964260990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 --2- 172.21.15.92:0/2351378991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963e831b0 0x56096425dc50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 -- 172.21.15.92:0/2351378991 >> 172.21.15.92:0/2351378991 conn(0x560963ea0460 msgr2=0x560964260280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 -- 172.21.15.92:0/2351378991 shutdown_connections 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 -- 172.21.15.92:0/2351378991 wait complete. 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.554+0000 7fe58fea8740 1 -- start start 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 Processor -- start 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e831b0 0x5609642634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964263ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x5609642dab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841b0880 con 0x560963eac070 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841b01b0 con 0x560963e8b390 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841ad3f0 con 0x560963e831b0 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe58fea8740 10 client.0 _ll_register_callbacks cb 0x7fe57c00ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x5609642dab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x5609642dab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49134/0 (socket says 172.21.15.92:49134) 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 -- 172.21.15.92:0/3405858589 learned_addr learned my addr 172.21.15.92:0/3405858589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe57a7fc640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964263ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe582ffd640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e831b0 0x5609642634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 -- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e831b0 msgr2=0x5609642634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e831b0 0x5609642634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 -- 172.21.15.92:0/3405858589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 msgr2=0x560964263ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964263ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.555+0000 7fe5837fe640 1 -- 172.21.15.92:0/3405858589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609642db070 con 0x560963eac070 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe5837fe640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x5609642dab30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000f80 tx=0x7fe57c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9aa8a0e0ff47165 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe562ffd640 1 -- 172.21.15.92:0/3405858589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c01a5a0 con 0x560963eac070 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe562ffd640 1 -- 172.21.15.92:0/3405858589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe57c01cd30 con 0x560963eac070 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5740ad190 con 0x560963eac070 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe562ffd640 1 -- 172.21.15.92:0/3405858589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe57c042070 con 0x560963eac070 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 10 client.16689 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609642dc280 con 0x560963eac070 2024-09-21T02:08:55.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe562ffd640 1 -- 172.21.15.92:0/3405858589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5740ad190 con 0x560963eac070 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe562ffd640 1 -- 172.21.15.92:0/3405858589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe57c03b030 con 0x560963eac070 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 10 client.16689 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 10 client.16689 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 20 client.16689 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e3fda0 0x5609642dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 4 client.16689 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.556+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609641c78a0 con 0x560963e3fda0 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe582ffd640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e3fda0 0x5609642dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe562ffd640 10 client.16689 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe582ffd640 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e3fda0 0x5609642dea70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef0f86348c2d7007 server_cookie=36389ca82487fb7a in_seq=0 out_seq=0 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe562ffd640 1 -- 172.21.15.92:0/3405858589 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609641c78a0 con 0x560963e3fda0 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe562ffd640 10 client.16689 handle_command_reply: tid=0 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe58fea8740 1 client.16689 shutdown 2024-09-21T02:08:55.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 msgr2=0x5609642dab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.557+0000 7fe58fea8740 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x5609642dab30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000f80 tx=0x7fe57c011df0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 shutdown_connections 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560963e3fda0 0x5609642dea70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560963eac070 0x5609642dab30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe562ffd640 0 client.16689 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560963e8b390 0x560964263ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 --2- 172.21.15.92:0/3405858589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560963e831b0 0x5609642634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 >> 172.21.15.92:0/3405858589 conn(0x560963ea0460 msgr2=0x5609642546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 shutdown_connections 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 1 -- 172.21.15.92:0/3405858589 wait complete. 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe58fea8740 20 client.16689 trim_cache size 0 max 16384 2024-09-21T02:08:55.566 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:55.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe5700b6540 msgr2=0x7fe5700b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe5700b6540 0x7fe5700b8a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00af00 tx=0x7fe57c00a6b0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 msgr2=0x7fe58419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:55.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 0x7fe58419a550 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c920 tx=0x7fe57400cde0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 shutdown_connections 2024-09-21T02:08:55.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe5700b6540 0x7fe5700b8a50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5841084d0 0x7fe58419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe584107b80 0x7fe58419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 --2- 172.21.15.92:0/3767895462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe584102240 0x7fe58419a550 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.558+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 >> 172.21.15.92:0/3767895462 conn(0x7fe584079f90 msgr2=0x7fe584105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.559+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 shutdown_connections 2024-09-21T02:08:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:55.559+0000 7fe589934640 1 -- 172.21.15.92:0/3767895462 wait complete. 2024-09-21T02:08:56.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:08:56.589 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:57.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 -- 172.21.15.92:0/830756014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 msgr2=0x7f206c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 --2- 172.21.15.92:0/830756014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c1087b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2060009900 tx=0x7f206002a410 comp rx=0 tx=0).stop 2024-09-21T02:08:57.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 -- 172.21.15.92:0/830756014 shutdown_connections 2024-09-21T02:08:57.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 --2- 172.21.15.92:0/830756014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f206c108cf0 0x7f206c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 --2- 172.21.15.92:0/830756014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c1087b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 --2- 172.21.15.92:0/830756014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f206c102a60 0x7f206c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 -- 172.21.15.92:0/830756014 >> 172.21.15.92:0/830756014 conn(0x7f206c079730 msgr2=0x7f206c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 -- 172.21.15.92:0/830756014 shutdown_connections 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.053+0000 7f2071189640 1 -- 172.21.15.92:0/830756014 wait complete. 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.054+0000 7f2071189640 1 Processor -- start 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.054+0000 7f2071189640 1 -- start start 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f2071189640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f206c102a60 0x7f206c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f2071189640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f2071189640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f206c108cf0 0x7f206c19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f2071189640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f206c112a20 con 0x7f206c1083a0 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f2071189640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1128a0 con 0x7f206c108cf0 2024-09-21T02:08:57.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f2071189640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f206c112ba0 con 0x7f206c102a60 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49158/0 (socket says 172.21.15.92:49158) 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 -- 172.21.15.92:0/2914383499 learned_addr learned my addr 172.21.15.92:0/2914383499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206b59e640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f206c108cf0 0x7f206c19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f206c102a60 0x7f206c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 -- 172.21.15.92:0/2914383499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f206c102a60 msgr2=0x7f206c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f206c102a60 0x7f206c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 -- 172.21.15.92:0/2914383499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f206c108cf0 msgr2=0x7f206c19fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f206c108cf0 0x7f206c19fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 -- 172.21.15.92:0/2914383499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c1a05d0 con 0x7f206c1083a0 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.055+0000 7f206a59c640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c19ac00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2060031a90 tx=0x7f2060004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.056+0000 7f2053fff640 1 -- 172.21.15.92:0/2914383499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206003f030 con 0x7f206c1083a0 2024-09-21T02:08:57.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.056+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206c1a0860 con 0x7f206c1083a0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.056+0000 7f2053fff640 1 -- 172.21.15.92:0/2914383499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2060003cb0 con 0x7f206c1083a0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.056+0000 7f2053fff640 1 -- 172.21.15.92:0/2914383499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060035070 con 0x7f206c1083a0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.056+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206c1a7020 con 0x7f206c1083a0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.058+0000 7f20776fd740 1 Processor -- start 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.058+0000 7f20776fd740 1 -- start start 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f2053fff640 1 -- 172.21.15.92:0/2914383499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2060042030 con 0x7f206c1083a0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f20776fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52d76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f20776fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 0x5616b52d7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f20776fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f0f390 0x5616b52d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f20776fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5616b524dea0 con 0x5616b4f071b0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f20776fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5616b523d2d0 con 0x5616b4f0f390 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f20776fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616b52410e0 con 0x5616b4f30070 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 0x5616b52d7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 0x5616b52d7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49160/0 (socket says 172.21.15.92:49160) 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 -- 172.21.15.92:0/3880751125 learned_addr learned my addr 172.21.15.92:0/3880751125 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206b59e640 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52d76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206a59c640 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f0f390 0x5616b52d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 -- 172.21.15.92:0/3880751125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 msgr2=0x5616b52d76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52d76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 -- 172.21.15.92:0/3880751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f0f390 msgr2=0x5616b52d8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f0f390 0x5616b52d8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 -- 172.21.15.92:0/3880751125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b4f9b650 con 0x5616b4f071b0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.059+0000 7f206ad9d640 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 0x5616b52d7be0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b820 tx=0x7f205c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5ff7c12f87bc52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f20517fa640 1 -- 172.21.15.92:0/3880751125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c00d8f0 con 0x5616b4f071b0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f20517fa640 1 -- 172.21.15.92:0/3880751125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f205c019070 con 0x5616b4f071b0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f20517fa640 1 -- 172.21.15.92:0/3880751125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c014e30 con 0x5616b4f071b0 2024-09-21T02:08:57.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f2053fff640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20540b6510 0x7f20540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f206b59e640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20540b6510 0x7f20540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f2053fff640 1 -- 172.21.15.92:0/2914383499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f20601078c0 con 0x7f206c1083a0 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f20776fd740 1 -- 172.21.15.92:0/3880751125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 msgr2=0x5616b52d7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f206b59e640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20540b6510 0x7f20540b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f206400a400 tx=0x7f206400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.060+0000 7f20776fd740 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 0x5616b52d7be0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b820 tx=0x7f205c00bce0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- 172.21.15.92:0/3880751125 shutdown_connections 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f0f390 0x5616b52d8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f071b0 0x5616b52d7be0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 --2- 172.21.15.92:0/3880751125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52d76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- 172.21.15.92:0/3880751125 >> 172.21.15.92:0/3880751125 conn(0x5616b4f24460 msgr2=0x5616b52d5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- 172.21.15.92:0/3880751125 shutdown_connections 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- 172.21.15.92:0/3880751125 wait complete. 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- start start 2024-09-21T02:08:57.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 Processor -- start 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f071b0 0x5616b52e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f30070 0x5616b52e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5616b509e700 con 0x5616b4f0f390 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5616b51d8160 con 0x5616b4f30070 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616b52ea1e0 con 0x5616b4f071b0 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.061+0000 7f20776fd740 10 client.0 _ll_register_callbacks cb 0x5616b51f19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49170/0 (socket says 172.21.15.92:49170) 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f071b0 0x5616b52e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 -- 172.21.15.92:0/2356001745 learned_addr learned my addr 172.21.15.92:0/2356001745 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206b59e640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f30070 0x5616b52e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 -- 172.21.15.92:0/2356001745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f071b0 msgr2=0x5616b52e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f071b0 0x5616b52e70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 -- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f30070 msgr2=0x5616b52e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f30070 0x5616b52e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 -- 172.21.15.92:0/2356001745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b5205c90 con 0x5616b4f0f390 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206b59e640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f30070 0x5616b52e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f206a59c640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e7620 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f20600cd9c0 tx=0x7f206003b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745605cd25d753d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f204eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f204eff5640 1 -- 172.21.15.92:0/2356001745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060049030 con 0x5616b4f0f390 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f204eff5640 1 -- 172.21.15.92:0/2356001745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f20600d6fc0 con 0x5616b4f0f390 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f205c019070 con 0x5616b4f0f390 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f204eff5640 1 -- 172.21.15.92:0/2356001745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f20600cd380 con 0x5616b4f0f390 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f20776fd740 10 client.16701 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.062+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616b52f5ea0 con 0x5616b4f0f390 2024-09-21T02:08:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f204eff5640 1 -- 172.21.15.92:0/2356001745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f205c019070 con 0x5616b4f0f390 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f204eff5640 1 -- 172.21.15.92:0/2356001745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f20600d2110 con 0x5616b4f0f390 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f20776fd740 10 client.16701 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f20776fd740 10 client.16701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f20776fd740 20 client.16701 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f20776fd740 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4eee9f0 0x5616b52f9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f20776fd740 4 client.16701 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b524fdf0 con 0x5616b4eee9f0 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.063+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4eee9f0 0x5616b52f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.064+0000 7f204eff5640 10 client.16701 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.064+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4eee9f0 0x5616b52f9340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4e69250c78a50c server_cookie=bf7956a3706b875b in_seq=0 out_seq=0 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.064+0000 7f204eff5640 1 -- 172.21.15.92:0/2356001745 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616b524fdf0 con 0x5616b4eee9f0 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.064+0000 7f204eff5640 10 client.16701 handle_command_reply: tid=0 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.064+0000 7f20776fd740 1 client.16701 shutdown 2024-09-21T02:08:57.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 msgr2=0x5616b52e7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e7620 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f20600cd9c0 tx=0x7f206003b690 comp rx=0 tx=0).stop 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 shutdown_connections 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f30070 0x5616b52e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4eee9f0 0x5616b52f9340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e7620 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 --2- 172.21.15.92:0/2356001745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f071b0 0x5616b52e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f204eff5640 0 client.16701 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 >> 172.21.15.92:0/2356001745 conn(0x5616b4f24460 msgr2=0x5616b52d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 shutdown_connections 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 1 -- 172.21.15.92:0/2356001745 wait complete. 2024-09-21T02:08:57.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.065+0000 7f20776fd740 20 client.16701 trim_cache size 0 max 16384 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 Processor -- start 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 -- start start 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52e4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1ad350 con 0x5616b4f0f390 2024-09-21T02:08:57.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1aee90 con 0x5616b4f071b0 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f20776fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1ada20 con 0x5616b4f30070 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49180/0 (socket says 172.21.15.92:49180) 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 -- 172.21.15.92:0/2242640804 learned_addr learned my addr 172.21.15.92:0/2242640804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206b59e640 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 -- 172.21.15.92:0/2242640804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 msgr2=0x5616b52e4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52e4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 -- 172.21.15.92:0/2242640804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 msgr2=0x5616b52dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52dcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 -- 172.21.15.92:0/2242640804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b52e9820 con 0x5616b4f0f390 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206b59e640 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f206a59c640 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e42d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2060003fd0 tx=0x7f2060005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127d494174b4060d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f2050ff9640 1 -- 172.21.15.92:0/2242640804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060049070 con 0x5616b4f0f390 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f2050ff9640 1 -- 172.21.15.92:0/2242640804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20600cd540 con 0x5616b4f0f390 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.079+0000 7f2050ff9640 1 -- 172.21.15.92:0/2242640804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20600cd1a0 con 0x5616b4f0f390 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 -- 172.21.15.92:0/2242640804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 msgr2=0x5616b52e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e42d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2060003fd0 tx=0x7f2060005a40 comp rx=0 tx=0).stop 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 -- 172.21.15.92:0/2242640804 shutdown_connections 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f30070 0x5616b52e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f0f390 0x5616b52e42d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 --2- 172.21.15.92:0/2242640804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 -- 172.21.15.92:0/2242640804 >> 172.21.15.92:0/2242640804 conn(0x5616b4f24460 msgr2=0x5616b52dc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 -- 172.21.15.92:0/2242640804 shutdown_connections 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 -- 172.21.15.92:0/2242640804 wait complete. 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 -- start start 2024-09-21T02:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.080+0000 7f20776fd740 1 Processor -- start 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f0f390 0x5616b52eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 0x5616b52ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1ae7c0 con 0x5616b4f30070 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1ae0f0 con 0x5616b4f071b0 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1ab330 con 0x5616b4f0f390 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f20776fd740 10 client.0 _ll_register_callbacks cb 0x7f2060005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 0x5616b52ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 0x5616b52ebeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49188/0 (socket says 172.21.15.92:49188) 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 -- 172.21.15.92:0/2163229851 learned_addr learned my addr 172.21.15.92:0/2163229851 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206a59c640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f0f390 0x5616b52eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 -- 172.21.15.92:0/2163229851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f0f390 msgr2=0x5616b52eb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f0f390 0x5616b52eb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 -- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 msgr2=0x5616b52eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 -- 172.21.15.92:0/2163229851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b5362d00 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206a59c640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f0f390 0x5616b52eb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f206b59e640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 0x5616b52ebeb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2064016730 tx=0x7f2064004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e08a5b55b4d907d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.081+0000 7f204eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f204eff5640 1 -- 172.21.15.92:0/2163229851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064028020 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20600cd540 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f204eff5640 1 -- 172.21.15.92:0/2163229851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2064017890 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f204eff5640 1 -- 172.21.15.92:0/2163229851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2064033050 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 10 client.16707 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616b5363f00 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f204eff5640 1 -- 172.21.15.92:0/2163229851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20600cd540 con 0x5616b4f30070 2024-09-21T02:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f204eff5640 1 -- 172.21.15.92:0/2163229851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2064024b70 con 0x5616b4f30070 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 10 client.16707 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 10 client.16707 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 20 client.16707 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4ec3da0 0x5616b5366770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 4 client.16707 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.082+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b524b8a0 con 0x5616b4ec3da0 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4ec3da0 0x5616b5366770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f204eff5640 10 client.16707 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f206ad9d640 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4ec3da0 0x5616b5366770 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed7b9863e4b8fea4 server_cookie=267d4102b879bd00 in_seq=0 out_seq=0 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f204eff5640 1 -- 172.21.15.92:0/2163229851 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b524b8a0 con 0x5616b4ec3da0 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f204eff5640 10 client.16707 handle_command_reply: tid=0 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 client.16707 shutdown 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 msgr2=0x5616b52ebeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 0x5616b52ebeb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2064016730 tx=0x7f2064004a40 comp rx=0 tx=0).stop 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 shutdown_connections 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5616b4ec3da0 0x5616b5366770 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616b4f30070 0x5616b52ebeb0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616b4f0f390 0x5616b52eb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f204eff5640 0 client.16707 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 --2- 172.21.15.92:0/2163229851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616b4f071b0 0x5616b52eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.083+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 >> 172.21.15.92:0/2163229851 conn(0x5616b4f24460 msgr2=0x5616b52dc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 shutdown_connections 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f20776fd740 1 -- 172.21.15.92:0/2163229851 wait complete. 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f20776fd740 20 client.16707 trim_cache size 0 max 16384 2024-09-21T02:08:57.092 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20540b6510 msgr2=0x7f20540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20540b6510 0x7f20540b8a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f206400a400 tx=0x7f206400d040 comp rx=0 tx=0).stop 2024-09-21T02:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 msgr2=0x7f206c19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c19ac00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2060031a90 tx=0x7f2060004270 comp rx=0 tx=0).stop 2024-09-21T02:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 shutdown_connections 2024-09-21T02:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f20540b6510 0x7f20540b8a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f206c108cf0 0x7f206c19fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f206c1083a0 0x7f206c19ac00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 --2- 172.21.15.92:0/2914383499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f206c102a60 0x7f206c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 >> 172.21.15.92:0/2914383499 conn(0x7f206c079730 msgr2=0x7f206c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 shutdown_connections 2024-09-21T02:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:57.084+0000 7f2071189640 1 -- 172.21.15.92:0/2914383499 wait complete. 2024-09-21T02:08:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:57 smithi136 ceph-mon[34754]: pgmap v479: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:57.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:57 smithi092 ceph-mon[31190]: pgmap v479: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:57 smithi099 ceph-mon[34404]: pgmap v479: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:08:57.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:08:58.090 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:08:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 -- 172.21.15.92:0/2110462062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c181081b0 msgr2=0x7f2c181085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 --2- 172.21.15.92:0/2110462062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c181081b0 0x7f2c181085c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802a410 comp rx=0 tx=0).stop 2024-09-21T02:08:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 -- 172.21.15.92:0/2110462062 shutdown_connections 2024-09-21T02:08:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 --2- 172.21.15.92:0/2110462062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c18108b00 0x7f2c1810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 --2- 172.21.15.92:0/2110462062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c181081b0 0x7f2c181085c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 --2- 172.21.15.92:0/2110462062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c18102870 0x7f2c18102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 -- 172.21.15.92:0/2110462062 >> 172.21.15.92:0/2110462062 conn(0x7f2c180fe2c0 msgr2=0x7f2c18100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 -- 172.21.15.92:0/2110462062 shutdown_connections 2024-09-21T02:08:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.581+0000 7f2c1759e640 1 -- 172.21.15.92:0/2110462062 wait complete. 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 Processor -- start 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 -- start start 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c18102870 0x7f2c1819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c181081b0 0x7f2c1819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 0x7f2c1819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18112770 con 0x7f2c18108b00 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181125f0 con 0x7f2c181081b0 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1759e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181128f0 con 0x7f2c18102870 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c16d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 0x7f2c1819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c16d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 0x7f2c1819fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49212/0 (socket says 172.21.15.92:49212) 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c16d9d640 1 -- 172.21.15.92:0/3874645175 learned_addr learned my addr 172.21.15.92:0/3874645175 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c181081b0 0x7f2c1819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.582+0000 7f2c1659c640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c18102870 0x7f2c1819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c16d9d640 1 -- 172.21.15.92:0/3874645175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c18102870 msgr2=0x7f2c1819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c16d9d640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c18102870 0x7f2c1819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c16d9d640 1 -- 172.21.15.92:0/3874645175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c181081b0 msgr2=0x7f2c1819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c16d9d640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c181081b0 0x7f2c1819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c16d9d640 1 -- 172.21.15.92:0/3874645175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181a0180 con 0x7f2c18108b00 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c16d9d640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 0x7f2c1819fb10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000cce0 tx=0x7f2c1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c0f7fe640 1 -- 172.21.15.92:0/3874645175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000f450 con 0x7f2c18108b00 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c0f7fe640 1 -- 172.21.15.92:0/3874645175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1000faa0 con 0x7f2c18108b00 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181a0470 con 0x7f2c18108b00 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c0f7fe640 1 -- 172.21.15.92:0/3874645175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10018b70 con 0x7f2c18108b00 2024-09-21T02:08:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.583+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a0930 con 0x7f2c18108b00 2024-09-21T02:08:58.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 Processor -- start 2024-09-21T02:08:58.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c0f7fe640 1 -- 172.21.15.92:0/3874645175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2c10018d10 con 0x7f2c18108b00 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 -- start start 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbff0070 0x559afc397860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc397da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559afc30dea0 con 0x559afbfcf390 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559afc2fd2d0 con 0x559afbff0070 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.586+0000 7f2c23378740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559afc3010e0 con 0x559afbfc71b0 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49226/0 (socket says 172.21.15.92:49226) 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 -- 172.21.15.92:0/303718838 learned_addr learned my addr 172.21.15.92:0/303718838 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbff0070 0x559afc397860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c1659c640 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc397da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c0f7fe640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2c040b6540 0x7f2c040b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 -- 172.21.15.92:0/303718838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 msgr2=0x559afc397da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc397da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 -- 172.21.15.92:0/303718838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbff0070 msgr2=0x559afc397860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbff0070 0x559afc397860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 -- 172.21.15.92:0/303718838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559afc05b650 con 0x559afbfcf390 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c16d9d640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2c040b6540 0x7f2c040b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3982e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2c080057c0 tx=0x7f2c08000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e082dfd60b3a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.587+0000 7f2c0f7fe640 1 -- 172.21.15.92:0/3874645175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2c100e76b0 con 0x7f2c18108b00 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.588+0000 7f2c0cff9640 1 -- 172.21.15.92:0/303718838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08035070 con 0x559afbfcf390 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.588+0000 7f2c0cff9640 1 -- 172.21.15.92:0/303718838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c08005030 con 0x559afbfcf390 2024-09-21T02:08:58.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.588+0000 7f2c0cff9640 1 -- 172.21.15.92:0/303718838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080044e0 con 0x559afbfcf390 2024-09-21T02:08:58.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.588+0000 7f2c23378740 1 -- 172.21.15.92:0/303718838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 msgr2=0x559afc3982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.588+0000 7f2c23378740 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3982e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2c080057c0 tx=0x7f2c08000bc0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.588+0000 7f2c16d9d640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2c040b6540 0x7f2c040b8a50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2c100ad7b0 tx=0x7f2c100de2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- 172.21.15.92:0/303718838 shutdown_connections 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3982e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc397da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 --2- 172.21.15.92:0/303718838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbff0070 0x559afc397860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- 172.21.15.92:0/303718838 >> 172.21.15.92:0/303718838 conn(0x559afbfe4460 msgr2=0x559afc395750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- 172.21.15.92:0/303718838 shutdown_connections 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- 172.21.15.92:0/303718838 wait complete. 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- start start 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 Processor -- start 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc3a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc3a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 0x559afc3af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559afc15e700 con 0x559afbff0070 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559afc298160 con 0x559afbfcf390 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559afc3a5fe0 con 0x559afbfc71b0 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.589+0000 7f2c23378740 10 client.0 _ll_register_callbacks cb 0x559afc2b19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 0x559afc3af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 0x559afc3af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49242/0 (socket says 172.21.15.92:49242) 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 -- 172.21.15.92:0/331687110 learned_addr learned my addr 172.21.15.92:0/331687110 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc3a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c1659c640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc3a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 -- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 msgr2=0x559afc3a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc3a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 -- 172.21.15.92:0/331687110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 msgr2=0x559afc3a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc3a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 -- 172.21.15.92:0/331687110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559afc2c5c90 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 0x559afc3af300 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2c080096c0 tx=0x7f2c0802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2c93e97494a324 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2bfdffb640 1 -- 172.21.15.92:0/331687110 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08035040 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2bfdffb640 1 -- 172.21.15.92:0/331687110 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2c080053e0 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c08005030 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2bfdffb640 1 -- 172.21.15.92:0/331687110 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c080055e0 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c23378740 10 client.16719 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.590+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559afc3b1d30 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2bfdffb640 1 -- 172.21.15.92:0/331687110 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c08005030 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2bfdffb640 1 -- 172.21.15.92:0/331687110 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c080035f0 con 0x559afbff0070 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c23378740 10 client.16719 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c23378740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:58.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c23378740 20 client.16719 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c23378740 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbfae9f0 0x559afc3b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c23378740 4 client.16719 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559afc30fdf0 con 0x559afbfae9f0 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.591+0000 7f2c1659c640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbfae9f0 0x559afc3b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.592+0000 7f2bfdffb640 10 client.16719 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.592+0000 7f2c1659c640 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbfae9f0 0x559afc3b5090 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c02c046814bb85 server_cookie=d2cdc4213adb3ee8 in_seq=0 out_seq=0 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2bfdffb640 1 -- 172.21.15.92:0/331687110 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559afc30fdf0 con 0x559afbfae9f0 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2bfdffb640 10 client.16719 handle_command_reply: tid=0 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 client.16719 shutdown 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 msgr2=0x559afc3af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 0x559afc3af300 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2c080096c0 tx=0x7f2c0802a410 comp rx=0 tx=0).stop 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 shutdown_connections 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbfae9f0 0x559afc3b5090 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbff0070 0x559afc3af300 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2bfdffb640 0 client.16719 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc3a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 --2- 172.21.15.92:0/331687110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc3a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 >> 172.21.15.92:0/331687110 conn(0x559afbfe4460 msgr2=0x559afc39a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 shutdown_connections 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 1 -- 172.21.15.92:0/331687110 wait complete. 2024-09-21T02:08:58.598 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.593+0000 7f2c23378740 20 client.16719 trim_cache size 0 max 16384 2024-09-21T02:08:58.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 Processor -- start 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 -- start start 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 0x559afc398fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc394d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc395260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181af2a0 con 0x559afbfae9f0 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181b0de0 con 0x559afbfcf390 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c23378740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181af970 con 0x559afbfc71b0 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 0x559afc398fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 0x559afc398fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49258/0 (socket says 172.21.15.92:49258) 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.606+0000 7f2c15d9b640 1 -- 172.21.15.92:0/232526918 learned_addr learned my addr 172.21.15.92:0/232526918 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2c15d9b640 1 -- 172.21.15.92:0/232526918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 msgr2=0x559afc394d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc394d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2c15d9b640 1 -- 172.21.15.92:0/232526918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 msgr2=0x559afc395260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc395260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2c15d9b640 1 -- 172.21.15.92:0/232526918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559afc39db90 con 0x559afbfae9f0 2024-09-21T02:08:58.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 0x559afc398fa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2c08003b30 tx=0x7f2c08009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aa6d76db4b5473 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:58.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2bfffff640 1 -- 172.21.15.92:0/232526918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08003bb0 con 0x559afbfae9f0 2024-09-21T02:08:58.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2bfffff640 1 -- 172.21.15.92:0/232526918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c080035f0 con 0x559afbfae9f0 2024-09-21T02:08:58.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.607+0000 7f2bfffff640 1 -- 172.21.15.92:0/232526918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08004460 con 0x559afbfae9f0 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 -- 172.21.15.92:0/232526918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 msgr2=0x559afc398fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 0x559afc398fa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2c08003b30 tx=0x7f2c08009850 comp rx=0 tx=0).stop 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 -- 172.21.15.92:0/232526918 shutdown_connections 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfcf390 0x559afc395260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfc71b0 0x559afc394d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 --2- 172.21.15.92:0/232526918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfae9f0 0x559afc398fa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 -- 172.21.15.92:0/232526918 >> 172.21.15.92:0/232526918 conn(0x559afbfe4460 msgr2=0x559afc398890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 -- 172.21.15.92:0/232526918 shutdown_connections 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 -- 172.21.15.92:0/232526918 wait complete. 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 -- start start 2024-09-21T02:08:58.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.608+0000 7f2c23378740 1 Processor -- start 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfae9f0 0x559afc3b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfc71b0 0x559afc3b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3b6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181b0710 con 0x559afbfcf390 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181b0040 con 0x559afbfc71b0 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181ad280 con 0x559afbfae9f0 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 10 client.0 _ll_register_callbacks cb 0x7f2c08032e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49268/0 (socket says 172.21.15.92:49268) 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 -- 172.21.15.92:0/2755645501 learned_addr learned my addr 172.21.15.92:0/2755645501 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c1659c640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfc71b0 0x559afc3b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfae9f0 0x559afc3b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 -- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfae9f0 msgr2=0x559afc3b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfae9f0 0x559afc3b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 -- 172.21.15.92:0/2755645501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfc71b0 msgr2=0x559afc3b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfc71b0 0x559afc3b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 -- 172.21.15.92:0/2755645501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559afc3b6580 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c15d9b640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3b6040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x559afc3bb4f0 tx=0x7f2c0805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60760268c559d6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2bfdffb640 1 -- 172.21.15.92:0/2755645501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08003bd0 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2bfdffb640 1 -- 172.21.15.92:0/2755645501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2c080046c0 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.609+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c080035f0 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2bfdffb640 1 -- 172.21.15.92:0/2755645501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c08031c50 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 10 client.16725 fetch_fsmap learned FSMap version 29 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559afc3b6580 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2bfdffb640 1 -- 172.21.15.92:0/2755645501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c080035f0 con 0x559afbfcf390 2024-09-21T02:08:58.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2bfdffb640 1 -- 172.21.15.92:0/2755645501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c0803d450 con 0x559afbfcf390 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 10 client.16725 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 20 client.16725 populate_metadata read hostname 'smithi092' 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbff0070 0x559afc4218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 4 client.16725 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559afc30b8a0 con 0x559afbff0070 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.610+0000 7f2c1659c640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbff0070 0x559afc4218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2bfdffb640 10 client.16725 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c1659c640 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbff0070 0x559afc4218d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f20ed91f930791 server_cookie=770374952ba9f60a in_seq=0 out_seq=0 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2bfdffb640 1 -- 172.21.15.92:0/2755645501 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559afc30b8a0 con 0x559afbff0070 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2bfdffb640 10 client.16725 handle_command_reply: tid=0 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 client.16725 shutdown 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 msgr2=0x559afc3b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3b6040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x559afc3bb4f0 tx=0x7f2c0805b040 comp rx=0 tx=0).stop 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 shutdown_connections 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559afbff0070 0x559afc4218d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559afbfcf390 0x559afc3b6040 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559afbfc71b0 0x559afc3b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 --2- 172.21.15.92:0/2755645501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559afbfae9f0 0x559afc3b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 >> 172.21.15.92:0/2755645501 conn(0x559afbfe4460 msgr2=0x559afc398890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2bfdffb640 0 client.16725 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 shutdown_connections 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 1 -- 172.21.15.92:0/2755645501 wait complete. 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.611+0000 7f2c23378740 20 client.16725 trim_cache size 0 max 16384 2024-09-21T02:08:58.617 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:08:58.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2c040b6540 msgr2=0x7f2c040b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2c040b6540 0x7f2c040b8a50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2c100ad7b0 tx=0x7f2c100de2c0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 msgr2=0x7f2c1819fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:08:58.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 0x7f2c1819fb10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000cce0 tx=0x7f2c1000a620 comp rx=0 tx=0).stop 2024-09-21T02:08:58.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 shutdown_connections 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2c040b6540 0x7f2c040b8a50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c18108b00 0x7f2c1819fb10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c181081b0 0x7f2c1819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 --2- 172.21.15.92:0/3874645175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c18102870 0x7f2c1819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 >> 172.21.15.92:0/3874645175 conn(0x7f2c180fe2c0 msgr2=0x7f2c181068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 shutdown_connections 2024-09-21T02:08:58.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:08:58.612+0000 7f2c1759e640 1 -- 172.21.15.92:0/3874645175 wait complete. 2024-09-21T02:08:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:08:59 smithi136 ceph-mon[34754]: pgmap v480: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:59.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:08:59 smithi092 ceph-mon[31190]: pgmap v480: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:08:59.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:08:59 smithi099 ceph-mon[34404]: pgmap v480: 97 pgs: 97 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:00.379 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:09:00.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:09:00.644 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:01.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:01 smithi092 ceph-mon[31190]: pgmap v481: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 -- 172.21.15.92:0/620382358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8068106c10 msgr2=0x7f8068110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 --2- 172.21.15.92:0/620382358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8068106c10 0x7f8068110650 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8060009900 tx=0x7f806002a410 comp rx=0 tx=0).stop 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 -- 172.21.15.92:0/620382358 shutdown_connections 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 --2- 172.21.15.92:0/620382358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8068106c10 0x7f8068110650 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 --2- 172.21.15.92:0/620382358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80681062c0 0x7f80681066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 --2- 172.21.15.92:0/620382358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681020d0 0x7f80681024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 -- 172.21.15.92:0/620382358 >> 172.21.15.92:0/620382358 conn(0x7f8068077cc0 msgr2=0x7f8068078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 -- 172.21.15.92:0/620382358 shutdown_connections 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.090+0000 7f8070498640 1 -- 172.21.15.92:0/620382358 wait complete. 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 Processor -- start 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 -- start start 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 0x7f806819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681062c0 0x7f806819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8068106c10 0x7f806819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8068113ba0 con 0x7f80681020d0 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8068113a20 con 0x7f80681062c0 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f8070498640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8068113d20 con 0x7f8068106c10 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f806e234640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 0x7f806819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f806e234640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 0x7f806819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49276/0 (socket says 172.21.15.92:49276) 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f806e234640 1 -- 172.21.15.92:0/2664766008 learned_addr learned my addr 172.21.15.92:0/2664766008 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f806da33640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681062c0 0x7f806819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.091+0000 7f806ea35640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8068106c10 0x7f806819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806e234640 1 -- 172.21.15.92:0/2664766008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8068106c10 msgr2=0x7f806819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806e234640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8068106c10 0x7f806819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806e234640 1 -- 172.21.15.92:0/2664766008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681062c0 msgr2=0x7f806819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806e234640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681062c0 0x7f806819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806e234640 1 -- 172.21.15.92:0/2664766008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80681a0500 con 0x7f80681020d0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806da33640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681062c0 0x7f806819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f806e234640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 0x7f806819a590 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f805800ba50 tx=0x7f805800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f80677fe640 1 -- 172.21.15.92:0/2664766008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058019070 con 0x7f80681020d0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80681a07f0 con 0x7f80681020d0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f80677fe640 1 -- 172.21.15.92:0/2664766008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805800cbe0 con 0x7f80681020d0 2024-09-21T02:09:01.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f80677fe640 1 -- 172.21.15.92:0/2664766008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580075d0 con 0x7f80681020d0 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.092+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80681a6f50 con 0x7f80681020d0 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 Processor -- start 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f80677fe640 1 -- 172.21.15.92:0/2664766008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f805801c070 con 0x7f80681020d0 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 -- start start 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e51643430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e51273490 0x559e51643970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e51643eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559e515b9e60 con 0x559e5127b670 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559e515a9290 con 0x559e5129c350 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f8076a0c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559e515ad0a0 con 0x559e51273490 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f806da33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e51643eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.095+0000 7f806da33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e51643eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49284/0 (socket says 172.21.15.92:49284) 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 -- 172.21.15.92:0/4244368612 learned_addr learned my addr 172.21.15.92:0/4244368612 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806e234640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e51273490 0x559e51643970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806ea35640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e51643430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 -- 172.21.15.92:0/4244368612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e51273490 msgr2=0x559e51643970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e51273490 0x559e51643970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 -- 172.21.15.92:0/4244368612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 msgr2=0x559e51643430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e51643430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 -- 172.21.15.92:0/4244368612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e51307930 con 0x559e5127b670 2024-09-21T02:09:01.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806e234640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e51273490 0x559e51643970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:01.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806ea35640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e51643430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f80677fe640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80500b6510 0x7f80500b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806da33640 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e51643eb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f805c0077a0 tx=0x7f805c007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afa994088e0844 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f806ea35640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80500b6510 0x7f80500b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f8064d9a640 1 -- 172.21.15.92:0/4244368612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00d950 con 0x559e5127b670 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f80677fe640 1 -- 172.21.15.92:0/2664766008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f80580e68b0 con 0x7f80681020d0 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f8064d9a640 1 -- 172.21.15.92:0/4244368612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c018070 con 0x559e5127b670 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.096+0000 7f8064d9a640 1 -- 172.21.15.92:0/4244368612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c013e10 con 0x559e5127b670 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 -- 172.21.15.92:0/4244368612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 msgr2=0x559e51643eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f806ea35640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80500b6510 0x7f80500b8a20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8060005940 tx=0x7f80600058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:01.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e51643eb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f805c0077a0 tx=0x7f805c007c60 comp rx=0 tx=0).stop 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 -- 172.21.15.92:0/4244368612 shutdown_connections 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e51643eb0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e51273490 0x559e51643970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 --2- 172.21.15.92:0/4244368612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e51643430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.097+0000 7f8076a0c740 1 -- 172.21.15.92:0/4244368612 >> 172.21.15.92:0/4244368612 conn(0x559e51290740 msgr2=0x559e51641320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 -- 172.21.15.92:0/4244368612 shutdown_connections 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 -- 172.21.15.92:0/4244368612 wait complete. 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 -- start start 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 Processor -- start 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e51273490 0x559e5164f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e5164f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e5165af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559e5140a890 con 0x559e5127b670 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559e51543d40 con 0x559e51273490 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559e51651c10 con 0x559e5129c350 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f8076a0c740 10 client.0 _ll_register_callbacks cb 0x559e515a7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.098+0000 7f806da33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e5164f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e5164f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49290/0 (socket says 172.21.15.92:49290) 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 -- 172.21.15.92:0/1159433712 learned_addr learned my addr 172.21.15.92:0/1159433712 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806e234640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e51273490 0x559e5164f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806ea35640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e5165af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 -- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 msgr2=0x559e5165af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e5165af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 -- 172.21.15.92:0/1159433712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e51273490 msgr2=0x559e5164f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e51273490 0x559e5164f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 -- 172.21.15.92:0/1159433712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5165b510 con 0x559e5127b670 2024-09-21T02:09:01.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806e234640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e51273490 0x559e5164f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f806da33640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e5164f5b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f805c013a30 tx=0x7f805c013f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805eac0c0d6a2eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f804dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f804dffb640 1 -- 172.21.15.92:0/1159433712 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00db50 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f804dffb640 1 -- 172.21.15.92:0/1159433712 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f805c00dcf0 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c018070 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f804dffb640 1 -- 172.21.15.92:0/1159433712 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f805c024420 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f8076a0c740 10 client.16737 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.099+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e5165de40 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f804dffb640 1 -- 172.21.15.92:0/1159433712 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c018070 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f804dffb640 1 -- 172.21.15.92:0/1159433712 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f805c01c3f0 con 0x559e5127b670 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f8076a0c740 10 client.16737 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f8076a0c740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:01.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f8076a0c740 20 client.16737 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f8076a0c740 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e5125acd0 0x559e51661200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f8076a0c740 4 client.16737 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e515bbdb0 con 0x559e5125acd0 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.100+0000 7f806e234640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e5125acd0 0x559e51661200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.101+0000 7f804dffb640 10 client.16737 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.101+0000 7f806e234640 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e5125acd0 0x559e51661200 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c473e4e61b1d02 server_cookie=66d953ee70c7002f in_seq=0 out_seq=0 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.101+0000 7f804dffb640 1 -- 172.21.15.92:0/1159433712 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e515bbdb0 con 0x559e5125acd0 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.101+0000 7f804dffb640 10 client.16737 handle_command_reply: tid=0 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.101+0000 7f8076a0c740 1 client.16737 shutdown 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 msgr2=0x559e5164f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e5164f5b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f805c013a30 tx=0x7f805c013f00 comp rx=0 tx=0).stop 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 shutdown_connections 2024-09-21T02:09:01.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e5125acd0 0x559e51661200 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e5165af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e5127b670 0x559e5164f5b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 --2- 172.21.15.92:0/1159433712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e51273490 0x559e5164f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 >> 172.21.15.92:0/1159433712 conn(0x559e51290740 msgr2=0x559e51646610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f804dffb640 0 client.16737 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 shutdown_connections 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 1 -- 172.21.15.92:0/1159433712 wait complete. 2024-09-21T02:09:01.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.102+0000 7f8076a0c740 20 client.16737 trim_cache size 0 max 16384 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.119+0000 7f8076a0c740 1 Processor -- start 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.119+0000 7f8076a0c740 1 -- start start 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e5165ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5127b670 0x559e51648c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e5164e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f8076a0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80681af4f0 con 0x559e51273490 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f8076a0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80681b1030 con 0x559e5129c350 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f8076a0c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80681afbc0 con 0x559e5127b670 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e5165ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e5165ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49294/0 (socket says 172.21.15.92:49294) 2024-09-21T02:09:01.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 -- 172.21.15.92:0/17894386 learned_addr learned my addr 172.21.15.92:0/17894386 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 -- 172.21.15.92:0/17894386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5127b670 msgr2=0x559e51648c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5127b670 0x559e51648c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 -- 172.21.15.92:0/17894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 msgr2=0x559e5164e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e5164e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 -- 172.21.15.92:0/17894386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e51645b80 con 0x559e51273490 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f806e234640 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e5165ae70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f80580a8ff0 tx=0x7f80580e4260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aea5a2bb3e2f1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f804ffff640 1 -- 172.21.15.92:0/17894386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580d00c0 con 0x559e51273490 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f804ffff640 1 -- 172.21.15.92:0/17894386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80580ad120 con 0x559e51273490 2024-09-21T02:09:01.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.120+0000 7f804ffff640 1 -- 172.21.15.92:0/17894386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580a8370 con 0x559e51273490 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 -- 172.21.15.92:0/17894386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 msgr2=0x559e5165ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e5165ae70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f80580a8ff0 tx=0x7f80580e4260 comp rx=0 tx=0).stop 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 -- 172.21.15.92:0/17894386 shutdown_connections 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5129c350 0x559e5164e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5127b670 0x559e51648c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 --2- 172.21.15.92:0/17894386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e5165ae70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 -- 172.21.15.92:0/17894386 >> 172.21.15.92:0/17894386 conn(0x559e51290740 msgr2=0x559e51648540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 -- 172.21.15.92:0/17894386 shutdown_connections 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.121+0000 7f8076a0c740 1 -- 172.21.15.92:0/17894386 wait complete. 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 -- start start 2024-09-21T02:09:01.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 Processor -- start 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e51662340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5127b670 0x559e51662880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e51662dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80681b0960 con 0x559e51273490 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80681b0290 con 0x559e5127b670 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80681ad4d0 con 0x559e5129c350 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f8076a0c740 10 client.0 _ll_register_callbacks cb 0x7f805c0075f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e51662340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e51662340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49310/0 (socket says 172.21.15.92:49310) 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 -- 172.21.15.92:0/2720291903 learned_addr learned my addr 172.21.15.92:0/2720291903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806ea35640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e51662dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806da33640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5127b670 0x559e51662880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 -- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 msgr2=0x559e51662dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e51662dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 -- 172.21.15.92:0/2720291903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5127b670 msgr2=0x559e51662880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5127b670 0x559e51662880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.122+0000 7f806e234640 1 -- 172.21.15.92:0/2720291903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e516caab0 con 0x559e51273490 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f806e234640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e51662340 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f80580ac720 tx=0x7f8058014c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d00c782286a608 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f804dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f804dffb640 1 -- 172.21.15.92:0/2720291903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580a8c90 con 0x559e51273490 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f804dffb640 1 -- 172.21.15.92:0/2720291903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f80580abc80 con 0x559e51273490 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80580ad120 con 0x559e51273490 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f804dffb640 1 -- 172.21.15.92:0/2720291903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80580a8e30 con 0x559e51273490 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 10 client.16743 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:01.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e516cbd50 con 0x559e51273490 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f804dffb640 1 -- 172.21.15.92:0/2720291903 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80580ad120 con 0x559e51273490 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f804dffb640 1 -- 172.21.15.92:0/2720291903 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f80580ab6f0 con 0x559e51273490 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 10 client.16743 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 20 client.16743 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e51230080 0x559e516ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 4 client.16743 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.123+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e515b7860 con 0x559e51230080 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.124+0000 7f806da33640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e51230080 0x559e516ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.124+0000 7f806da33640 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e51230080 0x559e516ce560 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbedf8dbd925f5ad server_cookie=619ba7d3dde16a00 in_seq=0 out_seq=0 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.124+0000 7f804dffb640 10 client.16743 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f804dffb640 1 -- 172.21.15.92:0/2720291903 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e515b7860 con 0x559e51230080 2024-09-21T02:09:01.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f804dffb640 10 client.16743 handle_command_reply: tid=0 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 client.16743 shutdown 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 msgr2=0x559e51662340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e51662340 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f80580ac720 tx=0x7f8058014c90 comp rx=0 tx=0).stop 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 shutdown_connections 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559e51230080 0x559e516ce560 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e5129c350 0x559e51662dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e5127b670 0x559e51662880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 --2- 172.21.15.92:0/2720291903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559e51273490 0x559e51662340 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f804dffb640 0 client.16743 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:01.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 >> 172.21.15.92:0/2720291903 conn(0x559e51290740 msgr2=0x559e51644460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:01.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 shutdown_connections 2024-09-21T02:09:01.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 1 -- 172.21.15.92:0/2720291903 wait complete. 2024-09-21T02:09:01.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.126+0000 7f8076a0c740 20 client.16743 trim_cache size 0 max 16384 2024-09-21T02:09:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "3b26b928-2f6b-4636-9658-0e6fe939ca98", 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80500b6510 msgr2=0x7f80500b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80500b6510 0x7f80500b8a20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8060005940 tx=0x7f80600058b0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 msgr2=0x7f806819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 0x7f806819a590 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f805800ba50 tx=0x7f805800bf10 comp rx=0 tx=0).stop 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 shutdown_connections 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80500b6510 0x7f80500b8a20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8068106c10 0x7f806819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80681062c0 0x7f806819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 --2- 172.21.15.92:0/2664766008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80681020d0 0x7f806819a590 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 >> 172.21.15.92:0/2664766008 conn(0x7f8068077cc0 msgr2=0x7f80681085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 shutdown_connections 2024-09-21T02:09:01.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:01.127+0000 7f8070498640 1 -- 172.21.15.92:0/2664766008 wait complete. 2024-09-21T02:09:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:01 smithi136 ceph-mon[34754]: pgmap v481: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:01 smithi099 ceph-mon[34404]: pgmap v481: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:01.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:09:02.227 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:02 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:09:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:02 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:02 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:09:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:02 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:02.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:02 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:09:02.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:02 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:02.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:02 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:09:02.306 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:02 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:02 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:09:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:02 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:02 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:09:02.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:02 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 -- 172.21.15.92:0/4167867704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 msgr2=0x7fba30108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 --2- 172.21.15.92:0/4167867704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30108830 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fba20009900 tx=0x7fba2002a410 comp rx=0 tx=0).stop 2024-09-21T02:09:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 -- 172.21.15.92:0/4167867704 shutdown_connections 2024-09-21T02:09:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 --2- 172.21.15.92:0/4167867704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba30108d70 0x7fba3010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 --2- 172.21.15.92:0/4167867704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30108830 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 --2- 172.21.15.92:0/4167867704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba30102ae0 0x7fba30102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 -- 172.21.15.92:0/4167867704 >> 172.21.15.92:0/4167867704 conn(0x7fba30079770 msgr2=0x7fba30079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 -- 172.21.15.92:0/4167867704 shutdown_connections 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.669+0000 7fba36f10640 1 -- 172.21.15.92:0/4167867704 wait complete. 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 Processor -- start 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 -- start start 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba30102ae0 0x7fba30199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba30108d70 0x7fba3019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba30112a30 con 0x7fba30108420 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba301128b0 con 0x7fba30108d70 2024-09-21T02:09:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba36f10640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba30112bb0 con 0x7fba30102ae0 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.670+0000 7fba27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49324/0 (socket says 172.21.15.92:49324) 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 -- 172.21.15.92:0/2700478349 learned_addr learned my addr 172.21.15.92:0/2700478349 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba34cac640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba30102ae0 0x7fba30199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba354ad640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba30108d70 0x7fba3019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 -- 172.21.15.92:0/2700478349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba30102ae0 msgr2=0x7fba30199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba30102ae0 0x7fba30199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 -- 172.21.15.92:0/2700478349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba30108d70 msgr2=0x7fba3019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba30108d70 0x7fba3019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 -- 172.21.15.92:0/2700478349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3019f330 con 0x7fba30108420 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba27fff640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30199a20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba2002a930 tx=0x7fba20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:02.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba25ffb640 1 -- 172.21.15.92:0/2700478349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2003f030 con 0x7fba30108420 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3019f5c0 con 0x7fba30108420 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba25ffb640 1 -- 172.21.15.92:0/2700478349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba20003aa0 con 0x7fba30108420 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba25ffb640 1 -- 172.21.15.92:0/2700478349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20035070 con 0x7fba30108420 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.671+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3019faf0 con 0x7fba30108420 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.674+0000 7fba25ffb640 1 -- 172.21.15.92:0/2700478349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fba20042030 con 0x7fba30108420 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 Processor -- start 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 -- start start 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba25ffb640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba180b6540 0x7fba180b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c04c070 0x55926c3f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c3f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c02b390 0x55926c3f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55926c369e50 con 0x55926c0231b0 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55926c359280 con 0x55926c04c070 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba3d484740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55926c35d090 con 0x55926c02b390 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba354ad640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba180b6540 0x7fba180b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba25ffb640 1 -- 172.21.15.92:0/2700478349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fba20107af0 con 0x7fba30108420 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c3f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c3f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49332/0 (socket says 172.21.15.92:49332) 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.675+0000 7fba27fff640 1 -- 172.21.15.92:0/1143220744 learned_addr learned my addr 172.21.15.92:0/1143220744 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba34cac640 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c04c070 0x55926c3f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba354ad640 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c02b390 0x55926c3f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba27fff640 1 -- 172.21.15.92:0/1143220744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c02b390 msgr2=0x55926c3f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba27fff640 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c02b390 0x55926c3f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba27fff640 1 -- 172.21.15.92:0/1143220744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c04c070 msgr2=0x55926c3f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba27fff640 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c04c070 0x55926c3f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba27fff640 1 -- 172.21.15.92:0/1143220744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926c0b7650 con 0x55926c0231b0 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba27fff640 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c3f5da0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba200caed0 tx=0x7fba20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3b82a99d143589 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba354ad640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba180b6540 0x7fba180b8a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba28007f10 tx=0x7fba280127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba16ffd640 1 -- 172.21.15.92:0/1143220744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20049070 con 0x55926c0231b0 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba16ffd640 1 -- 172.21.15.92:0/1143220744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba200cd770 con 0x55926c0231b0 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.676+0000 7fba16ffd640 1 -- 172.21.15.92:0/1143220744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2003bdc0 con 0x55926c0231b0 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 -- 172.21.15.92:0/1143220744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 msgr2=0x55926c3f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c3f5da0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba200caed0 tx=0x7fba20000bc0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 -- 172.21.15.92:0/1143220744 shutdown_connections 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c02b390 0x55926c3f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c3f5da0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 --2- 172.21.15.92:0/1143220744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c04c070 0x55926c3f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 -- 172.21.15.92:0/1143220744 >> 172.21.15.92:0/1143220744 conn(0x55926c040460 msgr2=0x55926c3f3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 -- 172.21.15.92:0/1143220744 shutdown_connections 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 -- 172.21.15.92:0/1143220744 wait complete. 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 -- start start 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.677+0000 7fba3d484740 1 Processor -- start 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3fef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c3ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 0x55926c40f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55926c1ba700 con 0x55926c04c070 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55926c2f4110 con 0x55926c02b390 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55926c3f0e70 con 0x55926c0231b0 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba3d484740 10 client.0 _ll_register_callbacks cb 0x55926c30d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 0x55926c40f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 0x55926c40f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49346/0 (socket says 172.21.15.92:49346) 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 -- 172.21.15.92:0/2311838446 learned_addr learned my addr 172.21.15.92:0/2311838446 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba27fff640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c3ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba34cac640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 -- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 msgr2=0x55926c3fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3fef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 -- 172.21.15.92:0/2311838446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 msgr2=0x55926c3ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c3ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba354ad640 1 -- 172.21.15.92:0/2311838446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926c321c40 con 0x55926c04c070 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.678+0000 7fba27fff640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c3ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba354ad640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 0x55926c40f190 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba28010dc0 tx=0x7fba28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4e7e25653e5609 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:02.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba0ffff640 1 -- 172.21.15.92:0/2311838446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28024070 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba0ffff640 1 -- 172.21.15.92:0/2311838446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fba2801e670 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba200cd770 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba0ffff640 1 -- 172.21.15.92:0/2311838446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba28030050 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 10 client.16755 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55926c411c30 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba0ffff640 1 -- 172.21.15.92:0/2311838446 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba200cd770 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba0ffff640 1 -- 172.21.15.92:0/2311838446 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba2801c040 con 0x55926c04c070 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 20 client.16755 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c00a9f0 0x55926c4150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 4 client.16755 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.679+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55926c36bda0 con 0x55926c00a9f0 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.680+0000 7fba34cac640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c00a9f0 0x55926c4150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.680+0000 7fba0ffff640 10 client.16755 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.680+0000 7fba34cac640 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c00a9f0 0x55926c4150b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29efee0fe669b0b server_cookie=956fda9aef34e6a2 in_seq=0 out_seq=0 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba0ffff640 1 -- 172.21.15.92:0/2311838446 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55926c36bda0 con 0x55926c00a9f0 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba0ffff640 10 client.16755 handle_command_reply: tid=0 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 client.16755 shutdown 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 msgr2=0x55926c40f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 0x55926c40f190 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba28010dc0 tx=0x7fba28004ab0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 shutdown_connections 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c00a9f0 0x55926c4150b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c04c070 0x55926c40f190 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba0ffff640 0 client.16755 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:02.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c3ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 --2- 172.21.15.92:0/2311838446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.681+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 >> 172.21.15.92:0/2311838446 conn(0x55926c040460 msgr2=0x55926c3f6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.682+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 shutdown_connections 2024-09-21T02:09:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.682+0000 7fba3d484740 1 -- 172.21.15.92:0/2311838446 wait complete. 2024-09-21T02:09:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.682+0000 7fba3d484740 20 client.16755 trim_cache size 0 max 16384 2024-09-21T02:09:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 Processor -- start 2024-09-21T02:09:02.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 -- start start 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c00a9f0 0x55926c3ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 0x55926c3f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba301ad1e0 con 0x55926c02b390 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba301aed20 con 0x55926c00a9f0 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba3d484740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba301ad8b0 con 0x55926c0231b0 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.699+0000 7fba354ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 0x55926c3f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 0x55926c3f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49352/0 (socket says 172.21.15.92:49352) 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 -- 172.21.15.92:0/1332214341 learned_addr learned my addr 172.21.15.92:0/1332214341 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba27fff640 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba34cac640 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c00a9f0 0x55926c3ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 -- 172.21.15.92:0/1332214341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 msgr2=0x55926c3f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 -- 172.21.15.92:0/1332214341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c00a9f0 msgr2=0x55926c3ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c00a9f0 0x55926c3ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 -- 172.21.15.92:0/1332214341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926c3f7b50 con 0x55926c02b390 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba34cac640 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c00a9f0 0x55926c3ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba354ad640 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 0x55926c3f25d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fba2801d890 tx=0x7fba28010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a079752b0a13913a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba167fc640 1 -- 172.21.15.92:0/1332214341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2802d020 con 0x55926c02b390 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba167fc640 1 -- 172.21.15.92:0/1332214341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2800dba0 con 0x55926c02b390 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.700+0000 7fba167fc640 1 -- 172.21.15.92:0/1332214341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28024040 con 0x55926c02b390 2024-09-21T02:09:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 -- 172.21.15.92:0/1332214341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 msgr2=0x55926c3f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 0x55926c3f25d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fba2801d890 tx=0x7fba28010aa0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 -- 172.21.15.92:0/1332214341 shutdown_connections 2024-09-21T02:09:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c02b390 0x55926c3f25d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c0231b0 0x55926c3f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 --2- 172.21.15.92:0/1332214341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c00a9f0 0x55926c3ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 -- 172.21.15.92:0/1332214341 >> 172.21.15.92:0/1332214341 conn(0x55926c040460 msgr2=0x55926c3fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 -- 172.21.15.92:0/1332214341 shutdown_connections 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 -- 172.21.15.92:0/1332214341 wait complete. 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.701+0000 7fba3d484740 1 -- start start 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 Processor -- start 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c00a9f0 0x55926c408f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c4094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c4099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba301ae650 con 0x55926c0231b0 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba301adf80 con 0x55926c02b390 2024-09-21T02:09:02.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba301ab1c0 con 0x55926c00a9f0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba3d484740 10 client.0 _ll_register_callbacks cb 0x7fba28010370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c4094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c4094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49360/0 (socket says 172.21.15.92:49360) 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 -- 172.21.15.92:0/504667153 learned_addr learned my addr 172.21.15.92:0/504667153 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba34cac640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c00a9f0 0x55926c408f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba354ad640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c4099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 -- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c00a9f0 msgr2=0x55926c408f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c00a9f0 0x55926c408f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 -- 172.21.15.92:0/504667153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 msgr2=0x55926c4099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.702+0000 7fba27fff640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c4099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba27fff640 1 -- 172.21.15.92:0/504667153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926c40e900 con 0x55926c0231b0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba34cac640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c00a9f0 0x55926c408f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba27fff640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c4094b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fba200cdbf0 tx=0x7fba2003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc48bae124aa3802 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:02.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba0ffff640 1 -- 172.21.15.92:0/504667153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20049040 con 0x55926c0231b0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba0ffff640 1 -- 172.21.15.92:0/504667153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fba200b5070 con 0x55926c0231b0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba2800dba0 con 0x55926c0231b0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba0ffff640 1 -- 172.21.15.92:0/504667153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba200ce360 con 0x55926c0231b0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba3d484740 10 client.16761 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55926c47ff10 con 0x55926c0231b0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba0ffff640 1 -- 172.21.15.92:0/504667153 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2800dba0 con 0x55926c0231b0 2024-09-21T02:09:02.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.703+0000 7fba0ffff640 1 -- 172.21.15.92:0/504667153 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba200c9ed0 con 0x55926c0231b0 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba3d484740 10 client.16761 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba3d484740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba3d484740 20 client.16761 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba3d484740 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c04c070 0x55926c4825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba3d484740 4 client.16761 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55926c367850 con 0x55926c04c070 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba34cac640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c04c070 0x55926c4825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba0ffff640 10 client.16761 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.704+0000 7fba34cac640 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c04c070 0x55926c4825e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a21d5815e095864 server_cookie=3aeaa20698bd2ac2 in_seq=0 out_seq=0 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba0ffff640 1 -- 172.21.15.92:0/504667153 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55926c367850 con 0x55926c04c070 2024-09-21T02:09:02.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba0ffff640 10 client.16761 handle_command_reply: tid=0 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 client.16761 shutdown 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 msgr2=0x55926c4094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c4094b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fba200cdbf0 tx=0x7fba2003b6b0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 shutdown_connections 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55926c04c070 0x55926c4825e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55926c02b390 0x55926c4099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55926c0231b0 0x55926c4094b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 --2- 172.21.15.92:0/504667153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55926c00a9f0 0x55926c408f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 >> 172.21.15.92:0/504667153 conn(0x55926c040460 msgr2=0x55926c3f2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:02.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba0ffff640 0 client.16761 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:02.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 shutdown_connections 2024-09-21T02:09:02.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 1 -- 172.21.15.92:0/504667153 wait complete. 2024-09-21T02:09:02.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.705+0000 7fba3d484740 20 client.16761 trim_cache size 0 max 16384 2024-09-21T02:09:02.719 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:09:02.719 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "3b26b928-2f6b-4636-9658-0e6fe939ca98", 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:09:02.720 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.721 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.722 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:09:02.723 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:02.724 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:02.725 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:02.725 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:02.725 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:02.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:02.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:02.725 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:09:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba180b6540 msgr2=0x7fba180b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba180b6540 0x7fba180b8a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba28007f10 tx=0x7fba280127c0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 msgr2=0x7fba30199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30199a20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba2002a930 tx=0x7fba20004270 comp rx=0 tx=0).stop 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 shutdown_connections 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fba180b6540 0x7fba180b8a50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba30108d70 0x7fba3019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba30108420 0x7fba30199a20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 --2- 172.21.15.92:0/2700478349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba30102ae0 0x7fba30199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 >> 172.21.15.92:0/2700478349 conn(0x7fba30079770 msgr2=0x7fba301044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 shutdown_connections 2024-09-21T02:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:02.706+0000 7fba36f10640 1 -- 172.21.15.92:0/2700478349 wait complete. 2024-09-21T02:09:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:03 smithi136 ceph-mon[34754]: pgmap v482: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:03.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:03 smithi092 ceph-mon[31190]: pgmap v482: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:03.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:03 smithi099 ceph-mon[34404]: pgmap v482: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:03.598 INFO:tasks.cephfs.filesystem:scrub status for tag:3b26b928-2f6b-4636-9658-0e6fe939ca98 - {'path': '/', 'tag': '3b26b928-2f6b-4636-9658-0e6fe939ca98', 'options': 'recursive,force'} 2024-09-21T02:09:04.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:04 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:09:04.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:04 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:09:04.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:04 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 -- 172.21.15.92:0/2718944173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103fd0 msgr2=0x7fb9e810a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 --2- 172.21.15.92:0/2718944173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103fd0 0x7fb9e810a970 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009ca0 tx=0x7fb9e002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 -- 172.21.15.92:0/2718944173 shutdown_connections 2024-09-21T02:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 --2- 172.21.15.92:0/2718944173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103fd0 0x7fb9e810a970 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 --2- 172.21.15.92:0/2718944173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9e8103630 0x7fb9e8103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 --2- 172.21.15.92:0/2718944173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e81025b0 0x7fb9e8102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 -- 172.21.15.92:0/2718944173 >> 172.21.15.92:0/2718944173 conn(0x7fb9e80fdf30 msgr2=0x7fb9e81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 -- 172.21.15.92:0/2718944173 shutdown_connections 2024-09-21T02:09:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.916+0000 7fb9edb98640 1 -- 172.21.15.92:0/2718944173 wait complete. 2024-09-21T02:09:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 Processor -- start 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 -- start start 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e81025b0 0x7fb9e8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 0x7fb9e8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9e8103fd0 0x7fb9e819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e810e210 con 0x7fb9e8103630 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e810e090 con 0x7fb9e81025b0 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9edb98640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e810e390 con 0x7fb9e8103fd0 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 0x7fb9e8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 0x7fb9e8199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37820/0 (socket says 172.21.15.92:37820) 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.917+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1198971527 learned_addr learned my addr 172.21.15.92:0/1198971527 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9e8103fd0 0x7fb9e819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e81025b0 0x7fb9e8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1198971527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9e8103fd0 msgr2=0x7fb9e819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9e8103fd0 0x7fb9e819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1198971527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e81025b0 msgr2=0x7fb9e8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e81025b0 0x7fb9e8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1198971527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e819a840 con 0x7fb9e8103630 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 0x7fb9e8199470 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00e9f0 tx=0x7fb9dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e4ff9640 1 -- 172.21.15.92:0/1198971527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00cd80 con 0x7fb9e8103630 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e4ff9640 1 -- 172.21.15.92:0/1198971527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9dc0054d0 con 0x7fb9e8103630 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e819ab30 con 0x7fb9e8103630 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9e4ff9640 1 -- 172.21.15.92:0/1198971527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc010690 con 0x7fb9e8103630 2024-09-21T02:09:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.918+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81a1220 con 0x7fb9e8103630 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 Processor -- start 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 -- start start 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e4ff9640 1 -- 172.21.15.92:0/1198971527 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb9dc004e90 con 0x7fb9e8103630 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb7171b0 0x561eebac1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6ee2f0 0x561eebac16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 0x561eebac2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561eeba316f0 con 0x561eeb6f64d0 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561eeba20b20 con 0x561eeb6ee2f0 2024-09-21T02:09:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561eeba24930 con 0x561eeb7171b0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 0x561eebac2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 0x561eebac2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37824/0 (socket says 172.21.15.92:37824) 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/4075039656 learned_addr learned my addr 172.21.15.92:0/4075039656 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb7171b0 0x561eebac1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6ee2f0 0x561eebac16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/4075039656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb7171b0 msgr2=0x561eebac1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb7171b0 0x561eebac1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/4075039656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6ee2f0 msgr2=0x561eebac16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6ee2f0 0x561eebac16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.921+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/4075039656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eeb9e6530 con 0x561eeb6f64d0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 0x561eebac2220 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0eda80 tx=0x7fb9dc0edb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bd81e05a402fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9e4ff9640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb9d00b6510 0x7fb9d00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9cdffb640 1 -- 172.21.15.92:0/4075039656 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0f6070 con 0x561eeb6f64d0 2024-09-21T02:09:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb9d00b6510 0x7fb9d00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9cdffb640 1 -- 172.21.15.92:0/4075039656 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9dc071350 con 0x561eeb6f64d0 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9cdffb640 1 -- 172.21.15.92:0/4075039656 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0acfa0 con 0x561eeb6f64d0 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.922+0000 7fb9e4ff9640 1 -- 172.21.15.92:0/1198971527 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb9dc0e76c0 con 0x7fb9e8103630 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4075039656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 msgr2=0x561eebac2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb9d00b6510 0x7fb9d00b8a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0004af0 tx=0x7fb9e0009380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 0x561eebac2220 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0eda80 tx=0x7fb9dc0edb70 comp rx=0 tx=0).stop 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4075039656 shutdown_connections 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6f64d0 0x561eebac2220 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6ee2f0 0x561eebac16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4075039656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb7171b0 0x561eebac1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4075039656 >> 172.21.15.92:0/4075039656 conn(0x561eeb70b300 msgr2=0x561eebabcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4075039656 shutdown_connections 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4075039656 wait complete. 2024-09-21T02:09:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.923+0000 7fb9f4be2740 1 -- start start 2024-09-21T02:09:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 Processor -- start 2024-09-21T02:09:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebaca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebacae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb7171b0 0x561eebacb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561eeb958ab0 con 0x561eeb6ee2f0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561eeb9c6730 con 0x561eeb7171b0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561eebac2d40 con 0x561eeb6f64d0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9f4be2740 10 client.0 _ll_register_callbacks cb 0x561eeba1a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebaca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebaca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37834/0 (socket says 172.21.15.92:37834) 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 -- 172.21.15.92:0/4214654428 learned_addr learned my addr 172.21.15.92:0/4214654428 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb7171b0 0x561eebacb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebacae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 -- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 msgr2=0x561eebacae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebacae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 -- 172.21.15.92:0/4214654428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb7171b0 msgr2=0x561eebacb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb7171b0 0x561eebacb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 -- 172.21.15.92:0/4214654428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eeb782790 con 0x561eeb6ee2f0 2024-09-21T02:09:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebacae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebaca770 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800bfb0 tx=0x7fb9d80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90459e55d4282664 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.924+0000 7fb9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/4214654428 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8005220 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/4214654428 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb9d80053c0 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9dc071350 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/4214654428 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb9d80055c0 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 10 client.16773 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561eeb7fb620 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/4214654428 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9dc071350 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/4214654428 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb9d8010c30 con 0x561eeb6ee2f0 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 10 client.16773 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 20 client.16773 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb6d5d80 0x561eebad7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 4 client.16773 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.925+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eeba33640 con 0x561eeb6d5d80 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.926+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb6d5d80 0x561eebad7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.926+0000 7fb9c6ffd640 10 client.16773 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.926+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb6d5d80 0x561eebad7fd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c5719dd639c84d server_cookie=95fd3d4c71944302 in_seq=0 out_seq=0 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/4214654428 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561eeba33640 con 0x561eeb6d5d80 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9c6ffd640 10 client.16773 handle_command_reply: tid=0 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 client.16773 shutdown 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 msgr2=0x561eebaca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebaca770 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800bfb0 tx=0x7fb9d80128e0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 shutdown_connections 2024-09-21T02:09:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb6d5d80 0x561eebad7fd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb7171b0 0x561eebacb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebacae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9c6ffd640 0 client.16773 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/4214654428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebaca770 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 >> 172.21.15.92:0/4214654428 conn(0x561eeb70b300 msgr2=0x561eebab8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 shutdown_connections 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 1 -- 172.21.15.92:0/4214654428 wait complete. 2024-09-21T02:09:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.927+0000 7fb9f4be2740 20 client.16773 trim_cache size 0 max 16384 2024-09-21T02:09:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 Processor -- start 2024-09-21T02:09:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 -- start start 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6d5d80 0x561eebac1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebac1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebac91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81a8e00 con 0x561eeb6ee2f0 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81aa940 con 0x561eeb6d5d80 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81a94d0 con 0x561eeb6f64d0 2024-09-21T02:09:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebac1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebac1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37850/0 (socket says 172.21.15.92:37850) 2024-09-21T02:09:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/2708014912 learned_addr learned my addr 172.21.15.92:0/2708014912 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebac91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/2708014912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 msgr2=0x561eebac91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebac91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/2708014912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6d5d80 msgr2=0x561eebac1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6d5d80 0x561eebac1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/2708014912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eebacbd10 con 0x561eeb6ee2f0 2024-09-21T02:09:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebac91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.942+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebac1540 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0aaaa0 tx=0x7fb9dc004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a779c6f4ba055927 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9cd7fa640 1 -- 172.21.15.92:0/2708014912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0a6480 con 0x561eeb6ee2f0 2024-09-21T02:09:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9cd7fa640 1 -- 172.21.15.92:0/2708014912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9dc0ed320 con 0x561eeb6ee2f0 2024-09-21T02:09:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9cd7fa640 1 -- 172.21.15.92:0/2708014912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0a4480 con 0x561eeb6ee2f0 2024-09-21T02:09:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 -- 172.21.15.92:0/2708014912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 msgr2=0x561eebac1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebac1540 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0aaaa0 tx=0x7fb9dc004a90 comp rx=0 tx=0).stop 2024-09-21T02:09:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 -- 172.21.15.92:0/2708014912 shutdown_connections 2024-09-21T02:09:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6f64d0 0x561eebac91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebac1540 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/2708014912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6d5d80 0x561eebac1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 -- 172.21.15.92:0/2708014912 >> 172.21.15.92:0/2708014912 conn(0x561eeb70b300 msgr2=0x561eebabc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 -- 172.21.15.92:0/2708014912 shutdown_connections 2024-09-21T02:09:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.943+0000 7fb9f4be2740 1 -- 172.21.15.92:0/2708014912 wait complete. 2024-09-21T02:09:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 -- start start 2024-09-21T02:09:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 Processor -- start 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6d5d80 0x561eebabb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebabb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6f64d0 0x561eebb42990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81aa270 con 0x561eeb6ee2f0 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81a9ba0 con 0x561eeb6f64d0 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e81a6de0 con 0x561eeb6d5d80 2024-09-21T02:09:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9f4be2740 10 client.0 _ll_register_callbacks cb 0x7fb9d800bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebabb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebabb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37862/0 (socket says 172.21.15.92:37862) 2024-09-21T02:09:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1673010043 learned_addr learned my addr 172.21.15.92:0/1673010043 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.944+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6d5d80 0x561eebabb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e7fff640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6f64d0 0x561eebb42990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6d5d80 msgr2=0x561eebabb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6d5d80 0x561eebabb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1673010043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6f64d0 msgr2=0x561eebb42990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6f64d0 0x561eebb42990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e6ffd640 1 -- 172.21.15.92:0/1673010043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eebb43000 con 0x561eeb6ee2f0 2024-09-21T02:09:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9e6ffd640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebabb5e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0ed5d0 tx=0x7fb9dc0edb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711aaa0956bbccc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/1673010043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0f6030 con 0x561eeb6ee2f0 2024-09-21T02:09:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/1673010043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb9dc0a9aa0 con 0x561eeb6ee2f0 2024-09-21T02:09:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9dc0ed320 con 0x561eeb6ee2f0 2024-09-21T02:09:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/1673010043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb9dc0a4800 con 0x561eeb6ee2f0 2024-09-21T02:09:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 10 client.16779 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561eebb44200 con 0x561eeb6ee2f0 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/1673010043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9dc0ed320 con 0x561eeb6ee2f0 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/1673010043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb9dc0a3070 con 0x561eeb6ee2f0 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 10 client.16779 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 20 client.16779 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb7171b0 0x561eebb46550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 4 client.16779 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:09:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.945+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561eeba2f0f0 con 0x561eeb7171b0 2024-09-21T02:09:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.946+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb7171b0 0x561eebb46550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.946+0000 7fb9c6ffd640 10 client.16779 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.946+0000 7fb9e77fe640 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb7171b0 0x561eebb46550 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deebe164248909b4 server_cookie=ce5fe15eb80de4fe in_seq=0 out_seq=0 2024-09-21T02:09:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9c6ffd640 1 -- 172.21.15.92:0/1673010043 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561eeba2f0f0 con 0x561eeb7171b0 2024-09-21T02:09:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9c6ffd640 10 client.16779 handle_command_reply: tid=0 2024-09-21T02:09:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 client.16779 shutdown 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 msgr2=0x561eebabb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebabb5e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0ed5d0 tx=0x7fb9dc0edb70 comp rx=0 tx=0).stop 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 shutdown_connections 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561eeb7171b0 0x561eebb46550 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561eeb6f64d0 0x561eebb42990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9c6ffd640 0 client.16779 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561eeb6ee2f0 0x561eebabb5e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 --2- 172.21.15.92:0/1673010043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eeb6d5d80 0x561eebabb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 >> 172.21.15.92:0/1673010043 conn(0x561eeb70b300 msgr2=0x561eebabc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 shutdown_connections 2024-09-21T02:09:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.959+0000 7fb9f4be2740 1 -- 172.21.15.92:0/1673010043 wait complete. 2024-09-21T02:09:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9f4be2740 20 client.16779 trim_cache size 0 max 16384 2024-09-21T02:09:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:09:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:09:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:09:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:09:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb9d00b6510 msgr2=0x7fb9d00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb9d00b6510 0x7fb9d00b8a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0004af0 tx=0x7fb9e0009380 comp rx=0 tx=0).stop 2024-09-21T02:09:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 msgr2=0x7fb9e8199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 0x7fb9e8199470 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00e9f0 tx=0x7fb9dc00edf0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 shutdown_connections 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb9d00b6510 0x7fb9d00b8a20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9e8103fd0 0x7fb9e819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9e8103630 0x7fb9e8199470 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 --2- 172.21.15.92:0/1198971527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e81025b0 0x7fb9e8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 >> 172.21.15.92:0/1198971527 conn(0x7fb9e80fdf30 msgr2=0x7fb9e8106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 shutdown_connections 2024-09-21T02:09:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:04.960+0000 7fb9edb98640 1 -- 172.21.15.92:0/1198971527 wait complete. 2024-09-21T02:09:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:05 smithi136 ceph-mon[34754]: pgmap v483: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:05.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:05 smithi092 ceph-mon[31190]: pgmap v483: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:05 smithi099 ceph-mon[34404]: pgmap v483: 97 pgs: 97 active+clean; 3.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:07.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:07 smithi092 ceph-mon[31190]: pgmap v484: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:09:07.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:07 smithi099 ceph-mon[34404]: pgmap v484: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:09:07.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:07 smithi136 ceph-mon[34754]: pgmap v484: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:09:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:09 smithi092 ceph-mon[31190]: pgmap v485: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:09:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:09 smithi099 ceph-mon[34404]: pgmap v485: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:09:09.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:09 smithi136 ceph-mon[34754]: pgmap v485: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:09:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:11 smithi092 ceph-mon[31190]: pgmap v486: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:09:11.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:11 smithi099 ceph-mon[34404]: pgmap v486: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:09:11.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:11 smithi136 ceph-mon[34754]: pgmap v486: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:09:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:13 smithi092 ceph-mon[31190]: pgmap v487: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:13 smithi099 ceph-mon[34404]: pgmap v487: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:13.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:13 smithi136 ceph-mon[34754]: pgmap v487: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:15 smithi092 ceph-mon[31190]: pgmap v488: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:09:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:15 smithi099 ceph-mon[34404]: pgmap v488: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:09:15.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:15 smithi136 ceph-mon[34754]: pgmap v488: 97 pgs: 97 active+clean; 3.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:09:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:17 smithi092 ceph-mon[31190]: pgmap v489: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:17 smithi099 ceph-mon[34404]: pgmap v489: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:17 smithi136 ceph-mon[34754]: pgmap v489: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.067+0000 7fceefe84640 1 -- 172.21.15.92:0/112167779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103620 msgr2=0x7fcee8103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.067+0000 7fceefe84640 1 --2- 172.21.15.92:0/112167779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103620 0x7fcee8103a30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009900 tx=0x7fcee402a410 comp rx=0 tx=0).stop 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 -- 172.21.15.92:0/112167779 shutdown_connections 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 --2- 172.21.15.92:0/112167779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee8103fc0 0x7fcee810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 --2- 172.21.15.92:0/112167779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103620 0x7fcee8103a30 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 --2- 172.21.15.92:0/112167779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee81025a0 0x7fcee8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 -- 172.21.15.92:0/112167779 >> 172.21.15.92:0/112167779 conn(0x7fcee80fdee0 msgr2=0x7fcee8100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 -- 172.21.15.92:0/112167779 shutdown_connections 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 -- 172.21.15.92:0/112167779 wait complete. 2024-09-21T02:09:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 Processor -- start 2024-09-21T02:09:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.068+0000 7fceefe84640 1 -- start start 2024-09-21T02:09:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceefe84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee81025a0 0x7fcee8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceefe84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee8103620 0x7fcee8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceefe84640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 0x7fcee819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceefe84640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcee810e240 con 0x7fcee8103fc0 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceefe84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcee810e0c0 con 0x7fcee8103620 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceefe84640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcee810e3c0 con 0x7fcee81025a0 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 0x7fcee819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 0x7fcee819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41138/0 (socket says 172.21.15.92:41138) 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 -- 172.21.15.92:0/1458043164 learned_addr learned my addr 172.21.15.92:0/1458043164 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceed41f640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee8103620 0x7fcee8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceedc20640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee81025a0 0x7fcee8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 -- 172.21.15.92:0/1458043164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee81025a0 msgr2=0x7fcee8198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee81025a0 0x7fcee8198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 -- 172.21.15.92:0/1458043164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee8103620 msgr2=0x7fcee8199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee8103620 0x7fcee8199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 -- 172.21.15.92:0/1458043164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee819a8d0 con 0x7fcee8103fc0 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fceee421640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 0x7fcee819a0b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcee000dbd0 tx=0x7fcee000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.069+0000 7fcedeffd640 1 -- 172.21.15.92:0/1458043164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0004b90 con 0x7fcee8103fc0 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.070+0000 7fcedeffd640 1 -- 172.21.15.92:0/1458043164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee0004d30 con 0x7fcee8103fc0 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.070+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee819abc0 con 0x7fcee8103fc0 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.070+0000 7fcedeffd640 1 -- 172.21.15.92:0/1458043164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0004720 con 0x7fcee8103fc0 2024-09-21T02:09:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.070+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee81a12b0 con 0x7fcee8103fc0 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.072+0000 7fcef6def740 1 Processor -- start 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.072+0000 7fcedeffd640 1 -- 172.21.15.92:0/1458043164 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fcee00039c0 con 0x7fcee8103fc0 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.072+0000 7fcef6def740 1 -- start start 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcef6def740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c44b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcef6def740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0782f0 0x564d1c44b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcef6def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c44c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcef6def740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564d1c3bb6f0 con 0x564d1c0a11b0 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcef6def740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564d1c3aab20 con 0x564d1c0782f0 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcef6def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564d1c3ae930 con 0x564d1c0804d0 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c44b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c44b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41154/0 (socket says 172.21.15.92:41154) 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 -- 172.21.15.92:0/2935458070 learned_addr learned my addr 172.21.15.92:0/2935458070 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceedc20640 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0782f0 0x564d1c44b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c44c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 -- 172.21.15.92:0/2935458070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 msgr2=0x564d1c44c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c44c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 -- 172.21.15.92:0/2935458070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0782f0 msgr2=0x564d1c44b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0782f0 0x564d1c44b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fcedeffd640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fced40b6510 0x7fced40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 -- 172.21.15.92:0/2935458070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1c370530 con 0x564d1c0a11b0 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceee421640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fced40b6510 0x7fced40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.073+0000 7fceed41f640 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c44b060 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009230 tx=0x7fcee4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194f9fb58fbe6694 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.074+0000 7fcedeffd640 1 -- 172.21.15.92:0/1458043164 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcee00e7510 con 0x7fcee8103fc0 2024-09-21T02:09:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.074+0000 7fced3fff640 1 -- 172.21.15.92:0/2935458070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4035030 con 0x564d1c0a11b0 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.074+0000 7fced3fff640 1 -- 172.21.15.92:0/2935458070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee4003ca0 con 0x564d1c0a11b0 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.074+0000 7fced3fff640 1 -- 172.21.15.92:0/2935458070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4031c90 con 0x564d1c0a11b0 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 -- 172.21.15.92:0/2935458070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 msgr2=0x564d1c44b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fceee421640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fced40b6510 0x7fced40b8a20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcee00ad380 tx=0x7fcee0071c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c44b060 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009230 tx=0x7fcee4005fd0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 -- 172.21.15.92:0/2935458070 shutdown_connections 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c44c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0782f0 0x564d1c44b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 --2- 172.21.15.92:0/2935458070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c44b060 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 -- 172.21.15.92:0/2935458070 >> 172.21.15.92:0/2935458070 conn(0x564d1c095300 msgr2=0x564d1c4448e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 -- 172.21.15.92:0/2935458070 shutdown_connections 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 -- 172.21.15.92:0/2935458070 wait complete. 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 -- start start 2024-09-21T02:09:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.075+0000 7fcef6def740 1 Processor -- start 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c454770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0804d0 0x564d1c454e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c4559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564d1c2e2ab0 con 0x564d1c0a11b0 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564d1c350730 con 0x564d1c0804d0 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564d1c44cce0 con 0x564d1c0782f0 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fcef6def740 10 client.0 _ll_register_callbacks cb 0x564d1c3a46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c4559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c4559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41162/0 (socket says 172.21.15.92:41162) 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 -- 172.21.15.92:0/2708298484 learned_addr learned my addr 172.21.15.92:0/2708298484 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceedc20640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0804d0 0x564d1c454e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c454770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 -- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 msgr2=0x564d1c454770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c454770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 -- 172.21.15.92:0/2708298484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0804d0 msgr2=0x564d1c454e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0804d0 0x564d1c454e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 -- 172.21.15.92:0/2708298484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d1c10c790 con 0x564d1c0a11b0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fceed41f640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c4559a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000c00 tx=0x7fcee4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9aa64827ceffb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fced17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.076+0000 7fced17fa640 1 -- 172.21.15.92:0/2708298484 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee404b030 con 0x564d1c0a11b0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fced17fa640 1 -- 172.21.15.92:0/2708298484 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcee4031c40 con 0x564d1c0a11b0 2024-09-21T02:09:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee4003ca0 con 0x564d1c0a11b0 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fced17fa640 1 -- 172.21.15.92:0/2708298484 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcee4035040 con 0x564d1c0a11b0 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 10 client.16791 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d1c185620 con 0x564d1c0a11b0 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fced17fa640 1 -- 172.21.15.92:0/2708298484 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee4003ca0 con 0x564d1c0a11b0 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fced17fa640 1 -- 172.21.15.92:0/2708298484 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcee403bab0 con 0x564d1c0a11b0 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 10 client.16791 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 20 client.16791 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c05fd80 0x564d1c461a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 4 client.16791 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d1c3bd640 con 0x564d1c05fd80 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.077+0000 7fceedc20640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c05fd80 0x564d1c461a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.078+0000 7fced17fa640 10 client.16791 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.078+0000 7fceedc20640 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c05fd80 0x564d1c461a80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4c9901c0725a63 server_cookie=96cd2d531582f938 in_seq=0 out_seq=0 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fced17fa640 1 -- 172.21.15.92:0/2708298484 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d1c3bd640 con 0x564d1c05fd80 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fced17fa640 10 client.16791 handle_command_reply: tid=0 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 client.16791 shutdown 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 msgr2=0x564d1c4559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c4559a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000c00 tx=0x7fcee4005850 comp rx=0 tx=0).stop 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 shutdown_connections 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c05fd80 0x564d1c461a80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0a11b0 0x564d1c4559a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0804d0 0x564d1c454e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 --2- 172.21.15.92:0/2708298484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c454770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fced17fa640 0 client.16791 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 >> 172.21.15.92:0/2708298484 conn(0x564d1c095300 msgr2=0x564d1c442ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 shutdown_connections 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 1 -- 172.21.15.92:0/2708298484 wait complete. 2024-09-21T02:09:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.079+0000 7fcef6def740 20 client.16791 trim_cache size 0 max 16384 2024-09-21T02:09:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 Processor -- start 2024-09-21T02:09:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 -- start start 2024-09-21T02:09:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 0x564d1c445010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c445550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c445a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81a8cb0 con 0x564d1c0782f0 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81aa7f0 con 0x564d1c0a11b0 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fcef6def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81a9380 con 0x564d1c0804d0 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 0x564d1c445010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 0x564d1c445010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41172/0 (socket says 172.21.15.92:41172) 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 -- 172.21.15.92:0/2296751183 learned_addr learned my addr 172.21.15.92:0/2296751183 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceedc20640 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c445550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c445a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 -- 172.21.15.92:0/2296751183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 msgr2=0x564d1c445550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c445550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 -- 172.21.15.92:0/2296751183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 msgr2=0x564d1c445a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c445a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 -- 172.21.15.92:0/2296751183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1c455290 con 0x564d1c0782f0 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fceed41f640 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 0x564d1c445010 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcee4005f70 tx=0x7fcee4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef93b44bd5f8c9da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.094+0000 7fced37fe640 1 -- 172.21.15.92:0/2296751183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee404b070 con 0x564d1c0782f0 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fced37fe640 1 -- 172.21.15.92:0/2296751183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee4004260 con 0x564d1c0782f0 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fced37fe640 1 -- 172.21.15.92:0/2296751183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee403b6c0 con 0x564d1c0782f0 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 -- 172.21.15.92:0/2296751183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 msgr2=0x564d1c445010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 0x564d1c445010 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcee4005f70 tx=0x7fcee4005fd0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 -- 172.21.15.92:0/2296751183 shutdown_connections 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c445a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0804d0 0x564d1c445550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 --2- 172.21.15.92:0/2296751183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0782f0 0x564d1c445010 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 -- 172.21.15.92:0/2296751183 >> 172.21.15.92:0/2296751183 conn(0x564d1c095300 msgr2=0x564d1c44bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 -- 172.21.15.92:0/2296751183 shutdown_connections 2024-09-21T02:09:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.095+0000 7fcef6def740 1 -- 172.21.15.92:0/2296751183 wait complete. 2024-09-21T02:09:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 -- start start 2024-09-21T02:09:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 Processor -- start 2024-09-21T02:09:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c45c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 0x564d1c45b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c45b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81aa120 con 0x564d1c0804d0 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81a9a50 con 0x564d1c0a11b0 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcee81a6c90 con 0x564d1c0782f0 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fcef6def740 10 client.0 _ll_register_callbacks cb 0x7fced8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fceedc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 0x564d1c45b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fceedc20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 0x564d1c45b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41184/0 (socket says 172.21.15.92:41184) 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fceedc20640 1 -- 172.21.15.92:0/164251368 learned_addr learned my addr 172.21.15.92:0/164251368 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.096+0000 7fceed41f640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c45c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceedc20640 1 -- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 msgr2=0x564d1c45c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceedc20640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c45c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceedc20640 1 -- 172.21.15.92:0/164251368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 msgr2=0x564d1c45b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceed41f640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c45b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceedc20640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c45b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceedc20640 1 -- 172.21.15.92:0/164251368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d1c45bab0 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceed41f640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c45b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fceedc20640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 0x564d1c45b000 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fced800b8a0 tx=0x7fced80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c5179d280f7578 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fced17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fced17fa640 1 -- 172.21.15.92:0/164251368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8029060 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fced17fa640 1 -- 172.21.15.92:0/164251368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fced801d660 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee4004260 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fced17fa640 1 -- 172.21.15.92:0/164251368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fced801d860 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 10 client.16797 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d1c45a9e0 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fced17fa640 1 -- 172.21.15.92:0/164251368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee4004260 con 0x564d1c0804d0 2024-09-21T02:09:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fced17fa640 1 -- 172.21.15.92:0/164251368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fced8011b60 con 0x564d1c0804d0 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 10 client.16797 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 20 client.16797 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c0351c0 0x564d1c4ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 4 client.16797 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.097+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d1c3b90f0 con 0x564d1c0351c0 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.098+0000 7fceed41f640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c0351c0 0x564d1c4ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.098+0000 7fced17fa640 10 client.16797 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.098+0000 7fceed41f640 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c0351c0 0x564d1c4ceb30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a824d9ad760d6e51 server_cookie=7929534aab933935 in_seq=0 out_seq=0 2024-09-21T02:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fced17fa640 1 -- 172.21.15.92:0/164251368 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d1c3b90f0 con 0x564d1c0351c0 2024-09-21T02:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fced17fa640 10 client.16797 handle_command_reply: tid=0 2024-09-21T02:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 client.16797 shutdown 2024-09-21T02:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 msgr2=0x564d1c45b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 0x564d1c45b000 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fced800b8a0 tx=0x7fced80035f0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 shutdown_connections 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x564d1c0351c0 0x564d1c4ceb30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d1c0a11b0 0x564d1c45b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564d1c0804d0 0x564d1c45b000 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 --2- 172.21.15.92:0/164251368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d1c0782f0 0x564d1c45c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fced17fa640 0 client.16797 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 >> 172.21.15.92:0/164251368 conn(0x564d1c095300 msgr2=0x564d1c44f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 shutdown_connections 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 1 -- 172.21.15.92:0/164251368 wait complete. 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.113+0000 7fcef6def740 20 client.16797 trim_cache size 0 max 16384 2024-09-21T02:09:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fced40b6510 msgr2=0x7fced40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fced40b6510 0x7fced40b8a20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcee00ad380 tx=0x7fcee0071c60 comp rx=0 tx=0).stop 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 msgr2=0x7fcee819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 0x7fcee819a0b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcee000dbd0 tx=0x7fcee000b260 comp rx=0 tx=0).stop 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 shutdown_connections 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fced40b6510 0x7fced40b8a20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee8103fc0 0x7fcee819a0b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcee8103620 0x7fcee8199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 --2- 172.21.15.92:0/1458043164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee81025a0 0x7fcee8198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 >> 172.21.15.92:0/1458043164 conn(0x7fcee80fdee0 msgr2=0x7fcee8106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 shutdown_connections 2024-09-21T02:09:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:18.114+0000 7fceefe84640 1 -- 172.21.15.92:0/1458043164 wait complete. 2024-09-21T02:09:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:19 smithi092 ceph-mon[31190]: pgmap v490: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:19 smithi099 ceph-mon[34404]: pgmap v490: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:19.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:19 smithi136 ceph-mon[34754]: pgmap v490: 97 pgs: 97 active+clean; 3.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:21 smithi099 ceph-mon[34404]: pgmap v491: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-21T02:09:21.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:21 smithi136 ceph-mon[34754]: pgmap v491: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-21T02:09:21.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:21 smithi092 ceph-mon[31190]: pgmap v491: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-21T02:09:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:23 smithi099 ceph-mon[34404]: pgmap v492: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-21T02:09:23.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:23 smithi136 ceph-mon[34754]: pgmap v492: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-21T02:09:23.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:23 smithi092 ceph-mon[31190]: pgmap v492: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-21T02:09:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:25 smithi099 ceph-mon[34404]: pgmap v493: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:25.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:25 smithi136 ceph-mon[34754]: pgmap v493: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:25.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:25 smithi092 ceph-mon[31190]: pgmap v493: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:27 smithi099 ceph-mon[34404]: pgmap v494: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:09:27.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:27 smithi136 ceph-mon[34754]: pgmap v494: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:09:27.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:27 smithi092 ceph-mon[31190]: pgmap v494: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:09:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:29 smithi099 ceph-mon[34404]: pgmap v495: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:29.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:29 smithi136 ceph-mon[34754]: pgmap v495: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:29.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:29 smithi092 ceph-mon[31190]: pgmap v495: 97 pgs: 97 active+clean; 3.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:09:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 -- 172.21.15.92:0/347002076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898106b20 msgr2=0x7fe898108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 --2- 172.21.15.92:0/347002076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898106b20 0x7fe898108f40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe880009900 tx=0x7fe88002a410 comp rx=0 tx=0).stop 2024-09-21T02:09:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 -- 172.21.15.92:0/347002076 shutdown_connections 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 --2- 172.21.15.92:0/347002076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898109480 0x7fe89810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 --2- 172.21.15.92:0/347002076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898106b20 0x7fe898108f40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 --2- 172.21.15.92:0/347002076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898101a90 0x7fe898101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 -- 172.21.15.92:0/347002076 >> 172.21.15.92:0/347002076 conn(0x7fe8980fd6d0 msgr2=0x7fe8980ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 -- 172.21.15.92:0/347002076 shutdown_connections 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.217+0000 7fe89d590640 1 -- 172.21.15.92:0/347002076 wait complete. 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 Processor -- start 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 -- start start 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 0x7fe898199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898106b20 0x7fe89819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898109480 0x7fe89819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe89810f320 con 0x7fe898101a90 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe89810f1a0 con 0x7fe898106b20 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe89d590640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe89810f4a0 con 0x7fe898109480 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.218+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 0x7fe898199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 0x7fe898199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48928/0 (socket says 172.21.15.92:48928) 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 -- 172.21.15.92:0/4090298094 learned_addr learned my addr 172.21.15.92:0/4090298094 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe8967fc640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898106b20 0x7fe89819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe8977fe640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898109480 0x7fe89819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 -- 172.21.15.92:0/4090298094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898109480 msgr2=0x7fe89819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898109480 0x7fe89819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 -- 172.21.15.92:0/4090298094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898106b20 msgr2=0x7fe89819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898106b20 0x7fe89819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 -- 172.21.15.92:0/4090298094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89819b950 con 0x7fe898101a90 2024-09-21T02:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe8977fe640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898109480 0x7fe89819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe896ffd640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 0x7fe898199fa0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe88800ea80 tx=0x7fe88800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe88ffff640 1 -- 172.21.15.92:0/4090298094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88800ce40 con 0x7fe898101a90 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe88ffff640 1 -- 172.21.15.92:0/4090298094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe888005020 con 0x7fe898101a90 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89819bc40 con 0x7fe898101a90 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe88ffff640 1 -- 172.21.15.92:0/4090298094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880116d0 con 0x7fe898101a90 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.219+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a21e0 con 0x7fe898101a90 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 Processor -- start 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 -- start start 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe88ffff640 1 -- 172.21.15.92:0/4090298094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe8880054d0 con 0x7fe898101a90 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfec3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5560dfe366f0 con 0x5560dfaf32f0 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560dfe25b20 con 0x5560dfafb4d0 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe8a45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5560dfe29930 con 0x5560dfb1c1b0 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.222+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48942/0 (socket says 172.21.15.92:48942) 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 -- 172.21.15.92:0/1162506189 learned_addr learned my addr 172.21.15.92:0/1162506189 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe8977fe640 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfec3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe8967fc640 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 -- 172.21.15.92:0/1162506189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 msgr2=0x5560dfec3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfec3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 -- 172.21.15.92:0/1162506189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 msgr2=0x5560dfec5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 -- 172.21.15.92:0/1162506189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dfdeb530 con 0x5560dfaf32f0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe8967fc640 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe896ffd640 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec44c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe8880ed6f0 tx=0x7fe8880edbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c73fde21473b068 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe88ffff640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8840b6510 0x7fe8840b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1162506189 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880fa070 con 0x5560dfaf32f0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe8977fe640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8840b6510 0x7fe8840b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1162506189 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8880cf700 con 0x5560dfaf32f0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.223+0000 7fe88d7fa640 1 -- 172.21.15.92:0/1162506189 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880ace60 con 0x5560dfaf32f0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe88ffff640 1 -- 172.21.15.92:0/4090298094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe8880e7580 con 0x7fe898101a90 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 -- 172.21.15.92:0/1162506189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 msgr2=0x5560dfec44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8977fe640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8840b6510 0x7fe8840b8a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe89000bc20 tx=0x7fe890005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec44c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe8880ed6f0 tx=0x7fe8880edbb0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 -- 172.21.15.92:0/1162506189 shutdown_connections 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec44c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 --2- 172.21.15.92:0/1162506189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfec3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.224+0000 7fe8a45df740 1 -- 172.21.15.92:0/1162506189 >> 172.21.15.92:0/1162506189 conn(0x5560dfb10300 msgr2=0x5560dfebf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 -- 172.21.15.92:0/1162506189 shutdown_connections 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 -- 172.21.15.92:0/1162506189 wait complete. 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 -- start start 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 Processor -- start 2024-09-21T02:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfaf32f0 0x5560dfecd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dfecdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfece800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5560dfd5dab0 con 0x5560dfafb4d0 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560dfdcb730 con 0x5560dfaf32f0 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5560dfec5b30 con 0x5560dfb1c1b0 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8a45df740 10 client.0 _ll_register_callbacks cb 0x5560dfe1f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dfecdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dfecdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48954/0 (socket says 172.21.15.92:48954) 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe8967fc640 1 -- 172.21.15.92:0/4111552189 learned_addr learned my addr 172.21.15.92:0/4111552189 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.225+0000 7fe896ffd640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfaf32f0 0x5560dfecd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8977fe640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfece800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8967fc640 1 -- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 msgr2=0x5560dfece800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8967fc640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfece800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8967fc640 1 -- 172.21.15.92:0/4111552189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfaf32f0 msgr2=0x5560dfecd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8967fc640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfaf32f0 0x5560dfecd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8967fc640 1 -- 172.21.15.92:0/4111552189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dfb87790 con 0x5560dfafb4d0 2024-09-21T02:09:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8967fc640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dfecdc90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe880009900 tx=0x7fe880031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add6ae9c1b6b0c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe87e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe87e7fc640 1 -- 172.21.15.92:0/4111552189 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880035070 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe87e7fc640 1 -- 172.21.15.92:0/4111552189 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe880004240 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8880cf700 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe87e7fc640 1 -- 172.21.15.92:0/4111552189 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8800318f0 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8a45df740 10 client.16809 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560dfc00620 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.226+0000 7fe87e7fc640 1 -- 172.21.15.92:0/4111552189 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8880cf700 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe87e7fc640 1 -- 172.21.15.92:0/4111552189 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe880043020 con 0x5560dfafb4d0 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe8a45df740 10 client.16809 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe8a45df740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe8a45df740 20 client.16809 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe8a45df740 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfadad80 0x5560dfedad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe8a45df740 4 client.16809 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560dfe38640 con 0x5560dfadad80 2024-09-21T02:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe896ffd640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfadad80 0x5560dfedad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe87e7fc640 10 client.16809 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.227+0000 7fe896ffd640 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfadad80 0x5560dfedad40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466c4281c154ccc4 server_cookie=9e49f91a1bdcff12 in_seq=0 out_seq=0 2024-09-21T02:09:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe87e7fc640 1 -- 172.21.15.92:0/4111552189 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560dfe38640 con 0x5560dfadad80 2024-09-21T02:09:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe87e7fc640 10 client.16809 handle_command_reply: tid=0 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 client.16809 shutdown 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 msgr2=0x5560dfecdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dfecdc90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe880009900 tx=0x7fe880031e80 comp rx=0 tx=0).stop 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 shutdown_connections 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfadad80 0x5560dfedad40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfece800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe87e7fc640 0 client.16809 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dfecdc90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 --2- 172.21.15.92:0/4111552189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfaf32f0 0x5560dfecd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 >> 172.21.15.92:0/4111552189 conn(0x5560dfb10300 msgr2=0x5560dfebdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 shutdown_connections 2024-09-21T02:09:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.228+0000 7fe8a45df740 1 -- 172.21.15.92:0/4111552189 wait complete. 2024-09-21T02:09:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.229+0000 7fe8a45df740 20 client.16809 trim_cache size 0 max 16384 2024-09-21T02:09:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 Processor -- start 2024-09-21T02:09:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 -- start start 2024-09-21T02:09:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfecbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ab010 con 0x5560dfaf32f0 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981acb50 con 0x5560dfafb4d0 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe8a45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ab6e0 con 0x5560dfb1c1b0 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.243+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48968/0 (socket says 172.21.15.92:48968) 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 -- 172.21.15.92:0/385216932 learned_addr learned my addr 172.21.15.92:0/385216932 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe8967fc640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 -- 172.21.15.92:0/385216932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 msgr2=0x5560dfecbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe8977fe640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfecbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfecbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 -- 172.21.15.92:0/385216932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 msgr2=0x5560dfec4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 -- 172.21.15.92:0/385216932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dfeceb70 con 0x5560dfaf32f0 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe8977fe640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfecbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe8967fc640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe896ffd640 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec3df0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe8880ad3f0 tx=0x7fe8880edbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a30ddd373a5b2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe88cff9640 1 -- 172.21.15.92:0/385216932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880193b0 con 0x5560dfaf32f0 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe88cff9640 1 -- 172.21.15.92:0/385216932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88801e170 con 0x5560dfaf32f0 2024-09-21T02:09:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.244+0000 7fe88cff9640 1 -- 172.21.15.92:0/385216932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888019590 con 0x5560dfaf32f0 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 -- 172.21.15.92:0/385216932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 msgr2=0x5560dfec3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec3df0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe8880ad3f0 tx=0x7fe8880edbb0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 -- 172.21.15.92:0/385216932 shutdown_connections 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfb1c1b0 0x5560dfecbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfafb4d0 0x5560dfec4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 --2- 172.21.15.92:0/385216932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfaf32f0 0x5560dfec3df0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 -- 172.21.15.92:0/385216932 >> 172.21.15.92:0/385216932 conn(0x5560dfb10300 msgr2=0x5560dfecc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 -- 172.21.15.92:0/385216932 shutdown_connections 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 -- 172.21.15.92:0/385216932 wait complete. 2024-09-21T02:09:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 -- start start 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.245+0000 7fe8a45df740 1 Processor -- start 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfaf32f0 0x5560dff40770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dff40e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfb1c1b0 0x5560dff419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ac480 con 0x5560dfafb4d0 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981abdb0 con 0x5560dfb1c1b0 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981a8ff0 con 0x5560dfaf32f0 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 10 client.0 _ll_register_callbacks cb 0x7fe880009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dff40e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dff40e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48982/0 (socket says 172.21.15.92:48982) 2024-09-21T02:09:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 -- 172.21.15.92:0/3093166073 learned_addr learned my addr 172.21.15.92:0/3093166073 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8977fe640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfb1c1b0 0x5560dff419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe896ffd640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfaf32f0 0x5560dff40770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 -- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfaf32f0 msgr2=0x5560dff40770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfaf32f0 0x5560dff40770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 -- 172.21.15.92:0/3093166073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfb1c1b0 msgr2=0x5560dff419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfb1c1b0 0x5560dff419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 -- 172.21.15.92:0/3093166073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dff42100 con 0x5560dfafb4d0 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8977fe640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfb1c1b0 0x5560dff419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8967fc640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dff40e30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe88002a930 tx=0x7fe880050650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3cf263eda79d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe87e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe87e7fc640 1 -- 172.21.15.92:0/3093166073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880035030 con 0x5560dfafb4d0 2024-09-21T02:09:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe87e7fc640 1 -- 172.21.15.92:0/3093166073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe8800508c0 con 0x5560dfafb4d0 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.246+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560dfe340f0 con 0x5560dfafb4d0 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe87e7fc640 1 -- 172.21.15.92:0/3093166073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe880005de0 con 0x5560dfafb4d0 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 10 client.16815 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560dff49340 con 0x5560dfafb4d0 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe87e7fc640 1 -- 172.21.15.92:0/3093166073 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5560dfe340f0 con 0x5560dfafb4d0 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe87e7fc640 1 -- 172.21.15.92:0/3093166073 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe880043070 con 0x5560dfafb4d0 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 10 client.16815 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 20 client.16815 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfab01c0 0x5560dff4bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 4 client.16815 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:09:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560dfe31520 con 0x5560dfab01c0 2024-09-21T02:09:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.247+0000 7fe896ffd640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfab01c0 0x5560dff4bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.248+0000 7fe87e7fc640 10 client.16815 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.248+0000 7fe896ffd640 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfab01c0 0x5560dff4bdb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95947215334c5fb1 server_cookie=fdb7baccdbb6bca5 in_seq=0 out_seq=0 2024-09-21T02:09:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.261+0000 7fe87e7fc640 1 -- 172.21.15.92:0/3093166073 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560dfe31520 con 0x5560dfab01c0 2024-09-21T02:09:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.261+0000 7fe87e7fc640 10 client.16815 handle_command_reply: tid=0 2024-09-21T02:09:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.261+0000 7fe8a45df740 1 client.16815 shutdown 2024-09-21T02:09:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 msgr2=0x5560dff40e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dff40e30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe88002a930 tx=0x7fe880050650 comp rx=0 tx=0).stop 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 shutdown_connections 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5560dfab01c0 0x5560dff4bdb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560dfb1c1b0 0x5560dff419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5560dfafb4d0 0x5560dff40e30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe87e7fc640 0 client.16815 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 --2- 172.21.15.92:0/3093166073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560dfaf32f0 0x5560dff40770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 >> 172.21.15.92:0/3093166073 conn(0x5560dfb10300 msgr2=0x5560dfebe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 shutdown_connections 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 1 -- 172.21.15.92:0/3093166073 wait complete. 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe8a45df740 20 client.16815 trim_cache size 0 max 16384 2024-09-21T02:09:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:09:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:09:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:09:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8840b6510 msgr2=0x7fe8840b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8840b6510 0x7fe8840b8a20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe89000bc20 tx=0x7fe890005e90 comp rx=0 tx=0).stop 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 msgr2=0x7fe898199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 0x7fe898199fa0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe88800ea80 tx=0x7fe88800ef40 comp rx=0 tx=0).stop 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 shutdown_connections 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe8840b6510 0x7fe8840b8a20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898109480 0x7fe89819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898106b20 0x7fe89819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 --2- 172.21.15.92:0/4090298094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe898101a90 0x7fe898199fa0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.262+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 >> 172.21.15.92:0/4090298094 conn(0x7fe8980fd6d0 msgr2=0x7fe8981078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.263+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 shutdown_connections 2024-09-21T02:09:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:31.263+0000 7fe89d590640 1 -- 172.21.15.92:0/4090298094 wait complete. 2024-09-21T02:09:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:31 smithi099 ceph-mon[34404]: pgmap v496: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:09:31.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:31 smithi136 ceph-mon[34754]: pgmap v496: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:09:31.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:31 smithi092 ceph-mon[31190]: pgmap v496: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:09:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:33 smithi099 ceph-mon[34404]: pgmap v497: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:09:33.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:33 smithi136 ceph-mon[34754]: pgmap v497: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:09:33.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:33 smithi092 ceph-mon[31190]: pgmap v497: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:09:33.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:09:33.858 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 -- 172.21.15.92:0/2628598927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 msgr2=0x7fd0101087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 --2- 172.21.15.92:0/2628598927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd0101087b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd000009900 tx=0x7fd00002a410 comp rx=0 tx=0).stop 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 -- 172.21.15.92:0/2628598927 shutdown_connections 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 --2- 172.21.15.92:0/2628598927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd010108cf0 0x7fd01010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 --2- 172.21.15.92:0/2628598927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd0101087b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 --2- 172.21.15.92:0/2628598927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd010102a60 0x7fd010102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.343+0000 7fd017436640 1 -- 172.21.15.92:0/2628598927 >> 172.21.15.92:0/2628598927 conn(0x7fd010079730 msgr2=0x7fd010079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.344+0000 7fd017436640 1 -- 172.21.15.92:0/2628598927 shutdown_connections 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.344+0000 7fd017436640 1 -- 172.21.15.92:0/2628598927 wait complete. 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.344+0000 7fd017436640 1 Processor -- start 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.344+0000 7fd017436640 1 -- start start 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd010102a60 0x7fd01019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd01019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd010108cf0 0x7fd01019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101129a0 con 0x7fd0101083a0 2024-09-21T02:09:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd010112820 con 0x7fd010102a60 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd010112b20 con 0x7fd010108cf0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd01019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd01019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56272/0 (socket says 172.21.15.92:56272) 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 -- 172.21.15.92:0/4011749368 learned_addr learned my addr 172.21.15.92:0/4011749368 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0151d2640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd010102a60 0x7fd01019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0159d3640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd010108cf0 0x7fd01019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 -- 172.21.15.92:0/4011749368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd010108cf0 msgr2=0x7fd01019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd010108cf0 0x7fd01019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 -- 172.21.15.92:0/4011749368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd010102a60 msgr2=0x7fd01019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd010102a60 0x7fd01019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 -- 172.21.15.92:0/4011749368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0101a0450 con 0x7fd0101083a0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0149d1640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd01019aa80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd000031a90 tx=0x7fd000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0067fc640 1 -- 172.21.15.92:0/4011749368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00003f030 con 0x7fd0101083a0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0067fc640 1 -- 172.21.15.92:0/4011749368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd000003cb0 con 0x7fd0101083a0 2024-09-21T02:09:34.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0101a06e0 con 0x7fd0101083a0 2024-09-21T02:09:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.345+0000 7fd0067fc640 1 -- 172.21.15.92:0/4011749368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000035070 con 0x7fd0101083a0 2024-09-21T02:09:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.346+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0101a6ea0 con 0x7fd0101083a0 2024-09-21T02:09:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.348+0000 7fd01d9aa740 1 Processor -- start 2024-09-21T02:09:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.348+0000 7fd0067fc640 1 -- 172.21.15.92:0/4011749368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd000042030 con 0x7fd0101083a0 2024-09-21T02:09:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.348+0000 7fd01d9aa740 1 -- start start 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.348+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b99977a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993aa1b0 0x55b99977ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 0x55b99977b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9996f0e50 con 0x55b9993b2390 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b9996e0280 con 0x55b9993d3070 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b9996e4090 con 0x55b9993aa1b0 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 0x55b99977b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 0x55b99977b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56288/0 (socket says 172.21.15.92:56288) 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 -- 172.21.15.92:0/822860859 learned_addr learned my addr 172.21.15.92:0/822860859 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0159d3640 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b99977a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0151d2640 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993aa1b0 0x55b99977ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 -- 172.21.15.92:0/822860859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993aa1b0 msgr2=0x55b99977ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993aa1b0 0x55b99977ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 -- 172.21.15.92:0/822860859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 msgr2=0x55b99977a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b99977a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 -- 172.21.15.92:0/822860859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99943e650 con 0x55b9993b2390 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0159d3640 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b99977a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0067fc640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcffc0b6510 0x7fcffc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0149d1640 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 0x55b99977b0d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd00010dc10 tx=0x7fd00010dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d581445cdd6f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.349+0000 7fd0159d3640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcffc0b6510 0x7fcffc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.350+0000 7fd0067fc640 1 -- 172.21.15.92:0/4011749368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd000107920 con 0x7fd0101083a0 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.350+0000 7fcffb7fe640 1 -- 172.21.15.92:0/822860859 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000ca0e0 con 0x55b9993b2390 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.350+0000 7fcffb7fe640 1 -- 172.21.15.92:0/822860859 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0000cd5a0 con 0x55b9993b2390 2024-09-21T02:09:34.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.350+0000 7fcffb7fe640 1 -- 172.21.15.92:0/822860859 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000a1c40 con 0x55b9993b2390 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 -- 172.21.15.92:0/822860859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 msgr2=0x55b99977b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd0159d3640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcffc0b6510 0x7fcffc0b8a20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd00800b6a0 tx=0x7fd00800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 0x55b99977b0d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd00010dc10 tx=0x7fd00010dd00 comp rx=0 tx=0).stop 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 -- 172.21.15.92:0/822860859 shutdown_connections 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993b2390 0x55b99977b0d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993aa1b0 0x55b99977ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/822860859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b99977a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 -- 172.21.15.92:0/822860859 >> 172.21.15.92:0/822860859 conn(0x55b9993c7460 msgr2=0x55b999778540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 -- 172.21.15.92:0/822860859 shutdown_connections 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 -- 172.21.15.92:0/822860859 wait complete. 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 -- start start 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.351+0000 7fd01d9aa740 1 Processor -- start 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b999785ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b9997861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b999791e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b999541700 con 0x55b9993aa1b0 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b99967b110 con 0x55b9993d3070 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b999788bf0 con 0x55b9993b2390 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd01d9aa740 10 client.0 _ll_register_callbacks cb 0x55b999694990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b999785ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b999785ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56292/0 (socket says 172.21.15.92:56292) 2024-09-21T02:09:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 -- 172.21.15.92:0/2094153389 learned_addr learned my addr 172.21.15.92:0/2094153389 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0149d1640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b9997861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0159d3640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b999791e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 -- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 msgr2=0x55b9997861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b9997861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 -- 172.21.15.92:0/2094153389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 msgr2=0x55b999791e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b999791e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 -- 172.21.15.92:0/2094153389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9996a8c40 con 0x55b9993aa1b0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b999785ca0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd00c012770 tx=0x7fd00c012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14962e8d4d3c342c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.352+0000 7fcff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2094153389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c010d60 con 0x55b9993aa1b0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2094153389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd00c005020 con 0x55b9993aa1b0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0000cd5a0 con 0x55b9993aa1b0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2094153389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd00c005220 con 0x55b9993aa1b0 2024-09-21T02:09:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 10 client.16827 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b999794870 con 0x55b9993aa1b0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2094153389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0000cd5a0 con 0x55b9993aa1b0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2094153389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd00c01a430 con 0x55b9993aa1b0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 10 client.16827 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 20 client.16827 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993919f0 0x55b999797f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 4 client.16827 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.353+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9996f2da0 con 0x55b9993919f0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.354+0000 7fd0149d1640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993919f0 0x55b999797f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.354+0000 7fcff8ff9640 10 client.16827 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.354+0000 7fd0149d1640 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993919f0 0x55b999797f10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15533b59c16e8d8b server_cookie=fe7229d078699ee7 in_seq=0 out_seq=0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2094153389 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9996f2da0 con 0x55b9993919f0 2024-09-21T02:09:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fcff8ff9640 10 client.16827 handle_command_reply: tid=0 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 client.16827 shutdown 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 msgr2=0x55b999785ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b999785ca0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd00c012770 tx=0x7fd00c012c30 comp rx=0 tx=0).stop 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 shutdown_connections 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993919f0 0x55b999797f10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993d3070 0x55b999791e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b9997861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fcff8ff9640 0 client.16827 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2094153389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b999785ca0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 >> 172.21.15.92:0/2094153389 conn(0x55b9993c7460 msgr2=0x55b99977d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 shutdown_connections 2024-09-21T02:09:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2094153389 wait complete. 2024-09-21T02:09:34.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.355+0000 7fd01d9aa740 20 client.16827 trim_cache size 0 max 16384 2024-09-21T02:09:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.372+0000 7fd01d9aa740 1 Processor -- start 2024-09-21T02:09:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.372+0000 7fd01d9aa740 1 -- start start 2024-09-21T02:09:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993919f0 0x55b999784b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99977bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993b2390 0x55b999777b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101af520 con 0x55b9993aa1b0 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101b1060 con 0x55b9993b2390 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101afbf0 con 0x55b9993919f0 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99977bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99977bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56308/0 (socket says 172.21.15.92:56308) 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 -- 172.21.15.92:0/1523268762 learned_addr learned my addr 172.21.15.92:0/1523268762 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0151d2640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993919f0 0x55b999784b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0159d3640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993b2390 0x55b999777b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 -- 172.21.15.92:0/1523268762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993919f0 msgr2=0x55b999784b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993919f0 0x55b999784b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 -- 172.21.15.92:0/1523268762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993b2390 msgr2=0x55b999777b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993b2390 0x55b999777b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 -- 172.21.15.92:0/1523268762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99977c940 con 0x55b9993aa1b0 2024-09-21T02:09:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0159d3640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993b2390 0x55b999777b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0151d2640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993919f0 0x55b999784b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.373+0000 7fd0149d1640 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99977bd90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd000006010 tx=0x7fd000009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e319a6b745bd90fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fcffaffd640 1 -- 172.21.15.92:0/1523268762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000049040 con 0x55b9993aa1b0 2024-09-21T02:09:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fcffaffd640 1 -- 172.21.15.92:0/1523268762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00010d3a0 con 0x55b9993aa1b0 2024-09-21T02:09:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fcffaffd640 1 -- 172.21.15.92:0/1523268762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000a1e40 con 0x55b9993aa1b0 2024-09-21T02:09:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 -- 172.21.15.92:0/1523268762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 msgr2=0x55b99977bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99977bd90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd000006010 tx=0x7fd000009260 comp rx=0 tx=0).stop 2024-09-21T02:09:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 -- 172.21.15.92:0/1523268762 shutdown_connections 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993b2390 0x55b999777b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99977bd90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/1523268762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993919f0 0x55b999784b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 -- 172.21.15.92:0/1523268762 >> 172.21.15.92:0/1523268762 conn(0x55b9993c7460 msgr2=0x55b99977b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.374+0000 7fd01d9aa740 1 -- 172.21.15.92:0/1523268762 shutdown_connections 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 -- 172.21.15.92:0/1523268762 wait complete. 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 -- start start 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 Processor -- start 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993919f0 0x55b999789d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99978a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b99978a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101b0990 con 0x55b9993aa1b0 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101b02c0 con 0x55b9993919f0 2024-09-21T02:09:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101ad500 con 0x55b9993b2390 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.375+0000 7fd01d9aa740 10 client.0 _ll_register_callbacks cb 0x7fd00c00a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99978a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99978a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56314/0 (socket says 172.21.15.92:56314) 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 -- 172.21.15.92:0/2457980086 learned_addr learned my addr 172.21.15.92:0/2457980086 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0159d3640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b99978a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993919f0 0x55b999789d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 -- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 msgr2=0x55b99978a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b99978a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 -- 172.21.15.92:0/2457980086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993919f0 msgr2=0x55b999789d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993919f0 0x55b999789d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 -- 172.21.15.92:0/2457980086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9998018d0 con 0x55b9993aa1b0 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993919f0 0x55b999789d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd0149d1640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99978a290 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd000003fd0 tx=0x7fd00010dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2b8e162e4f6436 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fcff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2457980086 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000049070 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2457980086 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd00003b460 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00010d3a0 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2457980086 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0000a2040 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd01d9aa740 10 client.16833 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.376+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9998018d0 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2457980086 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00010d3a0 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2457980086 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd0000d21c0 con 0x55b9993aa1b0 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd01d9aa740 10 client.16833 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd01d9aa740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd01d9aa740 20 client.16833 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993d3070 0x55b999804c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd01d9aa740 4 client.16833 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:09:34.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9996ee850 con 0x55b9993d3070 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993d3070 0x55b999804c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fd0151d2640 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993d3070 0x55b999804c80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb0edd74adac752 server_cookie=8b1dfc940eed1906 in_seq=0 out_seq=0 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.377+0000 7fcff8ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fcff8ff9640 1 -- 172.21.15.92:0/2457980086 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x55b9996ee850 con 0x55b9993d3070 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fcff8ff9640 10 client.16833 handle_command_reply: tid=0 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 client.16833 shutdown 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 msgr2=0x55b99978a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99978a290 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd000003fd0 tx=0x7fd00010dd00 comp rx=0 tx=0).stop 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 shutdown_connections 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9993d3070 0x55b999804c80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9993b2390 0x55b99978a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9993aa1b0 0x55b99978a290 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 --2- 172.21.15.92:0/2457980086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9993919f0 0x55b999789d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 >> 172.21.15.92:0/2457980086 conn(0x55b9993c7460 msgr2=0x55b999787150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fcff8ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 shutdown_connections 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 1 -- 172.21.15.92:0/2457980086 wait complete. 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.378+0000 7fd01d9aa740 20 client.16833 trim_cache size 0 max 16384 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:09:34.384 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:34.385 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:34.388 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcffc0b6510 msgr2=0x7fcffc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcffc0b6510 0x7fcffc0b8a20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd00800b6a0 tx=0x7fd00800e040 comp rx=0 tx=0).stop 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 msgr2=0x7fd01019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd01019aa80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd000031a90 tx=0x7fd000004270 comp rx=0 tx=0).stop 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 shutdown_connections 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcffc0b6510 0x7fcffc0b8a20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd010108cf0 0x7fd01019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0101083a0 0x7fd01019aa80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 --2- 172.21.15.92:0/4011749368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd010102a60 0x7fd01019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 >> 172.21.15.92:0/4011749368 conn(0x7fd010079730 msgr2=0x7fd01006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 shutdown_connections 2024-09-21T02:09:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:34.379+0000 7fd017436640 1 -- 172.21.15.92:0/4011749368 wait complete. 2024-09-21T02:09:35.143 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:09:35.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:09:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:35 smithi092 ceph-mon[31190]: pgmap v498: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:09:35.397 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:35.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:35 smithi136 ceph-mon[34754]: pgmap v498: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:09:35.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:35 smithi099 ceph-mon[34404]: pgmap v498: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 -- 172.21.15.92:0/1179775889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 msgr2=0x7f630c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 --2- 172.21.15.92:0/1179775889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c10f520 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6304009ca0 tx=0x7f630402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 -- 172.21.15.92:0/1179775889 shutdown_connections 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 --2- 172.21.15.92:0/1179775889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c10f520 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 --2- 172.21.15.92:0/1179775889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630c1083a0 0x7f630c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 --2- 172.21.15.92:0/1179775889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630c102a60 0x7f630c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 -- 172.21.15.92:0/1179775889 >> 172.21.15.92:0/1179775889 conn(0x7f630c079730 msgr2=0x7f630c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 -- 172.21.15.92:0/1179775889 shutdown_connections 2024-09-21T02:09:35.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.883+0000 7f6310c15640 1 -- 172.21.15.92:0/1179775889 wait complete. 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.884+0000 7f6310c15640 1 Processor -- start 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.884+0000 7f6310c15640 1 -- start start 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6310c15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630c102a60 0x7f630c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6310c15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630c1083a0 0x7f630c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6310c15640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6310c15640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f630c113a80 con 0x7f630c108cf0 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6310c15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f630c113900 con 0x7f630c102a60 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6310c15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f630c113c00 con 0x7f630c1083a0 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56334/0 (socket says 172.21.15.92:56334) 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 -- 172.21.15.92:0/2988646393 learned_addr learned my addr 172.21.15.92:0/2988646393 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f6309d9b640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630c1083a0 0x7f630c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630a59c640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630c102a60 0x7f630c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 -- 172.21.15.92:0/2988646393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630c1083a0 msgr2=0x7f630c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630c1083a0 0x7f630c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 -- 172.21.15.92:0/2988646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630c102a60 msgr2=0x7f630c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630c102a60 0x7f630c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:35.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 -- 172.21.15.92:0/2988646393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c1a0380 con 0x7f630c108cf0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.885+0000 7f630ad9d640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c19fc50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6304005cc0 tx=0x7f630402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.886+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304003bf0 con 0x7f630c108cf0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.886+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6304003d90 con 0x7f630c108cf0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.886+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630402fbc0 con 0x7f630c108cf0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.886+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c1a0610 con 0x7f630c108cf0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.886+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c1a6dd0 con 0x7f630c108cf0 2024-09-21T02:09:35.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.887+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630c102e70 con 0x7f630c108cf0 2024-09-21T02:09:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.888+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f630402fd60 con 0x7f630c108cf0 2024-09-21T02:09:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.889+0000 7f63037fe640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f62f80b6590 0x7f62f80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.889+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f63041088f0 con 0x7f630c108cf0 2024-09-21T02:09:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.889+0000 7f630a59c640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f62f80b6590 0x7f62f80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:35.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.890+0000 7f630a59c640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f62f80b6590 0x7f62f80b8aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0058c0 tx=0x7f62fc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:35.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:35.892+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63040cf160 con 0x7f630c108cf0 2024-09-21T02:09:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.108+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f630c19b540 con 0x7f630c108cf0 2024-09-21T02:09:36.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.109+0000 7f63037fe640 1 -- 172.21.15.92:0/2988646393 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f630c19b540 con 0x7f630c108cf0 2024-09-21T02:09:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:09:36.110 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:09:36.110 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:09:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f62f80b6590 msgr2=0x7f62f80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f62f80b6590 0x7f62f80b8aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0058c0 tx=0x7f62fc00a4c0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 msgr2=0x7f630c19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:36.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c19fc50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6304005cc0 tx=0x7f630402f380 comp rx=0 tx=0).stop 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 shutdown_connections 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f62f80b6590 0x7f62f80b8aa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c108cf0 0x7f630c19fc50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630c1083a0 0x7f630c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 --2- 172.21.15.92:0/2988646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630c102a60 0x7f630c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.111+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 >> 172.21.15.92:0/2988646393 conn(0x7f630c079730 msgr2=0x7f630c104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.112+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 shutdown_connections 2024-09-21T02:09:36.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:36.112+0000 7f6310c15640 1 -- 172.21.15.92:0/2988646393 wait complete. 2024-09-21T02:09:36.413 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:36 smithi092 ceph-mon[31190]: pgmap v499: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:09:36.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:36 smithi136 ceph-mon[34754]: pgmap v499: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:09:36.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:36 smithi099 ceph-mon[34404]: pgmap v499: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:09:36.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:09:37.033 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:37.459 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:37 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2988646393' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:09:37.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:37 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2988646393' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 -- 172.21.15.92:0/122207311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 msgr2=0x7fe0881026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/122207311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0881026b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00aa10 tx=0x7fe07c02b750 comp rx=0 tx=0).stop 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 -- 172.21.15.92:0/122207311 shutdown_connections 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/122207311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088108530 0x7fe08810ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/122207311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088107be0 0x7fe088107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/122207311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0881026b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 -- 172.21.15.92:0/122207311 >> 172.21.15.92:0/122207311 conn(0x7fe088079ec0 msgr2=0x7fe08807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 -- 172.21.15.92:0/122207311 shutdown_connections 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.550+0000 7fe08eeb2640 1 -- 172.21.15.92:0/122207311 wait complete. 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 Processor -- start 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 -- start start 2024-09-21T02:09:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0880feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088107be0 0x7fe0880ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088108530 0x7fe0880ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe088111a30 con 0x7fe0881022a0 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881118b0 con 0x7fe088108530 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08eeb2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe088111bb0 con 0x7fe088107be0 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08cc4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0880feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08cc4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0880feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56356/0 (socket says 172.21.15.92:56356) 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.551+0000 7fe08cc4e640 1 -- 172.21.15.92:0/196565595 learned_addr learned my addr 172.21.15.92:0/196565595 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe087fff640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088107be0 0x7fe0880ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08d44f640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088108530 0x7fe0880ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08cc4e640 1 -- 172.21.15.92:0/196565595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088107be0 msgr2=0x7fe0880ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088107be0 0x7fe0880ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08cc4e640 1 -- 172.21.15.92:0/196565595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088108530 msgr2=0x7fe0880ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088108530 0x7fe0880ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08cc4e640 1 -- 172.21.15.92:0/196565595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0881a8940 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe087fff640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088107be0 0x7fe0880ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08d44f640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088108530 0x7fe0880ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0880feef0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe07c005940 tx=0x7fe07c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe085ffb640 1 -- 172.21.15.92:0/196565595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c03f070 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0881a8bd0 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe085ffb640 1 -- 172.21.15.92:0/196565595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07c032400 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe085ffb640 1 -- 172.21.15.92:0/196565595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c036040 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.552+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0881a90b0 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 Processor -- start 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe085ffb640 1 -- 172.21.15.92:0/196565595 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe07c042070 con 0x7fe0881022a0 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 -- start start 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce67582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce6758800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6758d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558ce66ceea0 con 0x558ce6390390 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ce66be2d0 con 0x558ce63881b0 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe095426740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ce66c20e0 con 0x558ce63b1070 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe087fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6758d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe087fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6758d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56358/0 (socket says 172.21.15.92:56358) 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe087fff640 1 -- 172.21.15.92:0/3943444414 learned_addr learned my addr 172.21.15.92:0/3943444414 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce6758800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.555+0000 7fe08d44f640 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce67582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe087fff640 1 -- 172.21.15.92:0/3943444414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 msgr2=0x558ce67582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe087fff640 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce67582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe087fff640 1 -- 172.21.15.92:0/3943444414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 msgr2=0x558ce6758800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe087fff640 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce6758800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe087fff640 1 -- 172.21.15.92:0/3943444414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce641c650 con 0x558ce6390390 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce6758800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe085ffb640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0780b6510 0x7fe0780b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe08d44f640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0780b6510 0x7fe0780b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe087fff640 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6758d40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe07400a970 tx=0x7fe07400ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7b61d86abece25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe085ffb640 1 -- 172.21.15.92:0/196565595 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe07c107c60 con 0x7fe0881022a0 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe072ffd640 1 -- 172.21.15.92:0/3943444414 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074015070 con 0x558ce6390390 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe072ffd640 1 -- 172.21.15.92:0/3943444414 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07400cc00 con 0x558ce6390390 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.556+0000 7fe072ffd640 1 -- 172.21.15.92:0/3943444414 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740105b0 con 0x558ce6390390 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 -- 172.21.15.92:0/3943444414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 msgr2=0x558ce6758d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6758d40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe07400a970 tx=0x7fe07400ae30 comp rx=0 tx=0).stop 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe08d44f640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0780b6510 0x7fe0780b8a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe080007f90 tx=0x7fe080007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 -- 172.21.15.92:0/3943444414 shutdown_connections 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6758d40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce6758800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 --2- 172.21.15.92:0/3943444414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce67582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 -- 172.21.15.92:0/3943444414 >> 172.21.15.92:0/3943444414 conn(0x558ce63a5460 msgr2=0x558ce67561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 -- 172.21.15.92:0/3943444414 shutdown_connections 2024-09-21T02:09:37.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 -- 172.21.15.92:0/3943444414 wait complete. 2024-09-21T02:09:37.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 -- start start 2024-09-21T02:09:37.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.557+0000 7fe095426740 1 Processor -- start 2024-09-21T02:09:37.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce67638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6763e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6764370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558ce651f700 con 0x558ce6390390 2024-09-21T02:09:37.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ce6659160 con 0x558ce63881b0 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ce6766aa0 con 0x558ce63b1070 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe095426740 10 client.0 _ll_register_callbacks cb 0x558ce66729e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe08d44f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6764370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe08d44f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6764370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:39022/0 (socket says 172.21.15.92:39022) 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.558+0000 7fe08d44f640 1 -- 172.21.15.92:0/3622838103 learned_addr learned my addr 172.21.15.92:0/3622838103 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6763e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce67638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 -- 172.21.15.92:0/3622838103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 msgr2=0x558ce6764370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6764370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 -- 172.21.15.92:0/3622838103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 msgr2=0x558ce67638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce67638d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 -- 172.21.15.92:0/3622838103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce6686c90 con 0x558ce6390390 2024-09-21T02:09:37.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe087fff640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6763e10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe074004810 tx=0x7fe0740119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03957d30887c211 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe067fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe067fff640 1 -- 172.21.15.92:0/3622838103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074020070 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe067fff640 1 -- 172.21.15.92:0/3622838103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe07401a650 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07400cc00 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe067fff640 1 -- 172.21.15.92:0/3622838103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe074015040 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.559+0000 7fe095426740 10 client.16851 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ce6783730 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe067fff640 1 -- 172.21.15.92:0/3622838103 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07400cc00 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe067fff640 1 -- 172.21.15.92:0/3622838103 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe07402e890 con 0x558ce6390390 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 10 client.16851 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 10 client.16851 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 20 client.16851 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce636f9f0 0x558ce6786a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 4 client.16851 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce66d0df0 con 0x558ce636f9f0 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce636f9f0 0x558ce6786a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.560+0000 7fe067fff640 10 client.16851 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce636f9f0 0x558ce6786a90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fbe874b8c4f1020 server_cookie=5cd0a77f9a0850ce in_seq=0 out_seq=0 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe067fff640 1 -- 172.21.15.92:0/3622838103 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ce66d0df0 con 0x558ce636f9f0 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe067fff640 10 client.16851 handle_command_reply: tid=0 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 client.16851 shutdown 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 msgr2=0x558ce6763e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6763e10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe074004810 tx=0x7fe0740119a0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 shutdown_connections 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce636f9f0 0x558ce6786a90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6764370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6763e10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe067fff640 0 client.16851 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 --2- 172.21.15.92:0/3622838103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63881b0 0x558ce67638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.561+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 >> 172.21.15.92:0/3622838103 conn(0x558ce63a5460 msgr2=0x558ce675b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.562+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 shutdown_connections 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.562+0000 7fe095426740 1 -- 172.21.15.92:0/3622838103 wait complete. 2024-09-21T02:09:37.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.562+0000 7fe095426740 20 client.16851 trim_cache size 0 max 16384 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 Processor -- start 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 -- start start 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63881b0 0x558ce6759cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6765500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63b1070 0x558ce6755760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe088075320 con 0x558ce6390390 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881aab00 con 0x558ce63b1070 2024-09-21T02:09:37.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe095426740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881a9690 con 0x558ce63881b0 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6765500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6765500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56370/0 (socket says 172.21.15.92:56370) 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 -- 172.21.15.92:0/3888968355 learned_addr learned my addr 172.21.15.92:0/3888968355 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 -- 172.21.15.92:0/3888968355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63881b0 msgr2=0x558ce6759cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63881b0 0x558ce6759cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63881b0 0x558ce6759cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 -- 172.21.15.92:0/3888968355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63b1070 msgr2=0x558ce6755760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63b1070 0x558ce6755760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 -- 172.21.15.92:0/3888968355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce676fc90 con 0x558ce6390390 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.575+0000 7fe087fff640 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6765500 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe074001eb0 tx=0x7fe07400c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e424289a73647c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe0727fc640 1 -- 172.21.15.92:0/3888968355 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074020040 con 0x558ce6390390 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe0727fc640 1 -- 172.21.15.92:0/3888968355 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe074011e50 con 0x558ce6390390 2024-09-21T02:09:37.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe0727fc640 1 -- 172.21.15.92:0/3888968355 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074023610 con 0x558ce6390390 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 -- 172.21.15.92:0/3888968355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 msgr2=0x558ce6765500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6765500 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe074001eb0 tx=0x7fe07400c560 comp rx=0 tx=0).stop 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 -- 172.21.15.92:0/3888968355 shutdown_connections 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce63b1070 0x558ce6755760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce6390390 0x558ce6765500 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 --2- 172.21.15.92:0/3888968355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63881b0 0x558ce6759cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 -- 172.21.15.92:0/3888968355 >> 172.21.15.92:0/3888968355 conn(0x558ce63a5460 msgr2=0x558ce6764df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.576+0000 7fe095426740 1 -- 172.21.15.92:0/3888968355 shutdown_connections 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 -- 172.21.15.92:0/3888968355 wait complete. 2024-09-21T02:09:37.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 -- start start 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 Processor -- start 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 0x558ce676c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce6390390 0x558ce6767a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6768040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881aa430 con 0x558ce63881b0 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0881a9d60 con 0x558ce6390390 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe088073300 con 0x558ce63b1070 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe095426740 10 client.0 _ll_register_callbacks cb 0x7fe07400a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.577+0000 7fe08cc4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 0x558ce676c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 0x558ce676c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56374/0 (socket says 172.21.15.92:56374) 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 -- 172.21.15.92:0/1233524948 learned_addr learned my addr 172.21.15.92:0/1233524948 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08d44f640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6768040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 -- 172.21.15.92:0/1233524948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 msgr2=0x558ce6768040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6768040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe087fff640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce6390390 0x558ce6767a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 -- 172.21.15.92:0/1233524948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce6390390 msgr2=0x558ce6767a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce6390390 0x558ce6767a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 -- 172.21.15.92:0/1233524948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce6768820 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe08cc4e640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 0x558ce676c590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0cdd60 tx=0x7fe07c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4af0cd9ce417bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe067fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe067fff640 1 -- 172.21.15.92:0/1233524948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c049040 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe067fff640 1 -- 172.21.15.92:0/1233524948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe07c0ca040 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe074011e50 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe067fff640 1 -- 172.21.15.92:0/1233524948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe07c0ce4d0 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe095426740 10 client.16857 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.578+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ce67f12b0 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe067fff640 1 -- 172.21.15.92:0/1233524948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe074011e50 con 0x558ce63881b0 2024-09-21T02:09:37.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe067fff640 1 -- 172.21.15.92:0/1233524948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe07c00b040 con 0x558ce63881b0 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe095426740 10 client.16857 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe095426740 10 client.16857 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe095426740 20 client.16857 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe095426740 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce6344da0 0x558ce67f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe095426740 4 client.16857 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ce66cc8a0 con 0x558ce6344da0 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe087fff640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce6344da0 0x558ce67f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe067fff640 10 client.16857 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.579+0000 7fe087fff640 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce6344da0 0x558ce67f3a70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=459cc64ad8cabf09 server_cookie=61790276193ec456 in_seq=0 out_seq=0 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe067fff640 1 -- 172.21.15.92:0/1233524948 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ce66cc8a0 con 0x558ce6344da0 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe067fff640 10 client.16857 handle_command_reply: tid=0 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 client.16857 shutdown 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 msgr2=0x558ce676c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 0x558ce676c590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0cdd60 tx=0x7fe07c004610 comp rx=0 tx=0).stop 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 shutdown_connections 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x558ce6344da0 0x558ce67f3a70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce63b1070 0x558ce6768040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ce6390390 0x558ce6767a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 --2- 172.21.15.92:0/1233524948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ce63881b0 0x558ce676c590 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 >> 172.21.15.92:0/1233524948 conn(0x558ce63a5460 msgr2=0x558ce6759a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe067fff640 0 client.16857 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 shutdown_connections 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 1 -- 172.21.15.92:0/1233524948 wait complete. 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.580+0000 7fe095426740 20 client.16857 trim_cache size 0 max 16384 2024-09-21T02:09:37.585 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:09:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0780b6510 msgr2=0x7fe0780b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0780b6510 0x7fe0780b8a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe080007f90 tx=0x7fe080007520 comp rx=0 tx=0).stop 2024-09-21T02:09:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 msgr2=0x7fe0880feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0880feef0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe07c005940 tx=0x7fe07c004360 comp rx=0 tx=0).stop 2024-09-21T02:09:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 shutdown_connections 2024-09-21T02:09:37.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0780b6510 0x7fe0780b8a20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe088108530 0x7fe0880ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe088107be0 0x7fe0880ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 --2- 172.21.15.92:0/196565595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0881022a0 0x7fe0880feef0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:37.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 >> 172.21.15.92:0/196565595 conn(0x7fe088079ec0 msgr2=0x7fe0881012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:37.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 shutdown_connections 2024-09-21T02:09:37.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:37.581+0000 7fe08eeb2640 1 -- 172.21.15.92:0/196565595 wait complete. 2024-09-21T02:09:37.587 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:37 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2988646393' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:09:38.280 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:38 smithi092 ceph-mon[31190]: pgmap v500: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:09:38.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:09:38.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:38 smithi136 ceph-mon[34754]: pgmap v500: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:09:38.550 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:38.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:38 smithi099 ceph-mon[34404]: pgmap v500: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:09:39.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.049+0000 7fb43134a640 1 -- 172.21.15.92:0/4160519107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 msgr2=0x7fb42c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.049+0000 7fb43134a640 1 --2- 172.21.15.92:0/4160519107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c1087b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb414009900 tx=0x7fb41402a410 comp rx=0 tx=0).stop 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 -- 172.21.15.92:0/4160519107 shutdown_connections 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 --2- 172.21.15.92:0/4160519107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42c108cf0 0x7fb42c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 --2- 172.21.15.92:0/4160519107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c1087b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 --2- 172.21.15.92:0/4160519107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c102a60 0x7fb42c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 -- 172.21.15.92:0/4160519107 >> 172.21.15.92:0/4160519107 conn(0x7fb42c079730 msgr2=0x7fb42c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 -- 172.21.15.92:0/4160519107 shutdown_connections 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 -- 172.21.15.92:0/4160519107 wait complete. 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 Processor -- start 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 -- start start 2024-09-21T02:09:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42c102a60 0x7fb42c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.050+0000 7fb43134a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb43134a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c108cf0 0x7fb42c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb43134a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1129a0 con 0x7fb42c1083a0 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb43134a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c112820 con 0x7fb42c102a60 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb43134a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c112b20 con 0x7fb42c108cf0 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56400/0 (socket says 172.21.15.92:56400) 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 -- 172.21.15.92:0/2215490981 learned_addr learned my addr 172.21.15.92:0/2215490981 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb42affd640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42c102a60 0x7fb42c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c108cf0 0x7fb42c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 -- 172.21.15.92:0/2215490981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c108cf0 msgr2=0x7fb42c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c108cf0 0x7fb42c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 -- 172.21.15.92:0/2215490981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42c102a60 msgr2=0x7fb42c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42c102a60 0x7fb42c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 -- 172.21.15.92:0/2215490981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c1a04d0 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c108cf0 0x7fb42c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb423fff640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c19ab00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb414032fd0 tx=0x7fb414004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb428ff9640 1 -- 172.21.15.92:0/2215490981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41403f030 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb428ff9640 1 -- 172.21.15.92:0/2215490981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb414004490 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c1a0760 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb428ff9640 1 -- 172.21.15.92:0/2215490981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414035070 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.051+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c1a6f20 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.054+0000 7fb4378be740 1 Processor -- start 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.054+0000 7fb4378be740 1 -- start start 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb428ff9640 1 -- 172.21.15.92:0/2215490981 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb414042030 con 0x7fb42c1083a0 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb4378be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb72070 0x55942cf192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb4378be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb4378be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb51390 0x55942cf19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb4378be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55942ce8fea0 con 0x55942cb491b0 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb4378be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55942ce7f2d0 con 0x55942cb51390 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb4378be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55942ce830e0 con 0x55942cb72070 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf19800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56414/0 (socket says 172.21.15.92:56414) 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 -- 172.21.15.92:0/3177529409 learned_addr learned my addr 172.21.15.92:0/3177529409 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb72070 0x55942cf192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb423fff640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb51390 0x55942cf19d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 -- 172.21.15.92:0/3177529409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb72070 msgr2=0x55942cf192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb72070 0x55942cf192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 -- 172.21.15.92:0/3177529409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb51390 msgr2=0x55942cf19d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb51390 0x55942cf19d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 -- 172.21.15.92:0/3177529409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942cbdd650 con 0x55942cb491b0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb72070 0x55942cf192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb423fff640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb51390 0x55942cf19d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb428ff9640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4180b6510 0x7fb4180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42affd640 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf19800 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009a70 tx=0x7fb41c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50700fd3187ee519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.055+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4180b6510 0x7fb4180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb4217fa640 1 -- 172.21.15.92:0/3177529409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017070 con 0x55942cb491b0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb428ff9640 1 -- 172.21.15.92:0/2215490981 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb414107810 con 0x7fb42c1083a0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb4217fa640 1 -- 172.21.15.92:0/3177529409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41c013470 con 0x55942cb491b0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb4217fa640 1 -- 172.21.15.92:0/3177529409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c012630 con 0x55942cb491b0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb4378be740 1 -- 172.21.15.92:0/3177529409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 msgr2=0x55942cf19800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb4378be740 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf19800 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009a70 tx=0x7fb41c009f30 comp rx=0 tx=0).stop 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.056+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4180b6510 0x7fb4180b8a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb4240095f0 tx=0x7fb424009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- 172.21.15.92:0/3177529409 shutdown_connections 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb51390 0x55942cf19d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf19800 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 --2- 172.21.15.92:0/3177529409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb72070 0x55942cf192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- 172.21.15.92:0/3177529409 >> 172.21.15.92:0/3177529409 conn(0x55942cb66460 msgr2=0x55942cf171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- 172.21.15.92:0/3177529409 shutdown_connections 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- 172.21.15.92:0/3177529409 wait complete. 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- start start 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 Processor -- start 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf24f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cf30dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55942cce0700 con 0x55942cb491b0 2024-09-21T02:09:39.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55942ce1a160 con 0x55942cb72070 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55942cf27aa0 con 0x55942cb51390 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.057+0000 7fb4378be740 10 client.0 _ll_register_callbacks cb 0x55942ce339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf249f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56418/0 (socket says 172.21.15.92:56418) 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 -- 172.21.15.92:0/1250697184 learned_addr learned my addr 172.21.15.92:0/1250697184 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb423fff640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf24f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cf30dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 -- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 msgr2=0x55942cf24f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf24f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 -- 172.21.15.92:0/1250697184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 msgr2=0x55942cf30dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cf30dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 -- 172.21.15.92:0/1250697184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942ce47c90 con 0x55942cb491b0 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cf30dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb42affd640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf249f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb41c013bd0 tx=0x7fb41c006ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de26bba20950ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb4127fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb4127fc640 1 -- 172.21.15.92:0/1250697184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017050 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb4127fc640 1 -- 172.21.15.92:0/1250697184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb41c01c650 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb41c013470 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.058+0000 7fb4127fc640 1 -- 172.21.15.92:0/1250697184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb41c02c070 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 10 client.16869 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55942cf33760 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4127fc640 1 -- 172.21.15.92:0/1250697184 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41c013470 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4127fc640 1 -- 172.21.15.92:0/1250697184 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb41c0267c0 con 0x55942cb491b0 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 10 client.16869 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 10 client.16869 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 20 client.16869 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb309f0 0x55942cf36b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 4 client.16869 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55942ce91df0 con 0x55942cb309f0 2024-09-21T02:09:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.059+0000 7fb423fff640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb309f0 0x55942cf36b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.060+0000 7fb4127fc640 10 client.16869 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:09:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.060+0000 7fb423fff640 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb309f0 0x55942cf36b70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=677448c668dfe66 server_cookie=2dca4968c00a63f9 in_seq=0 out_seq=0 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.060+0000 7fb4127fc640 1 -- 172.21.15.92:0/1250697184 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55942ce91df0 con 0x55942cb309f0 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.060+0000 7fb4127fc640 10 client.16869 handle_command_reply: tid=0 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.060+0000 7fb4378be740 1 client.16869 shutdown 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 msgr2=0x55942cf249f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf249f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb41c013bd0 tx=0x7fb41c006ed0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 shutdown_connections 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb309f0 0x55942cf36b70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cf30dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf24f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4127fc640 0 client.16869 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:09:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 --2- 172.21.15.92:0/1250697184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf249f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 >> 172.21.15.92:0/1250697184 conn(0x55942cb66460 msgr2=0x55942cf1c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 shutdown_connections 2024-09-21T02:09:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 1 -- 172.21.15.92:0/1250697184 wait complete. 2024-09-21T02:09:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.061+0000 7fb4378be740 20 client.16869 trim_cache size 0 max 16384 2024-09-21T02:09:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 Processor -- start 2024-09-21T02:09:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 -- start start 2024-09-21T02:09:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb491b0 0x55942cf26620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 0x55942cf21be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1af4d0 con 0x55942cb72070 2024-09-21T02:09:39.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1b1010 con 0x55942cb491b0 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb4378be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1afba0 con 0x55942cb51390 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb42b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 0x55942cf21be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb42b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 0x55942cf21be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56428/0 (socket says 172.21.15.92:56428) 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.074+0000 7fb42b7fe640 1 -- 172.21.15.92:0/2996820957 learned_addr learned my addr 172.21.15.92:0/2996820957 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42affd640 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb491b0 0x55942cf26620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb423fff640 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42b7fe640 1 -- 172.21.15.92:0/2996820957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 msgr2=0x55942cf30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf30d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42b7fe640 1 -- 172.21.15.92:0/2996820957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb491b0 msgr2=0x55942cf26620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb491b0 0x55942cf26620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42b7fe640 1 -- 172.21.15.92:0/2996820957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942cf17b20 con 0x55942cb72070 2024-09-21T02:09:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 0x55942cf21be0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb424009970 tx=0x7fb424019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476a928d2dcdd39c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb420ff9640 1 -- 172.21.15.92:0/2996820957 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42402c070 con 0x55942cb72070 2024-09-21T02:09:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb420ff9640 1 -- 172.21.15.92:0/2996820957 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb424010ce0 con 0x55942cb72070 2024-09-21T02:09:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.075+0000 7fb420ff9640 1 -- 172.21.15.92:0/2996820957 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424027d60 con 0x55942cb72070 2024-09-21T02:09:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 -- 172.21.15.92:0/2996820957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 msgr2=0x55942cf21be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 0x55942cf21be0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb424009970 tx=0x7fb424019000 comp rx=0 tx=0).stop 2024-09-21T02:09:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 -- 172.21.15.92:0/2996820957 shutdown_connections 2024-09-21T02:09:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb72070 0x55942cf21be0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 --2- 172.21.15.92:0/2996820957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb491b0 0x55942cf26620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 -- 172.21.15.92:0/2996820957 >> 172.21.15.92:0/2996820957 conn(0x55942cb66460 msgr2=0x55942cf25f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 -- 172.21.15.92:0/2996820957 shutdown_connections 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 -- 172.21.15.92:0/2996820957 wait complete. 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.076+0000 7fb4378be740 1 -- start start 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 Processor -- start 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf37fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cfa02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1b0940 con 0x55942cb491b0 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1b0270 con 0x55942cb72070 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1ad4b0 con 0x55942cb51390 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4378be740 10 client.0 _ll_register_callbacks cb 0x7fb41c0098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf37fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf37fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56444/0 (socket says 172.21.15.92:56444) 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 -- 172.21.15.92:0/469782665 learned_addr learned my addr 172.21.15.92:0/469782665 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb423fff640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cfa02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 -- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 msgr2=0x55942cf385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 -- 172.21.15.92:0/469782665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 msgr2=0x55942cfa02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cfa02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 -- 172.21.15.92:0/469782665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942cfa0830 con 0x55942cb491b0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42b7fe640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cfa02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb42affd640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf37fe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009a70 tx=0x7fb41c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3ef70ee0a2d849 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.077+0000 7fb4127fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4127fc640 1 -- 172.21.15.92:0/469782665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017040 con 0x55942cb491b0 2024-09-21T02:09:39.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb424010ce0 con 0x55942cb491b0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4127fc640 1 -- 172.21.15.92:0/469782665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb41c026800 con 0x55942cb491b0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4127fc640 1 -- 172.21.15.92:0/469782665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb41c03f070 con 0x55942cb491b0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 10 client.16875 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55942cfa19c0 con 0x55942cb491b0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4127fc640 1 -- 172.21.15.92:0/469782665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb424010ce0 con 0x55942cb491b0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4127fc640 1 -- 172.21.15.92:0/469782665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb41c030c20 con 0x55942cb491b0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 10 client.16875 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 10 client.16875 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 20 client.16875 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb05da0 0x55942cfa4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 4 client.16875 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55942ce8d8a0 con 0x55942cb05da0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.078+0000 7fb423fff640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb05da0 0x55942cfa4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:39.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4127fc640 10 client.16875 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb423fff640 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb05da0 0x55942cfa4150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=610079ef488a37d4 server_cookie=e6cf08802935b3e3 in_seq=0 out_seq=0 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4127fc640 1 -- 172.21.15.92:0/469782665 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55942ce8d8a0 con 0x55942cb05da0 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4127fc640 10 client.16875 handle_command_reply: tid=0 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 client.16875 shutdown 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 msgr2=0x55942cf37fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf37fe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009a70 tx=0x7fb41c0047d0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 shutdown_connections 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55942cb05da0 0x55942cfa4150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55942cb72070 0x55942cfa02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55942cb51390 0x55942cf385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4127fc640 0 client.16875 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 --2- 172.21.15.92:0/469782665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55942cb491b0 0x55942cf37fe0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 >> 172.21.15.92:0/469782665 conn(0x55942cb66460 msgr2=0x55942cf1acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 shutdown_connections 2024-09-21T02:09:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.079+0000 7fb4378be740 1 -- 172.21.15.92:0/469782665 wait complete. 2024-09-21T02:09:39.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb4378be740 20 client.16875 trim_cache size 0 max 16384 2024-09-21T02:09:39.086 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4180b6510 msgr2=0x7fb4180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4180b6510 0x7fb4180b8a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb4240095f0 tx=0x7fb424009580 comp rx=0 tx=0).stop 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 msgr2=0x7fb42c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c19ab00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb414032fd0 tx=0x7fb414004270 comp rx=0 tx=0).stop 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 shutdown_connections 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb4180b6510 0x7fb4180b8a20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb42c108cf0 0x7fb42c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb42c1083a0 0x7fb42c19ab00 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 --2- 172.21.15.92:0/2215490981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42c102a60 0x7fb42c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 >> 172.21.15.92:0/2215490981 conn(0x7fb42c079730 msgr2=0x7fb42c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 shutdown_connections 2024-09-21T02:09:39.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:39.080+0000 7fb43134a640 1 -- 172.21.15.92:0/2215490981 wait complete. 2024-09-21T02:09:39.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:09:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:40.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:40 smithi092 ceph-mon[31190]: pgmap v501: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:40.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:40 smithi136 ceph-mon[34754]: pgmap v501: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:40.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:40 smithi099 ceph-mon[34404]: pgmap v501: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:09:40.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 -- 172.21.15.92:0/4139214870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc105380 msgr2=0x7fb8cc10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 --2- 172.21.15.92:0/4139214870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc105380 0x7fb8cc10d8d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009900 tx=0x7fb8b402a410 comp rx=0 tx=0).stop 2024-09-21T02:09:40.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 -- 172.21.15.92:0/4139214870 shutdown_connections 2024-09-21T02:09:40.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 --2- 172.21.15.92:0/4139214870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8cc10de60 0x7fb8cc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 --2- 172.21.15.92:0/4139214870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc105380 0x7fb8cc10d8d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 --2- 172.21.15.92:0/4139214870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8cc104a30 0x7fb8cc104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 -- 172.21.15.92:0/4139214870 >> 172.21.15.92:0/4139214870 conn(0x7fb8cc079760 msgr2=0x7fb8cc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 -- 172.21.15.92:0/4139214870 shutdown_connections 2024-09-21T02:09:40.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.716+0000 7fb8d262d640 1 -- 172.21.15.92:0/4139214870 wait complete. 2024-09-21T02:09:40.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 Processor -- start 2024-09-21T02:09:40.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 -- start start 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 0x7fb8cc19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8cc105380 0x7fb8cc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8cc10de60 0x7fb8cc19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc113aa0 con 0x7fb8cc104a30 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc113920 con 0x7fb8cc10de60 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8d262d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc113c20 con 0x7fb8cc105380 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 0x7fb8cc19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 0x7fb8cc19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56454/0 (socket says 172.21.15.92:56454) 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.717+0000 7fb8cbfff640 1 -- 172.21.15.92:0/1011364524 learned_addr learned my addr 172.21.15.92:0/1011364524 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8cc105380 0x7fb8cc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8cbfff640 1 -- 172.21.15.92:0/1011364524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8cc105380 msgr2=0x7fb8cc19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8cc105380 0x7fb8cc19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8cbfff640 1 -- 172.21.15.92:0/1011364524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8cc10de60 msgr2=0x7fb8cc19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8cc10de60 0x7fb8cc19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8cc10de60 0x7fb8cc19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8cbfff640 1 -- 172.21.15.92:0/1011364524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc1a0440 con 0x7fb8cc104a30 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 0x7fb8cc19a4d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00d770 tx=0x7fb8bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8c9ffb640 1 -- 172.21.15.92:0/1011364524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc017070 con 0x7fb8cc104a30 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8c9ffb640 1 -- 172.21.15.92:0/1011364524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8bc009c30 con 0x7fb8cc104a30 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc1a0730 con 0x7fb8cc104a30 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8c9ffb640 1 -- 172.21.15.92:0/1011364524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc003cc0 con 0x7fb8cc104a30 2024-09-21T02:09:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.718+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1a6e90 con 0x7fb8cc104a30 2024-09-21T02:09:40.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 Processor -- start 2024-09-21T02:09:40.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8c9ffb640 1 -- 172.21.15.92:0/1011364524 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb8bc003e60 con 0x7fb8cc104a30 2024-09-21T02:09:40.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 -- start start 2024-09-21T02:09:40.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x56029463d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029463dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x56029463e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5602945b3ea0 con 0x56029426d1b0 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5602945a32d0 con 0x560294275390 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5602945a70e0 con 0x560294296070 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029463dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029463dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56468/0 (socket says 172.21.15.92:56468) 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 -- 172.21.15.92:0/4261697183 learned_addr learned my addr 172.21.15.92:0/4261697183 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x56029463d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x56029463e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 -- 172.21.15.92:0/4261697183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 msgr2=0x56029463d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x56029463d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 -- 172.21.15.92:0/4261697183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 msgr2=0x56029463e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x56029463e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8cbfff640 1 -- 172.21.15.92:0/4261697183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560294301650 con 0x56029426d1b0 2024-09-21T02:09:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.722+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x56029463e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8c9ffb640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029463dbe0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0eae80 tx=0x7fb8bc0ef7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32978082268e62c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8c9ffb640 1 -- 172.21.15.92:0/1011364524 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb8bc0e6ce0 con 0x7fb8cc104a30 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8c27fc640 1 -- 172.21.15.92:0/4261697183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc020020 con 0x56029426d1b0 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8c27fc640 1 -- 172.21.15.92:0/4261697183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8bc0ac960 con 0x56029426d1b0 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.723+0000 7fb8c27fc640 1 -- 172.21.15.92:0/4261697183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc0cee60 con 0x56029426d1b0 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4261697183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 msgr2=0x56029463dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400be30 tx=0x7fb8c400b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029463dbe0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0eae80 tx=0x7fb8bc0ef7b0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4261697183 shutdown_connections 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x56029463e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029463dbe0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4261697183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x56029463d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4261697183 >> 172.21.15.92:0/4261697183 conn(0x56029428a460 msgr2=0x56029463b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4261697183 shutdown_connections 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4261697183 wait complete. 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.724+0000 7fb8d8ba1740 1 -- start start 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 Processor -- start 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56029426d1b0 0x560294648d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x560294649270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 0x5602946497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560294404700 con 0x560294296070 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56029453e160 con 0x560294275390 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56029464be80 con 0x56029426d1b0 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d8ba1740 10 client.0 _ll_register_callbacks cb 0x5602945579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 0x5602946497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 0x5602946497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56472/0 (socket says 172.21.15.92:56472) 2024-09-21T02:09:40.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 -- 172.21.15.92:0/4227741033 learned_addr learned my addr 172.21.15.92:0/4227741033 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56029426d1b0 0x560294648d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x560294649270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 -- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56029426d1b0 msgr2=0x560294648d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56029426d1b0 0x560294648d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 -- 172.21.15.92:0/4227741033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 msgr2=0x560294649270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x560294649270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8d0bca640 1 -- 172.21.15.92:0/4227741033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029456bc90 con 0x560294296070 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.725+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x560294649270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 0x5602946497f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400eba0 tx=0x7fb8c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa3bcff6f0cc90e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/4227741033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400de30 con 0x560294296070 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/4227741033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb8c4025400 con 0x560294296070 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8bc0ac960 con 0x560294296070 2024-09-21T02:09:40.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/4227741033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8c4025600 con 0x560294296070 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 10 client.16887 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560294657bb0 con 0x560294296070 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/4227741033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8bc0ac960 con 0x560294296070 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/4227741033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8c401b4b0 con 0x560294296070 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 10 client.16887 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 10 client.16887 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 20 client.16887 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.726+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5602942549f0 0x56029465b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.727+0000 7fb8d8ba1740 4 client.16887 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.727+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602945b5df0 con 0x5602942549f0 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.727+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5602942549f0 0x56029465b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.727+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5602942549f0 0x56029465b0d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50a32c55cedc2c2c server_cookie=c5059ae1d1e38444 in_seq=0 out_seq=0 2024-09-21T02:09:40.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.727+0000 7fb8ab7fe640 10 client.16887 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:09:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/4227741033 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602945b5df0 con 0x5602942549f0 2024-09-21T02:09:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8ab7fe640 10 client.16887 handle_command_reply: tid=0 2024-09-21T02:09:40.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 client.16887 shutdown 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 msgr2=0x5602946497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 0x5602946497f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400eba0 tx=0x7fb8c4004a40 comp rx=0 tx=0).stop 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 shutdown_connections 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5602942549f0 0x56029465b0d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294296070 0x5602946497f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294275390 0x560294649270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/4227741033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56029426d1b0 0x560294648d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.728+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 >> 172.21.15.92:0/4227741033 conn(0x56029428a460 msgr2=0x560294640830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.729+0000 7fb8ab7fe640 0 client.16887 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.729+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 shutdown_connections 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.729+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/4227741033 wait complete. 2024-09-21T02:09:40.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.729+0000 7fb8d8ba1740 20 client.16887 trim_cache size 0 max 16384 2024-09-21T02:09:40.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.741+0000 7fb8d8ba1740 1 Processor -- start 2024-09-21T02:09:40.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 -- start start 2024-09-21T02:09:40.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029464a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294275390 0x56029463abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294296070 0x56029463b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1af540 con 0x56029426d1b0 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b1080 con 0x560294296070 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1afc10 con 0x560294275390 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029464a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029464a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56482/0 (socket says 172.21.15.92:56482) 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 -- 172.21.15.92:0/135394061 learned_addr learned my addr 172.21.15.92:0/135394061 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294275390 0x56029463abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 -- 172.21.15.92:0/135394061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294275390 msgr2=0x56029463abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294275390 0x56029463abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294296070 0x56029463b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 -- 172.21.15.92:0/135394061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294296070 msgr2=0x56029463b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294296070 0x56029463b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 -- 172.21.15.92:0/135394061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560294640500 con 0x56029426d1b0 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294275390 0x56029463abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.742+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029464a960 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0e4b30 tx=0x7fb8bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e984bd716d5398fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8c1ffb640 1 -- 172.21.15.92:0/135394061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc020040 con 0x56029426d1b0 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8c1ffb640 1 -- 172.21.15.92:0/135394061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8bc0af070 con 0x56029426d1b0 2024-09-21T02:09:40.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8c1ffb640 1 -- 172.21.15.92:0/135394061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc0cee60 con 0x56029426d1b0 2024-09-21T02:09:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/135394061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 msgr2=0x56029464a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029464a960 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0e4b30 tx=0x7fb8bc0046c0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/135394061 shutdown_connections 2024-09-21T02:09:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560294296070 0x56029463b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294275390 0x56029463abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/135394061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56029426d1b0 0x56029464a960 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/135394061 >> 172.21.15.92:0/135394061 conn(0x56029428a460 msgr2=0x56029464a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/135394061 shutdown_connections 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.743+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/135394061 wait complete. 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 -- start start 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 Processor -- start 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56029426d1b0 0x5602946bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 0x5602946bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x5602946be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b09b0 con 0x560294275390 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b02e0 con 0x56029426d1b0 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1ad520 con 0x560294296070 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8d8ba1740 10 client.0 _ll_register_callbacks cb 0x7fb8c400e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 0x5602946bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 0x5602946bdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56486/0 (socket says 172.21.15.92:56486) 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8c3fff640 1 -- 172.21.15.92:0/659647647 learned_addr learned my addr 172.21.15.92:0/659647647 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.744+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56029426d1b0 0x5602946bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8d0bca640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x5602946be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8c3fff640 1 -- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 msgr2=0x5602946be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x5602946be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8c3fff640 1 -- 172.21.15.92:0/659647647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56029426d1b0 msgr2=0x5602946bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56029426d1b0 0x5602946bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8c3fff640 1 -- 172.21.15.92:0/659647647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602946c8af0 con 0x560294275390 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8c3fff640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 0x5602946bdfb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009900 tx=0x7fb8b4031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247eb320e24a11a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/659647647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4035050 con 0x560294275390 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8bc0af070 con 0x560294275390 2024-09-21T02:09:40.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/659647647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb8b4004450 con 0x560294275390 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/659647647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8b4005030 con 0x560294275390 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8d8ba1740 10 client.16893 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602946c9d10 con 0x560294275390 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.745+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/659647647 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8bc0af070 con 0x560294275390 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/659647647 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8b40035f0 con 0x560294275390 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8d8ba1740 10 client.16893 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8d8ba1740 10 client.16893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8d8ba1740 20 client.16893 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560294229da0 0x5602946cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8d8ba1740 4 client.16893 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602945b18a0 con 0x560294229da0 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560294229da0 0x5602946cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8ab7fe640 10 client.16893 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:09:40.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.746+0000 7fb8cbfff640 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560294229da0 0x5602946cc500 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60f6e5c35ef2df12 server_cookie=d7a96f6d0b026a36 in_seq=0 out_seq=0 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8ab7fe640 1 -- 172.21.15.92:0/659647647 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602945b18a0 con 0x560294229da0 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8ab7fe640 10 client.16893 handle_command_reply: tid=0 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 client.16893 shutdown 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 msgr2=0x5602946bdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 0x5602946bdfb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009900 tx=0x7fb8b4031810 comp rx=0 tx=0).stop 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 shutdown_connections 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560294229da0 0x5602946cc500 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560294296070 0x5602946be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560294275390 0x5602946bdfb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 --2- 172.21.15.92:0/659647647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56029426d1b0 0x5602946bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 >> 172.21.15.92:0/659647647 conn(0x56029428a460 msgr2=0x560294654a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8ab7fe640 0 client.16893 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 shutdown_connections 2024-09-21T02:09:40.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 1 -- 172.21.15.92:0/659647647 wait complete. 2024-09-21T02:09:40.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.747+0000 7fb8d8ba1740 20 client.16893 trim_cache size 0 max 16384 2024-09-21T02:09:40.753 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:09:40.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb8b80b6590 msgr2=0x7fb8b80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400be30 tx=0x7fb8c400b580 comp rx=0 tx=0).stop 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 msgr2=0x7fb8cc19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 0x7fb8cc19a4d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00d770 tx=0x7fb8bc00dc30 comp rx=0 tx=0).stop 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 shutdown_connections 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb8b80b6590 0x7fb8b80b8aa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8cc10de60 0x7fb8cc19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8cc105380 0x7fb8cc19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 --2- 172.21.15.92:0/1011364524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8cc104a30 0x7fb8cc19a4d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 >> 172.21.15.92:0/1011364524 conn(0x7fb8cc079760 msgr2=0x7fb8cc100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 shutdown_connections 2024-09-21T02:09:40.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:40.748+0000 7fb8d262d640 1 -- 172.21.15.92:0/1011364524 wait complete. 2024-09-21T02:09:41.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:09:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 -- 172.21.15.92:0/2608410851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 msgr2=0x7f5d1810f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 --2- 172.21.15.92:0/2608410851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1810f5e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009ca0 tx=0x7f5d1002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 -- 172.21.15.92:0/2608410851 shutdown_connections 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 --2- 172.21.15.92:0/2608410851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1810f5e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 --2- 172.21.15.92:0/2608410851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d18108460 0x7f5d18108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 --2- 172.21.15.92:0/2608410851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 0x7f5d18103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 -- 172.21.15.92:0/2608410851 >> 172.21.15.92:0/2608410851 conn(0x7f5d18077cc0 msgr2=0x7f5d18078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 -- 172.21.15.92:0/2608410851 shutdown_connections 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 -- 172.21.15.92:0/2608410851 wait complete. 2024-09-21T02:09:42.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.301+0000 7f5d1df21640 1 Processor -- start 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 -- start start 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 0x7f5d181993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d18108460 0x7f5d18199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18112a70 con 0x7f5d18108db0 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181128f0 con 0x7f5d18103180 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d1df21640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18112bf0 con 0x7f5d18108460 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56510/0 (socket says 172.21.15.92:56510) 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 -- 172.21.15.92:0/424425561 learned_addr learned my addr 172.21.15.92:0/424425561 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d18108460 0x7f5d18199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d177fe640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 0x7f5d181993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 -- 172.21.15.92:0/424425561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d18108460 msgr2=0x7f5d18199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d18108460 0x7f5d18199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 -- 172.21.15.92:0/424425561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 msgr2=0x7f5d181993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 0x7f5d181993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 -- 172.21.15.92:0/424425561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1819f350 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d177fe640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 0x7f5d181993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.302+0000 7f5d17fff640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1819ec20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5d100098d0 tx=0x7f5d100317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.303+0000 7f5d157fa640 1 -- 172.21.15.92:0/424425561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10003b70 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.303+0000 7f5d157fa640 1 -- 172.21.15.92:0/424425561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d10003d10 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.303+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1819f5e0 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.303+0000 7f5d157fa640 1 -- 172.21.15.92:0/424425561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1002fa20 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.303+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d181a5da0 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.305+0000 7f5d157fa640 1 -- 172.21.15.92:0/424425561 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5d1002fc40 con 0x7f5d18108db0 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.305+0000 7f5d24495740 1 Processor -- start 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.305+0000 7f5d24495740 1 -- start start 2024-09-21T02:09:42.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d24495740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d24495740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3997be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d24495740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b3998120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d24495740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5620b390dea0 con 0x5620b35cf390 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d24495740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620b38fd2d0 con 0x5620b35f0070 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d24495740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5620b39010e0 con 0x5620b35c71b0 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b3998120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:35502/0 (socket says 172.21.15.92:35502) 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 -- 172.21.15.92:0/975992903 learned_addr learned my addr 172.21.15.92:0/975992903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d177fe640 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3997be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 -- 172.21.15.92:0/975992903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 msgr2=0x5620b3997be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3997be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 -- 172.21.15.92:0/975992903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 msgr2=0x5620b3998120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b3998120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d157fa640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d040b6510 0x7f5d040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 -- 172.21.15.92:0/975992903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620b365b650 con 0x5620b35f0070 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d17fff640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d040b6510 0x7f5d040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.306+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39976a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d18075ad0 tx=0x7f5d0000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b44a73d836f3b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d157fa640 1 -- 172.21.15.92:0/424425561 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5d101085b0 con 0x7f5d18108db0 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d0dffb640 1 -- 172.21.15.92:0/975992903 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00004020 con 0x5620b35f0070 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d0dffb640 1 -- 172.21.15.92:0/975992903 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0000fc10 con 0x5620b35f0070 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d0dffb640 1 -- 172.21.15.92:0/975992903 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00003810 con 0x5620b35f0070 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 -- 172.21.15.92:0/975992903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 msgr2=0x5620b39976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d17fff640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d040b6510 0x7f5d040b8a20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d100cb990 tx=0x7f5d1002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39976a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d18075ad0 tx=0x7f5d0000b410 comp rx=0 tx=0).stop 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 -- 172.21.15.92:0/975992903 shutdown_connections 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b3998120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3997be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 --2- 172.21.15.92:0/975992903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39976a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.307+0000 7f5d24495740 1 -- 172.21.15.92:0/975992903 >> 172.21.15.92:0/975992903 conn(0x5620b35e4460 msgr2=0x5620b3995590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 -- 172.21.15.92:0/975992903 shutdown_connections 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 -- 172.21.15.92:0/975992903 wait complete. 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 -- start start 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 Processor -- start 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b39a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5620b375e700 con 0x5620b35cf390 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620b3898160 con 0x5620b35c71b0 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.308+0000 7f5d24495740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5620b39a7fe0 con 0x5620b35f0070 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d24495740 10 client.0 _ll_register_callbacks cb 0x5620b38b19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:42.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56518/0 (socket says 172.21.15.92:56518) 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 -- 172.21.15.92:0/854159225 learned_addr learned my addr 172.21.15.92:0/854159225 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b39a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 -- 172.21.15.92:0/854159225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 msgr2=0x5620b39a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b39a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 -- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 msgr2=0x5620b39a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 -- 172.21.15.92:0/854159225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620b38c5c90 con 0x5620b35cf390 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d177fe640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a5400 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009d40 tx=0x7f5d080039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12581484602efe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/854159225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08012860 con 0x5620b35cf390 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/854159225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5d08012a00 con 0x5620b35cf390 2024-09-21T02:09:42.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.309+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0000fc10 con 0x5620b35cf390 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/854159225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d08022070 con 0x5620b35cf390 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 10 client.16905 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5620b39b3d10 con 0x5620b35cf390 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/854159225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0000fc10 con 0x5620b35cf390 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/854159225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d0801a020 con 0x5620b35cf390 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 10 client.16905 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 10 client.16905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 20 client.16905 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b35ae9f0 0x5620b39b7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 4 client.16905 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620b390fdf0 con 0x5620b35ae9f0 2024-09-21T02:09:42.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.310+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b35ae9f0 0x5620b39b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.311+0000 7f5cf6ffd640 10 client.16905 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.311+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b35ae9f0 0x5620b39b7290 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fd8d309e8acea46 server_cookie=ee7021bf4db28692 in_seq=0 out_seq=0 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.311+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/854159225 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620b390fdf0 con 0x5620b35ae9f0 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.311+0000 7f5cf6ffd640 10 client.16905 handle_command_reply: tid=0 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.311+0000 7f5d24495740 1 client.16905 shutdown 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 msgr2=0x5620b39a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a5400 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009d40 tx=0x7f5d080039c0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 shutdown_connections 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b39a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b35ae9f0 0x5620b39b7290 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a5400 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 --2- 172.21.15.92:0/854159225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5cf6ffd640 0 client.16905 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:09:42.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 >> 172.21.15.92:0/854159225 conn(0x5620b35e4460 msgr2=0x5620b3999090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 shutdown_connections 2024-09-21T02:09:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 1 -- 172.21.15.92:0/854159225 wait complete. 2024-09-21T02:09:42.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.312+0000 7f5d24495740 20 client.16905 trim_cache size 0 max 16384 2024-09-21T02:09:42.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 Processor -- start 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 -- start start 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3995600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181ad1e0 con 0x5620b35cf390 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181aed20 con 0x5620b35f0070 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d24495740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181ad8b0 con 0x5620b35c71b0 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56530/0 (socket says 172.21.15.92:56530) 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d177fe640 1 -- 172.21.15.92:0/1265116324 learned_addr learned my addr 172.21.15.92:0/1265116324 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.325+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3995600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d177fe640 1 -- 172.21.15.92:0/1265116324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 msgr2=0x5620b3995600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d177fe640 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3995600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d177fe640 1 -- 172.21.15.92:0/1265116324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 msgr2=0x5620b39a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d177fe640 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d177fe640 1 -- 172.21.15.92:0/1265116324 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620b399a4e0 con 0x5620b35cf390 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3995600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d177fe640 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a20b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800a010 tx=0x7f5d08011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8864664acd924ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d0d7fa640 1 -- 172.21.15.92:0/1265116324 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08011a50 con 0x5620b35cf390 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d0d7fa640 1 -- 172.21.15.92:0/1265116324 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d08011bf0 con 0x5620b35cf390 2024-09-21T02:09:42.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.326+0000 7f5d0d7fa640 1 -- 172.21.15.92:0/1265116324 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0801c6a0 con 0x5620b35cf390 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 -- 172.21.15.92:0/1265116324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 msgr2=0x5620b39a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a20b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800a010 tx=0x7f5d08011730 comp rx=0 tx=0).stop 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 -- 172.21.15.92:0/1265116324 shutdown_connections 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35f0070 0x5620b39a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39a20b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 --2- 172.21.15.92:0/1265116324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35c71b0 0x5620b3995600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 -- 172.21.15.92:0/1265116324 >> 172.21.15.92:0/1265116324 conn(0x5620b35e4460 msgr2=0x5620b3994ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 -- 172.21.15.92:0/1265116324 shutdown_connections 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 -- 172.21.15.92:0/1265116324 wait complete. 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.327+0000 7f5d24495740 1 -- start start 2024-09-21T02:09:42.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 Processor -- start 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b3a1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181ae650 con 0x5620b35cf390 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181adf80 con 0x5620b35c71b0 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181ab1c0 con 0x5620b35f0070 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d24495740 10 client.0 _ll_register_callbacks cb 0x7f5d08004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56546/0 (socket says 172.21.15.92:56546) 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 -- 172.21.15.92:0/2263797435 learned_addr learned my addr 172.21.15.92:0/2263797435 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b3a1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 -- 172.21.15.92:0/2263797435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 msgr2=0x5620b3a1ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b3a1ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 -- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 msgr2=0x5620b39a9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 -- 172.21.15.92:0/2263797435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620b3a1b290 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b3a1ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.328+0000 7f5d177fe640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39aa140 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5d0801cec0 tx=0x7f5d08011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6334195b560a934e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/2263797435 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08012860 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/2263797435 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5d08012a00 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d08011bf0 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/2263797435 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d08041050 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 10 client.16911 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5620b3a225b0 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/2263797435 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d08011bf0 con 0x5620b35cf390 2024-09-21T02:09:42.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/2263797435 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d08038070 con 0x5620b35cf390 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 10 client.16911 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 10 client.16911 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 20 client.16911 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b3583da0 0x5620b3a24bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 4 client.16911 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.329+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620b390b8a0 con 0x5620b3583da0 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b3583da0 0x5620b3a24bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5cf6ffd640 10 client.16911 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d0ffff640 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b3583da0 0x5620b3a24bd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b20de82e95e1ad1 server_cookie=2bba3b9bf056bf3f in_seq=0 out_seq=0 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5cf6ffd640 1 -- 172.21.15.92:0/2263797435 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620b390b8a0 con 0x5620b3583da0 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5cf6ffd640 10 client.16911 handle_command_reply: tid=0 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d24495740 1 client.16911 shutdown 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 msgr2=0x5620b39aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d24495740 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39aa140 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5d0801cec0 tx=0x7f5d08011fd0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 shutdown_connections 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d24495740 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5620b35f0070 0x5620b3a1ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.330+0000 7f5d24495740 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5620b3583da0 0x5620b3a24bd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d24495740 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b35cf390 0x5620b39aa140 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d24495740 1 --2- 172.21.15.92:0/2263797435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620b35c71b0 0x5620b39a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5cf6ffd640 0 client.16911 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 >> 172.21.15.92:0/2263797435 conn(0x5620b35e4460 msgr2=0x5620b39986d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 shutdown_connections 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d24495740 1 -- 172.21.15.92:0/2263797435 wait complete. 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d24495740 20 client.16911 trim_cache size 0 max 16384 2024-09-21T02:09:42.336 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d040b6510 msgr2=0x7f5d040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d040b6510 0x7f5d040b8a20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d100cb990 tx=0x7f5d1002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 msgr2=0x7f5d1819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1819ec20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5d100098d0 tx=0x7f5d100317b0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 shutdown_connections 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d040b6510 0x7f5d040b8a20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d18108db0 0x7f5d1819ec20 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d18108460 0x7f5d18199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 --2- 172.21.15.92:0/424425561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d18103180 0x7f5d181993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:42.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.331+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 >> 172.21.15.92:0/424425561 conn(0x7f5d18077cc0 msgr2=0x7f5d18104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:42.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.332+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 shutdown_connections 2024-09-21T02:09:42.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:42.332+0000 7f5d1df21640 1 -- 172.21.15.92:0/424425561 wait complete. 2024-09-21T02:09:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:42 smithi136 ceph-mon[34754]: pgmap v502: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:09:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:09:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:09:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:09:42.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:09:42.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:42 smithi092 ceph-mon[31190]: pgmap v502: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:09:42.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:09:42.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:09:42.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:09:42.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:09:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:42 smithi099 ceph-mon[34404]: pgmap v502: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:09:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:09:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:09:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:09:42.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:09:42.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:09:43.229 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:43.689 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:189/600: open fd = 3 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:190/600: open fd = 3 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:191/600: open fd = 3 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.690 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:192/600: open fd = 3 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:193/600: open fd = 3 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:194/600: open fd = 3 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:195/600: open fd = 3 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.691 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:196/600: open fd = 3 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:197/600: open fd = 3 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:198/600: open fd = 3 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:199/600: open fd = 3 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.692 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:200/600: open fd = 3 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:201/600: open fd = 3 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:202/600: open fd = 3 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:203/600: open fd = 3 2024-09-21T02:09:43.693 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:204/600: open fd = 3 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:205/600: open fd = 3 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:206/600: open fd = 3 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.694 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:207/600: open fd = 3 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:208/600: open fd = 3 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:209/600: open fd = 3 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:210/600: open fd = 3 2024-09-21T02:09:43.695 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:211/600: open fd = 3 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:212/600: open fd = 3 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:213/600: open fd = 3 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:214/600: open fd = 3 2024-09-21T02:09:43.696 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:215/600: open fd = 3 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:216/600: open fd = 3 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:217/600: open fd = 3 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.697 INFO:tasks.workunit.client.0.smithi092.stdout:218/600: open fd = 3 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:219/600: open fd = 3 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:220/600: open fd = 3 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:221/600: open fd = 3 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.698 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:222/600: open fd = 3 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:223/600: open fd = 3 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:224/600: open fd = 3 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:225/600: open fd = 3 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.699 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:226/600: open fd = 3 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:227/600: open fd = 3 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:228/600: open fd = 3 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:229/600: open fd = 3 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:230/600: open fd = 3 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:231/600: open fd = 3 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:232/600: open fd = 3 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:233/600: open fd = 3 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:234/600: open fd = 3 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:235/600: open fd = 3 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:236/600: open fd = 3 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:237/600: open fd = 3 2024-09-21T02:09:43.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:238/600: open fd = 3 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:239/600: open fd = 3 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:240/600: open fd = 3 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:241/600: open fd = 3 2024-09-21T02:09:43.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:242/600: open fd = 3 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:243/600: open fd = 3 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:244/600: open fd = 3 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:245/600: open fd = 3 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:246/600: open fd = 3 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:247/600: open fd = 3 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:248/600: open fd = 3 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:249/600: open fd = 3 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:250/600: open fd = 3 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 -- 172.21.15.92:0/1468223722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 msgr2=0x7fbc30110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/1468223722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc30110620 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009ca0 tx=0x7fbc2802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 -- 172.21.15.92:0/1468223722 shutdown_connections 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/1468223722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc30110620 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/1468223722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc301094a0 0x7fbc301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/1468223722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc30103b60 0x7fbc30103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 -- 172.21.15.92:0/1468223722 >> 172.21.15.92:0/1468223722 conn(0x7fbc3007b850 msgr2=0x7fbc3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 -- 172.21.15.92:0/1468223722 shutdown_connections 2024-09-21T02:09:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.716+0000 7fbc35c1a640 1 -- 172.21.15.92:0/1468223722 wait complete. 2024-09-21T02:09:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.717+0000 7fbc35c1a640 1 Processor -- start 2024-09-21T02:09:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.717+0000 7fbc35c1a640 1 -- start start 2024-09-21T02:09:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.717+0000 7fbc35c1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc30103b60 0x7fbc3019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.717+0000 7fbc35c1a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc301094a0 0x7fbc3019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.717+0000 7fbc35c1a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc3019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc35c1a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30113ab0 con 0x7fbc30109df0 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc35c1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30113930 con 0x7fbc30103b60 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc35c1a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30113c30 con 0x7fbc301094a0 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc3019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc3019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59826/0 (socket says 172.21.15.92:59826) 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 -- 172.21.15.92:0/2538585661 learned_addr learned my addr 172.21.15.92:0/2538585661 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc30103b60 0x7fbc3019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc301094a0 0x7fbc3019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 -- 172.21.15.92:0/2538585661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc301094a0 msgr2=0x7fbc3019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc301094a0 0x7fbc3019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 -- 172.21.15.92:0/2538585661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc30103b60 msgr2=0x7fbc3019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc30103b60 0x7fbc3019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 -- 172.21.15.92:0/2538585661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc301a0370 con 0x7fbc30109df0 2024-09-21T02:09:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc3019fc40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005cc0 tx=0x7fbc2802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2cff9640 1 -- 172.21.15.92:0/2538585661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28003cf0 con 0x7fbc30109df0 2024-09-21T02:09:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2cff9640 1 -- 172.21.15.92:0/2538585661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc280044f0 con 0x7fbc30109df0 2024-09-21T02:09:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc301a0600 con 0x7fbc30109df0 2024-09-21T02:09:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.718+0000 7fbc2cff9640 1 -- 172.21.15.92:0/2538585661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2802fd10 con 0x7fbc30109df0 2024-09-21T02:09:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.719+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc301a6dc0 con 0x7fbc30109df0 2024-09-21T02:09:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 Processor -- start 2024-09-21T02:09:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc2cff9640 1 -- 172.21.15.92:0/2538585661 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fbc280305c0 con 0x7fbc30109df0 2024-09-21T02:09:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 -- start start 2024-09-21T02:09:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e126070 0x56038e4cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0fd1b0 0x56038e4cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56038e443ea0 con 0x56038e105390 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56038e4332d0 con 0x56038e0fd1b0 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.721+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56038e4370e0 con 0x56038e126070 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59828/0 (socket says 172.21.15.92:59828) 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2effd640 1 -- 172.21.15.92:0/722940726 learned_addr learned my addr 172.21.15.92:0/722940726 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0fd1b0 0x56038e4cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2effd640 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e126070 0x56038e4cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/722940726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e126070 msgr2=0x56038e4cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e126070 0x56038e4cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/722940726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 msgr2=0x56038e4ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/722940726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038e191650 con 0x56038e0fd1b0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2cff9640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc180b6510 0x7fbc180b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2effd640 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc180b6510 0x7fbc180b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2cff9640 1 -- 172.21.15.92:0/2538585661 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbc281088d0 con 0x7fbc30109df0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.722+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0fd1b0 0x56038e4cdbe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ab40 tx=0x7fbc2000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a025e567af6474e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.723+0000 7fbc15ffb640 1 -- 172.21.15.92:0/722940726 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20004020 con 0x56038e0fd1b0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.723+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc180b6510 0x7fbc180b8a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400afd0 tx=0x7fbc2400a640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.723+0000 7fbc15ffb640 1 -- 172.21.15.92:0/722940726 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc200041c0 con 0x56038e0fd1b0 2024-09-21T02:09:43.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.723+0000 7fbc15ffb640 1 -- 172.21.15.92:0/722940726 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200186a0 con 0x56038e0fd1b0 2024-09-21T02:09:43.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 -- 172.21.15.92:0/722940726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0fd1b0 msgr2=0x56038e4cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0fd1b0 0x56038e4cdbe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ab40 tx=0x7fbc2000f3c0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 -- 172.21.15.92:0/722940726 shutdown_connections 2024-09-21T02:09:43.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0fd1b0 0x56038e4cdbe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/722940726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e126070 0x56038e4cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 -- 172.21.15.92:0/722940726 >> 172.21.15.92:0/722940726 conn(0x56038e11a460 msgr2=0x56038e4cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 -- 172.21.15.92:0/722940726 shutdown_connections 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 -- 172.21.15.92:0/722940726 wait complete. 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 -- start start 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.724+0000 7fbc3c18e740 1 Processor -- start 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4dd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e126070 0x56038e4ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56038e294700 con 0x56038e105390 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56038e3ce160 con 0x56038e126070 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56038e4e0120 con 0x56038e0fd1b0 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc3c18e740 10 client.0 _ll_register_callbacks cb 0x56038e3e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4dd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59830/0 (socket says 172.21.15.92:59830) 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 -- 172.21.15.92:0/2951226762 learned_addr learned my addr 172.21.15.92:0/2951226762 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4dd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e126070 0x56038e4ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 -- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 msgr2=0x56038e4dd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4dd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 -- 172.21.15.92:0/2951226762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e126070 msgr2=0x56038e4ddb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e126070 0x56038e4ddb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 -- 172.21.15.92:0/2951226762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038e3fbc90 con 0x56038e105390 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e126070 0x56038e4ddb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4dd5e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400fe80 tx=0x7fbc24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165525da06fbd793 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.725+0000 7fbc0effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc0effd640 1 -- 172.21.15.92:0/2951226762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24022430 con 0x56038e105390 2024-09-21T02:09:43.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc200041c0 con 0x56038e105390 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc0effd640 1 -- 172.21.15.92:0/2951226762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbc240225d0 con 0x56038e105390 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc0effd640 1 -- 172.21.15.92:0/2951226762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc240227d0 con 0x56038e105390 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 10 client.16923 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56038e4ebdd0 con 0x56038e105390 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc0effd640 1 -- 172.21.15.92:0/2951226762 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc200041c0 con 0x56038e105390 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc0effd640 1 -- 172.21.15.92:0/2951226762 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc2401f990 con 0x56038e105390 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 10 client.16923 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 20 client.16923 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e0e49f0 0x56038e4ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 4 client.16923 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.726+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56038e445df0 con 0x56038e0e49f0 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.727+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e0e49f0 0x56038e4ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.727+0000 7fbc0effd640 10 client.16923 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.727+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e0e49f0 0x56038e4ef400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af0d73f27c52b26 server_cookie=b5e8088751729573 in_seq=0 out_seq=0 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc0effd640 1 -- 172.21.15.92:0/2951226762 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56038e445df0 con 0x56038e0e49f0 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc0effd640 10 client.16923 handle_command_reply: tid=0 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 client.16923 shutdown 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 msgr2=0x56038e4dd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4dd5e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400fe80 tx=0x7fbc24004730 comp rx=0 tx=0).stop 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 shutdown_connections 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e0e49f0 0x56038e4ef400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e126070 0x56038e4ddb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc0effd640 0 client.16923 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e105390 0x56038e4dd5e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2951226762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4dd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 >> 172.21.15.92:0/2951226762 conn(0x56038e11a460 msgr2=0x56038e4c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 shutdown_connections 2024-09-21T02:09:43.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2951226762 wait complete. 2024-09-21T02:09:43.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.728+0000 7fbc3c18e740 20 client.16923 trim_cache size 0 max 16384 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.741+0000 7fbc3c18e740 1 Processor -- start 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.741+0000 7fbc3c18e740 1 -- start start 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0e49f0 0x56038e4cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 0x56038e4d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e105390 0x56038e4d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301ad220 con 0x56038e0fd1b0 2024-09-21T02:09:43.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301aed60 con 0x56038e0e49f0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301ad8f0 con 0x56038e105390 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 0x56038e4d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 0x56038e4d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59844/0 (socket says 172.21.15.92:59844) 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 -- 172.21.15.92:0/2752922624 learned_addr learned my addr 172.21.15.92:0/2752922624 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0e49f0 0x56038e4cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e105390 0x56038e4d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 -- 172.21.15.92:0/2752922624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e105390 msgr2=0x56038e4d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e105390 0x56038e4d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 -- 172.21.15.92:0/2752922624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0e49f0 msgr2=0x56038e4cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0e49f0 0x56038e4cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 -- 172.21.15.92:0/2752922624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038e4d04d0 con 0x56038e0fd1b0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 0x56038e4d2780 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400fe50 tx=0x7fbc2400aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c47625553f5a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc157fa640 1 -- 172.21.15.92:0/2752922624 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24039070 con 0x56038e0fd1b0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc157fa640 1 -- 172.21.15.92:0/2752922624 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc24022970 con 0x56038e0fd1b0 2024-09-21T02:09:43.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.742+0000 7fbc157fa640 1 -- 172.21.15.92:0/2752922624 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2401f680 con 0x56038e0fd1b0 2024-09-21T02:09:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2752922624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 msgr2=0x56038e4d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 0x56038e4d2780 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400fe50 tx=0x7fbc2400aa40 comp rx=0 tx=0).stop 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2752922624 shutdown_connections 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e105390 0x56038e4d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0fd1b0 0x56038e4d2780 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2752922624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e0e49f0 0x56038e4cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2752922624 >> 172.21.15.92:0/2752922624 conn(0x56038e11a460 msgr2=0x56038e4caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2752922624 shutdown_connections 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.743+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2752922624 wait complete. 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 -- start start 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 Processor -- start 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 0x56038e4e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e105390 0x56038e4e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301ae690 con 0x56038e0e49f0 2024-09-21T02:09:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301adfc0 con 0x56038e105390 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301ab200 con 0x56038e0fd1b0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc3c18e740 10 client.0 _ll_register_callbacks cb 0x7fbc2400f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc2f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 0x56038e4e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc2f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 0x56038e4e5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59854/0 (socket says 172.21.15.92:59854) 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/2091343641 learned_addr learned my addr 172.21.15.92:0/2091343641 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 msgr2=0x56038e4e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.744+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4e12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/2091343641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e105390 msgr2=0x56038e4e19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc2ffff640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e105390 0x56038e4e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e105390 0x56038e4e19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc2f7fe640 1 -- 172.21.15.92:0/2091343641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038e558dc0 con 0x56038e0e49f0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc2f7fe640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 0x56038e4e5dc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ab40 tx=0x7fbc2000a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15ed9f38ef60223 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:43.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc0effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc0effd640 1 -- 172.21.15.92:0/2091343641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20030060 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc0effd640 1 -- 172.21.15.92:0/2091343641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbc200042b0 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc24022970 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc0effd640 1 -- 172.21.15.92:0/2091343641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc20005040 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc3c18e740 10 client.16929 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56038e559ee0 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc0effd640 1 -- 172.21.15.92:0/2091343641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc24022970 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc0effd640 1 -- 172.21.15.92:0/2091343641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc20003ae0 con 0x56038e0e49f0 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc3c18e740 10 client.16929 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc3c18e740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.745+0000 7fbc3c18e740 20 client.16929 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e126070 0x56038e55c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:43.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc3c18e740 4 client.16929 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56038e4418a0 con 0x56038e126070 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e126070 0x56038e55c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc2effd640 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e126070 0x56038e55c440 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecdfbf7477a9ba49 server_cookie=c98860930ada82c in_seq=0 out_seq=0 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc0effd640 10 client.16929 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc0effd640 1 -- 172.21.15.92:0/2091343641 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56038e4418a0 con 0x56038e126070 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc0effd640 10 client.16929 handle_command_reply: tid=0 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.746+0000 7fbc3c18e740 1 client.16929 shutdown 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 msgr2=0x56038e4e5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 0x56038e4e5dc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ab40 tx=0x7fbc2000a700 comp rx=0 tx=0).stop 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 shutdown_connections 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56038e126070 0x56038e55c440 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56038e105390 0x56038e4e19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56038e0fd1b0 0x56038e4e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 --2- 172.21.15.92:0/2091343641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56038e0e49f0 0x56038e4e5dc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc0effd640 0 client.16929 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 >> 172.21.15.92:0/2091343641 conn(0x56038e11a460 msgr2=0x56038e4ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 shutdown_connections 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 1 -- 172.21.15.92:0/2091343641 wait complete. 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc3c18e740 20 client.16929 trim_cache size 0 max 16384 2024-09-21T02:09:43.753 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc180b6510 msgr2=0x7fbc180b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc180b6510 0x7fbc180b8a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400afd0 tx=0x7fbc2400a640 comp rx=0 tx=0).stop 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 msgr2=0x7fbc3019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc3019fc40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005cc0 tx=0x7fbc2802f470 comp rx=0 tx=0).stop 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 shutdown_connections 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc180b6510 0x7fbc180b8a20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc30109df0 0x7fbc3019fc40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc301094a0 0x7fbc3019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 --2- 172.21.15.92:0/2538585661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc30103b60 0x7fbc3019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:43.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.747+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 >> 172.21.15.92:0/2538585661 conn(0x7fbc3007b850 msgr2=0x7fbc30107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:43.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.748+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 shutdown_connections 2024-09-21T02:09:43.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:43.748+0000 7fbc35c1a640 1 -- 172.21.15.92:0/2538585661 wait complete. 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 -- 172.21.15.92:0/4199318056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103fe0 msgr2=0x7f109410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 --2- 172.21.15.92:0/4199318056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103fe0 0x7f109410a980 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f108c009ca0 tx=0x7f108c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 -- 172.21.15.92:0/4199318056 shutdown_connections 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 --2- 172.21.15.92:0/4199318056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103fe0 0x7f109410a980 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 --2- 172.21.15.92:0/4199318056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1094103640 0x7f1094103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 --2- 172.21.15.92:0/4199318056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10941025c0 0x7f1094102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 -- 172.21.15.92:0/4199318056 >> 172.21.15.92:0/4199318056 conn(0x7f10940fdf00 msgr2=0x7f10941003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 -- 172.21.15.92:0/4199318056 shutdown_connections 2024-09-21T02:09:44.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 -- 172.21.15.92:0/4199318056 wait complete. 2024-09-21T02:09:44.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.361+0000 7f10997e6640 1 Processor -- start 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 -- start start 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10941025c0 0x7f1094197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 0x7f10941983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1094103fe0 0x7f1094198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f109410d140 con 0x7f1094103640 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f109410cfc0 con 0x7f10941025c0 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10997e6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f109410d2c0 con 0x7f1094103fe0 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 0x7f10941983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 0x7f10941983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59874/0 (socket says 172.21.15.92:59874) 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 -- 172.21.15.92:0/889194239 learned_addr learned my addr 172.21.15.92:0/889194239 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f1092ffd640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10941025c0 0x7f1094197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f10937fe640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1094103fe0 0x7f1094198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 -- 172.21.15.92:0/889194239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1094103fe0 msgr2=0x7f1094198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1094103fe0 0x7f1094198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 -- 172.21.15.92:0/889194239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10941025c0 msgr2=0x7f1094197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10941025c0 0x7f1094197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 -- 172.21.15.92:0/889194239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1094199770 con 0x7f1094103640 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.362+0000 7f108bfff640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 0x7f10941983a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f107c010ad0 tx=0x7f107c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.363+0000 7f1090ff9640 1 -- 172.21.15.92:0/889194239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c01a020 con 0x7f1094103640 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.363+0000 7f1090ff9640 1 -- 172.21.15.92:0/889194239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f107c00ec30 con 0x7f1094103640 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.363+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1094199a60 con 0x7f1094103640 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.363+0000 7f1090ff9640 1 -- 172.21.15.92:0/889194239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c015640 con 0x7f1094103640 2024-09-21T02:09:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.363+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10941a0150 con 0x7f1094103640 2024-09-21T02:09:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.365+0000 7f10a082e740 1 Processor -- start 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.365+0000 7f10a082e740 1 -- start start 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.365+0000 7f10a082e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 0x55d5e3124e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.365+0000 7f10a082e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e31254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10a082e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3126010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10a082e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d5e30976f0 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10a082e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3086b20 con 0x55d5e2d542f0 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10a082e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d5e308a930 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 0x55d5e3124e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 0x55d5e3124e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59886/0 (socket says 172.21.15.92:59886) 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 -- 172.21.15.92:0/1782776320 learned_addr learned my addr 172.21.15.92:0/1782776320 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f1090ff9640 1 -- 172.21.15.92:0/889194239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f107c01d020 con 0x7f1094103640 2024-09-21T02:09:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f1092ffd640 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e31254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 -- 172.21.15.92:0/1782776320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d5c4d0 msgr2=0x55d5e3126010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3126010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 -- 172.21.15.92:0/1782776320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 msgr2=0x55d5e31254c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e31254c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 -- 172.21.15.92:0/1782776320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e304c530 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10937fe640 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 0x55d5e3124e50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f108c02f390 tx=0x7f108c02f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b472d10cea3178c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10897fa640 1 -- 172.21.15.92:0/1782776320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c031650 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10897fa640 1 -- 172.21.15.92:0/1782776320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c036040 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.366+0000 7f10897fa640 1 -- 172.21.15.92:0/1782776320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c004680 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f1090ff9640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f10800b6590 0x7f10800b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f1092ffd640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f10800b6590 0x7f10800b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f1090ff9640 1 -- 172.21.15.92:0/889194239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f107c0e9830 con 0x7f1094103640 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f10a082e740 1 -- 172.21.15.92:0/1782776320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 msgr2=0x55d5e3124e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f1092ffd640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f10800b6590 0x7f10800b8aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1084009cf0 tx=0x7f1084005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f10a082e740 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 0x55d5e3124e50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f108c02f390 tx=0x7f108c02f3c0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f10a082e740 1 -- 172.21.15.92:0/1782776320 shutdown_connections 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.367+0000 7f10a082e740 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3126010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e31254c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 --2- 172.21.15.92:0/1782776320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d7d1b0 0x55d5e3124e50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- 172.21.15.92:0/1782776320 >> 172.21.15.92:0/1782776320 conn(0x55d5e2d71300 msgr2=0x55d5e31208e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- 172.21.15.92:0/1782776320 shutdown_connections 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- 172.21.15.92:0/1782776320 wait complete. 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- start start 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 Processor -- start 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e312ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d7d1b0 0x55d5e312f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d5e2fbeab0 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5e302c730 con 0x55d5e2d542f0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3126ad0 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f10a082e740 10 client.0 _ll_register_callbacks cb 0x55d5e30806b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f108bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e312ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f108bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e312ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59888/0 (socket says 172.21.15.92:59888) 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f108bfff640 1 -- 172.21.15.92:0/304177599 learned_addr learned my addr 172.21.15.92:0/304177599 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.368+0000 7f1092ffd640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f108bfff640 1 -- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d7d1b0 msgr2=0x55d5e312f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f10937fe640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d7d1b0 0x55d5e312f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f108bfff640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d7d1b0 0x55d5e312f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f108bfff640 1 -- 172.21.15.92:0/304177599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 msgr2=0x55d5e312e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f108bfff640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f108bfff640 1 -- 172.21.15.92:0/304177599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e2de8790 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f108bfff640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e312ecc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f107c0e7360 tx=0x7f107c0aefd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ccaa9e2a7a8dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f107a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f107a7fc640 1 -- 172.21.15.92:0/304177599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c0af110 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f107a7fc640 1 -- 172.21.15.92:0/304177599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f107c0ae880 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108c036040 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f107a7fc640 1 -- 172.21.15.92:0/304177599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f107c0ab110 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f10a082e740 10 client.16941 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5e2e61620 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f107a7fc640 1 -- 172.21.15.92:0/304177599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c036040 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f107a7fc640 1 -- 172.21.15.92:0/304177599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f107c0e9ce0 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.369+0000 7f10a082e740 10 client.16941 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f10a082e740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f10a082e740 20 client.16941 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f10a082e740 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d3bd80 0x55d5e313bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f10a082e740 4 client.16941 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e3099640 con 0x55d5e2d3bd80 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f1092ffd640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d3bd80 0x55d5e313bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f1092ffd640 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d3bd80 0x55d5e313bd00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46a9f4516561561 server_cookie=51a6f7a40afddbc2 in_seq=0 out_seq=0 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.370+0000 7f107a7fc640 10 client.16941 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f107a7fc640 1 -- 172.21.15.92:0/304177599 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5e3099640 con 0x55d5e2d3bd80 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f107a7fc640 10 client.16941 handle_command_reply: tid=0 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 client.16941 shutdown 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 msgr2=0x55d5e312ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e312ecc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f107c0e7360 tx=0x7f107c0aefd0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 shutdown_connections 2024-09-21T02:09:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d3bd80 0x55d5e313bd00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d7d1b0 0x55d5e312f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f107a7fc640 0 client.16941 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e312ecc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 --2- 172.21.15.92:0/304177599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.371+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 >> 172.21.15.92:0/304177599 conn(0x55d5e2d71300 msgr2=0x55d5e311eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.372+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 shutdown_connections 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.372+0000 7f10a082e740 1 -- 172.21.15.92:0/304177599 wait complete. 2024-09-21T02:09:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.372+0000 7f10a082e740 20 client.16941 trim_cache size 0 max 16384 2024-09-21T02:09:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 Processor -- start 2024-09-21T02:09:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 -- start start 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d3bd80 0x55d5e312cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3136710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10941a8d80 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f10941aa8c0 con 0x55d5e2d542f0 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10a082e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10941a9450 con 0x55d5e2d3bd80 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3136710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3136710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59894/0 (socket says 172.21.15.92:59894) 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f10937fe640 1 -- 172.21.15.92:0/2401814674 learned_addr learned my addr 172.21.15.92:0/2401814674 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f108bfff640 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.387+0000 7f1092ffd640 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d3bd80 0x55d5e312cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f10937fe640 1 -- 172.21.15.92:0/2401814674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d3bd80 msgr2=0x55d5e312cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f10937fe640 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d3bd80 0x55d5e312cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f10937fe640 1 -- 172.21.15.92:0/2401814674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 msgr2=0x55d5e312d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f10937fe640 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f10937fe640 1 -- 172.21.15.92:0/2401814674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e3139a90 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f108bfff640 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f10937fe640 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3136710 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f108c009ca0 tx=0x7f108c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5e247a6be2f043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f1088ff9640 1 -- 172.21.15.92:0/2401814674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c052070 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f1088ff9640 1 -- 172.21.15.92:0/2401814674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c036070 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.388+0000 7f1088ff9640 1 -- 172.21.15.92:0/2401814674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c004570 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 -- 172.21.15.92:0/2401814674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 msgr2=0x55d5e3136710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3136710 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f108c009ca0 tx=0x7f108c005fd0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 -- 172.21.15.92:0/2401814674 shutdown_connections 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3136710 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d542f0 0x55d5e312d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 --2- 172.21.15.92:0/2401814674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d3bd80 0x55d5e312cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 -- 172.21.15.92:0/2401814674 >> 172.21.15.92:0/2401814674 conn(0x55d5e2d71300 msgr2=0x55d5e312d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 -- 172.21.15.92:0/2401814674 shutdown_connections 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 -- 172.21.15.92:0/2401814674 wait complete. 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 -- start start 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.389+0000 7f10a082e740 1 Processor -- start 2024-09-21T02:09:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 0x55d5e31327a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d542f0 0x55d5e3132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3133910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10941aa1f0 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f10941a9b20 con 0x55d5e2d3bd80 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10941a6d60 con 0x55d5e2d542f0 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10a082e740 10 client.0 _ll_register_callbacks cb 0x7f107c016c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f1092ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 0x55d5e31327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3133910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59896/0 (socket says 172.21.15.92:59896) 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f1092ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 0x55d5e31327a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:44438/0 (socket says 172.21.15.92:44438) 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 -- 172.21.15.92:0/1097028158 learned_addr learned my addr 172.21.15.92:0/1097028158 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f108bfff640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d542f0 0x55d5e3132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 -- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d542f0 msgr2=0x55d5e3132ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d542f0 0x55d5e3132ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 -- 172.21.15.92:0/1097028158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 msgr2=0x55d5e31327a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 0x55d5e31327a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 -- 172.21.15.92:0/1097028158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e3133e80 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f1092ffd640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 0x55d5e31327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f10937fe640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3133910 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f108c009a60 tx=0x7f108c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b72e4105a0f36a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f107a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.390+0000 7f107a7fc640 1 -- 172.21.15.92:0/1097028158 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c05c020 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f107a7fc640 1 -- 172.21.15.92:0/1097028158 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f108c0315d0 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108c036070 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f107a7fc640 1 -- 172.21.15.92:0/1097028158 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f108c052040 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 10 client.16947 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5e31a6ef0 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f107a7fc640 1 -- 172.21.15.92:0/1097028158 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c036070 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f107a7fc640 1 -- 172.21.15.92:0/1097028158 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f108c055030 con 0x55d5e2d5c4d0 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 10 client.16947 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 20 client.16947 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d7d1b0 0x55d5e31a9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 4 client.16947 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5e30950f0 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.391+0000 7f1092ffd640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d7d1b0 0x55d5e31a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.392+0000 7f107a7fc640 10 client.16947 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.392+0000 7f1092ffd640 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d7d1b0 0x55d5e31a9550 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1188607ddacd771 server_cookie=aa47d8667c83bcfc in_seq=0 out_seq=0 2024-09-21T02:09:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f107a7fc640 1 -- 172.21.15.92:0/1097028158 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5e30950f0 con 0x55d5e2d7d1b0 2024-09-21T02:09:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f107a7fc640 10 client.16947 handle_command_reply: tid=0 2024-09-21T02:09:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 client.16947 shutdown 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 msgr2=0x55d5e3133910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3133910 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f108c009a60 tx=0x7f108c031af0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 shutdown_connections 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5e2d7d1b0 0x55d5e31a9550 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5e2d5c4d0 0x55d5e3133910 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5e2d542f0 0x55d5e3132ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 --2- 172.21.15.92:0/1097028158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e2d3bd80 0x55d5e31327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f107a7fc640 0 client.16947 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 >> 172.21.15.92:0/1097028158 conn(0x55d5e2d71300 msgr2=0x55d5e312d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.407+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 shutdown_connections 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10a082e740 1 -- 172.21.15.92:0/1097028158 wait complete. 2024-09-21T02:09:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10a082e740 20 client.16947 trim_cache size 0 max 16384 2024-09-21T02:09:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:09:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:09:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:09:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f10800b6590 msgr2=0x7f10800b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f10800b6590 0x7f10800b8aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1084009cf0 tx=0x7f1084005f90 comp rx=0 tx=0).stop 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 msgr2=0x7f10941983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 0x7f10941983a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f107c010ad0 tx=0x7f107c010f90 comp rx=0 tx=0).stop 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 shutdown_connections 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f10800b6590 0x7f10800b8aa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1094103fe0 0x7f1094198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1094103640 0x7f10941983a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 --2- 172.21.15.92:0/889194239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10941025c0 0x7f1094197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 >> 172.21.15.92:0/889194239 conn(0x7f10940fdf00 msgr2=0x7f1094106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 shutdown_connections 2024-09-21T02:09:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:44.408+0000 7f10997e6640 1 -- 172.21.15.92:0/889194239 wait complete. 2024-09-21T02:09:44.489 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:44 smithi092 ceph-mon[31190]: pgmap v503: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:44.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:44 smithi136 ceph-mon[34754]: pgmap v503: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:44.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:44 smithi099 ceph-mon[34404]: pgmap v503: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:45.516 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:09:45.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:09:45.782 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 -- 172.21.15.92:0/3582650256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105170 msgr2=0x7f9618105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 --2- 172.21.15.92:0/3582650256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105170 0x7f9618105580 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f960402a410 comp rx=0 tx=0).stop 2024-09-21T02:09:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 -- 172.21.15.92:0/3582650256 shutdown_connections 2024-09-21T02:09:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 --2- 172.21.15.92:0/3582650256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9618105ac0 0x7f961810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 --2- 172.21.15.92:0/3582650256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105170 0x7f9618105580 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 --2- 172.21.15.92:0/3582650256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618103160 0x7f9618103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 -- 172.21.15.92:0/3582650256 >> 172.21.15.92:0/3582650256 conn(0x7f9618077c60 msgr2=0x7f96180780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 -- 172.21.15.92:0/3582650256 shutdown_connections 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.263+0000 7f961f01d640 1 -- 172.21.15.92:0/3582650256 wait complete. 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 Processor -- start 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 -- start start 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9618103160 0x7f961819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618105170 0x7f961819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 0x7f961819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96181129c0 con 0x7f9618105ac0 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9618112840 con 0x7f9618103160 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961f01d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9618112b40 con 0x7f9618105170 2024-09-21T02:09:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961d5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 0x7f961819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961d5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 0x7f961819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59914/0 (socket says 172.21.15.92:59914) 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961d5ba640 1 -- 172.21.15.92:0/4095923739 learned_addr learned my addr 172.21.15.92:0/4095923739 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f961cdb9640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9618103160 0x7f961819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.264+0000 7f9617fff640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618105170 0x7f961819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961d5ba640 1 -- 172.21.15.92:0/4095923739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618105170 msgr2=0x7f961819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961d5ba640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618105170 0x7f961819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961d5ba640 1 -- 172.21.15.92:0/4095923739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9618103160 msgr2=0x7f961819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961d5ba640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9618103160 0x7f961819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961d5ba640 1 -- 172.21.15.92:0/4095923739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96181a6660 con 0x7f9618105ac0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961d5ba640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 0x7f961819bdd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f961000cad0 tx=0x7f961000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f9615ffb640 1 -- 172.21.15.92:0/4095923739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961000ae20 con 0x7f9618105ac0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f9615ffb640 1 -- 172.21.15.92:0/4095923739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9610005020 con 0x7f9618105ac0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96181a6950 con 0x7f9618105ac0 2024-09-21T02:09:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f9615ffb640 1 -- 172.21.15.92:0/4095923739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961000f6b0 con 0x7f9618105ac0 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.265+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181a6e10 con 0x7f9618105ac0 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9615ffb640 1 -- 172.21.15.92:0/4095923739 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f96100051d0 con 0x7f9618105ac0 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 Processor -- start 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 -- start start 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4a9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4a9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571c4a14e60 con 0x5571c46f7350 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5571c4a04290 con 0x5571c46ce490 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9625591740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5571c4a080a0 con 0x5571c46d6670 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9617fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4a9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9617fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4a9e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59930/0 (socket says 172.21.15.92:59930) 2024-09-21T02:09:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.268+0000 7f9617fff640 1 -- 172.21.15.92:0/1321107328 learned_addr learned my addr 172.21.15.92:0/1321107328 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f961cdb9640 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4a9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 -- 172.21.15.92:0/1321107328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 msgr2=0x5571c4a9eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 -- 172.21.15.92:0/1321107328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 msgr2=0x5571c4a9e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4a9e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 -- 172.21.15.92:0/1321107328 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c4762930 con 0x5571c46f7350 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9615ffb640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96080b6510 0x7f96080b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f961cdb9640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96080b6510 0x7f96080b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9615ffb640 1 -- 172.21.15.92:0/4095923739 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f96100e7230 con 0x7f9618105ac0 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9617fff640 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4a9e430 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f9604000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453e866582da1777 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.269+0000 7f9602ffd640 1 -- 172.21.15.92:0/1321107328 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604044030 con 0x5571c46f7350 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9602ffd640 1 -- 172.21.15.92:0/1321107328 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9604031670 con 0x5571c46f7350 2024-09-21T02:09:46.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9602ffd640 1 -- 172.21.15.92:0/1321107328 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604035040 con 0x5571c46f7350 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 -- 172.21.15.92:0/1321107328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 msgr2=0x5571c4a9e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4a9e430 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f9604000bc0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f961cdb9640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96080b6510 0x7f96080b8a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5571c4a9d0b0 tx=0x7f960c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 -- 172.21.15.92:0/1321107328 shutdown_connections 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4a9e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 --2- 172.21.15.92:0/1321107328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4a9e430 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.270+0000 7f9625591740 1 -- 172.21.15.92:0/1321107328 >> 172.21.15.92:0/1321107328 conn(0x5571c46eb740 msgr2=0x5571c4a9c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 -- 172.21.15.92:0/1321107328 shutdown_connections 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 -- 172.21.15.92:0/1321107328 wait complete. 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 -- start start 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 Processor -- start 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4aae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46d6670 0x5571c4aae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.271+0000 7f9625591740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571c4865890 con 0x5571c46f7350 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9625591740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5571c499ed40 con 0x5571c46d6670 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9625591740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5571c4ab0f10 con 0x5571c46ce490 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9625591740 10 client.0 _ll_register_callbacks cb 0x5571c4a02c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59938/0 (socket says 172.21.15.92:59938) 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 -- 172.21.15.92:0/3637896382 learned_addr learned my addr 172.21.15.92:0/3637896382 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961cdb9640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4aae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9617fff640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46d6670 0x5571c4aae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 -- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 msgr2=0x5571c4aae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4aae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 -- 172.21.15.92:0/3637896382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46d6670 msgr2=0x5571c4aae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46d6670 0x5571c4aae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 -- 172.21.15.92:0/3637896382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c4aba8b0 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f961d5ba640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aba2c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f96100aa6c0 tx=0x7f96100acf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fb4409ac74df4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f95f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f95f7fff640 1 -- 172.21.15.92:0/3637896382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001d040 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f95f7fff640 1 -- 172.21.15.92:0/3637896382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f96100ac8c0 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9604031670 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f95f7fff640 1 -- 172.21.15.92:0/3637896382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96100b6dc0 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9625591740 10 client.16959 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.272+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5571c4abd1c0 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f95f7fff640 1 -- 172.21.15.92:0/3637896382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9604031670 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f95f7fff640 1 -- 172.21.15.92:0/3637896382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9610058ce0 con 0x5571c46f7350 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f9625591740 10 client.16959 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f9625591740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:46.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f9625591740 20 client.16959 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f9625591740 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c46b5cd0 0x5571c4ac0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f9625591740 4 client.16959 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c4a16db0 con 0x5571c46b5cd0 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.273+0000 7f961cdb9640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c46b5cd0 0x5571c4ac0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.274+0000 7f95f7fff640 10 client.16959 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.274+0000 7f961cdb9640 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c46b5cd0 0x5571c4ac0680 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5270d81f952fda server_cookie=254ad62d933bdb34 in_seq=0 out_seq=0 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f95f7fff640 1 -- 172.21.15.92:0/3637896382 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571c4a16db0 con 0x5571c46b5cd0 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f95f7fff640 10 client.16959 handle_command_reply: tid=0 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 client.16959 shutdown 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 msgr2=0x5571c4aba2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aba2c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f96100aa6c0 tx=0x7f96100acf60 comp rx=0 tx=0).stop 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 shutdown_connections 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c46b5cd0 0x5571c4ac0680 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aba2c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46d6670 0x5571c4aae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 --2- 172.21.15.92:0/3637896382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4aae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 >> 172.21.15.92:0/3637896382 conn(0x5571c46eb740 msgr2=0x5571c4a9fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f95f7fff640 0 client.16959 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 shutdown_connections 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 1 -- 172.21.15.92:0/3637896382 wait complete. 2024-09-21T02:09:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.275+0000 7f9625591740 20 client.16959 trim_cache size 0 max 16384 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 Processor -- start 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 -- start start 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4aadac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96181ad260 con 0x5571c46f7350 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96181aeda0 con 0x5571c46ce490 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9625591740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f96181ad930 con 0x5571c46d6670 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f961d5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f961d5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59950/0 (socket says 172.21.15.92:59950) 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f961d5ba640 1 -- 172.21.15.92:0/595287742 learned_addr learned my addr 172.21.15.92:0/595287742 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f961cdb9640 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4aadac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.293+0000 7f9617fff640 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961d5ba640 1 -- 172.21.15.92:0/595287742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 msgr2=0x5571c4a9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961d5ba640 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961d5ba640 1 -- 172.21.15.92:0/595287742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 msgr2=0x5571c4aadac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961d5ba640 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4aadac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961d5ba640 1 -- 172.21.15.92:0/595287742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c4aac730 con 0x5571c46f7350 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961cdb9640 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4aadac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f961d5ba640 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aab420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f96100e4d60 tx=0x7f9610005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8c88ed3eb491ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f96027fc640 1 -- 172.21.15.92:0/595287742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100c8070 con 0x5571c46f7350 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f96027fc640 1 -- 172.21.15.92:0/595287742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961005c070 con 0x5571c46f7350 2024-09-21T02:09:46.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.294+0000 7f96027fc640 1 -- 172.21.15.92:0/595287742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001d070 con 0x5571c46f7350 2024-09-21T02:09:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 -- 172.21.15.92:0/595287742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 msgr2=0x5571c4aab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aab420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f96100e4d60 tx=0x7f9610005890 comp rx=0 tx=0).stop 2024-09-21T02:09:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 -- 172.21.15.92:0/595287742 shutdown_connections 2024-09-21T02:09:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46f7350 0x5571c4aab420 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46d6670 0x5571c4a9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 --2- 172.21.15.92:0/595287742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46ce490 0x5571c4aadac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 -- 172.21.15.92:0/595287742 >> 172.21.15.92:0/595287742 conn(0x5571c46eb740 msgr2=0x5571c4a9bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 -- 172.21.15.92:0/595287742 shutdown_connections 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 -- 172.21.15.92:0/595287742 wait complete. 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.295+0000 7f9625591740 1 -- start start 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 Processor -- start 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4ab2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 0x5571c4ab30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46f7350 0x5571c4b240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96181ae6d0 con 0x5571c46d6670 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96181ae000 con 0x5571c46f7350 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f96181ab240 con 0x5571c46ce490 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9625591740 10 client.0 _ll_register_callbacks cb 0x7f96100a9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f961cdb9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4ab2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9617fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 0x5571c4ab30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f961cdb9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4ab2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:52832/0 (socket says 172.21.15.92:52832) 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f9617fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 0x5571c4ab30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59952/0 (socket says 172.21.15.92:59952) 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f961cdb9640 1 -- 172.21.15.92:0/1232794588 learned_addr learned my addr 172.21.15.92:0/1232794588 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.296+0000 7f961d5ba640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46f7350 0x5571c4b240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f961cdb9640 1 -- 172.21.15.92:0/1232794588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46f7350 msgr2=0x5571c4b240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f961cdb9640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46f7350 0x5571c4b240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f961cdb9640 1 -- 172.21.15.92:0/1232794588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 msgr2=0x5571c4ab30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f961cdb9640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 0x5571c4ab30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f961cdb9640 1 -- 172.21.15.92:0/1232794588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c4b24710 con 0x5571c46ce490 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f9617fff640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 0x5571c4ab30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f961cdb9640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4ab2b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5571c4ab0a80 tx=0x7f960c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a054892d9d4f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f95f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f95f7fff640 1 -- 172.21.15.92:0/1232794588 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c01ccd0 con 0x5571c46ce490 2024-09-21T02:09:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961005c070 con 0x5571c46ce490 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f95f7fff640 1 -- 172.21.15.92:0/1232794588 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f960c005020 con 0x5571c46ce490 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f95f7fff640 1 -- 172.21.15.92:0/1232794588 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f960c005220 con 0x5571c46ce490 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f9625591740 10 client.26206 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.297+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5571c4b2bae0 con 0x5571c46ce490 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f95f7fff640 1 -- 172.21.15.92:0/1232794588 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961005c070 con 0x5571c46ce490 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f95f7fff640 1 -- 172.21.15.92:0/1232794588 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f960c019040 con 0x5571c46ce490 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9625591740 10 client.26206 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9625591740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9625591740 20 client.26206 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9625591740 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c468b080 0x5571c4b2e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9625591740 4 client.26206 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571c4a12860 con 0x5571c468b080 2024-09-21T02:09:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.298+0000 7f9617fff640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c468b080 0x5571c4b2e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.299+0000 7f9617fff640 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c468b080 0x5571c4b2e320 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e06a31d53700f7 server_cookie=bc1ef672391367c in_seq=0 out_seq=0 2024-09-21T02:09:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.299+0000 7f95f7fff640 10 client.26206 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:46.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f95f7fff640 1 -- 172.21.15.92:0/1232794588 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571c4a12860 con 0x5571c468b080 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f95f7fff640 10 client.26206 handle_command_reply: tid=0 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 client.26206 shutdown 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 msgr2=0x5571c4ab2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4ab2b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5571c4ab0a80 tx=0x7f960c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 shutdown_connections 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5571c468b080 0x5571c4b2e320 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571c46f7350 0x5571c4b240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571c46d6670 0x5571c4ab30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f95f7fff640 0 client.26206 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 --2- 172.21.15.92:0/1232794588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571c46ce490 0x5571c4ab2b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.300+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 >> 172.21.15.92:0/1232794588 conn(0x5571c46eb740 msgr2=0x5571c4a9a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:46.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 shutdown_connections 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f9625591740 1 -- 172.21.15.92:0/1232794588 wait complete. 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f9625591740 20 client.26206 trim_cache size 0 max 16384 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84", 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:09:46.305 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96080b6510 msgr2=0x7f96080b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96080b6510 0x7f96080b8a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5571c4a9d0b0 tx=0x7f960c00a250 comp rx=0 tx=0).stop 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 msgr2=0x7f961819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 0x7f961819bdd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f961000cad0 tx=0x7f961000cf90 comp rx=0 tx=0).stop 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 shutdown_connections 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96080b6510 0x7f96080b8a20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9618105ac0 0x7f961819bdd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.301+0000 7f961f01d640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9618105170 0x7f961819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.302+0000 7f961f01d640 1 --2- 172.21.15.92:0/4095923739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9618103160 0x7f961819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.302+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 >> 172.21.15.92:0/4095923739 conn(0x7f9618077c60 msgr2=0x7f9618106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.302+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 shutdown_connections 2024-09-21T02:09:46.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:46.302+0000 7f961f01d640 1 -- 172.21.15.92:0/4095923739 wait complete. 2024-09-21T02:09:46.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:46 smithi136 ceph-mon[34754]: pgmap v504: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:09:46.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:46 smithi092 ceph-mon[31190]: pgmap v504: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:09:46.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:46 smithi099 ceph-mon[34404]: pgmap v504: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:09:47.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:09:47.284 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:09:47.530 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:47 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:09:47.530 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:47 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:47.530 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:47 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:09:47.530 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:47 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:47.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:47 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:09:47.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:47 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:47.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:47 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:09:47.535 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:47 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:47.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:47 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:09:47.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:47 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:47.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:47 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:09:47.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:47 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:09:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 -- 172.21.15.92:0/1918381411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 msgr2=0x7fe424103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1918381411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424103f70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe414009900 tx=0x7fe41402a410 comp rx=0 tx=0).stop 2024-09-21T02:09:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 -- 172.21.15.92:0/1918381411 shutdown_connections 2024-09-21T02:09:47.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1918381411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 0x7fe424110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1918381411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241094a0 0x7fe4241098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1918381411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424103f70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 -- 172.21.15.92:0/1918381411 >> 172.21.15.92:0/1918381411 conn(0x7fe42407b850 msgr2=0x7fe42407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 -- 172.21.15.92:0/1918381411 shutdown_connections 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 -- 172.21.15.92:0/1918381411 wait complete. 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.732+0000 7fe4291c2640 1 Processor -- start 2024-09-21T02:09:47.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 -- start start 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241094a0 0x7fe424075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 0x7fe424076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe424113ab0 con 0x7fe424103b60 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe424113930 con 0x7fe4241094a0 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe4291c2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe424113c30 con 0x7fe424109df0 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe422d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe422d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59972/0 (socket says 172.21.15.92:59972) 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe422d9d640 1 -- 172.21.15.92:0/1900096744 learned_addr learned my addr 172.21.15.92:0/1900096744 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe42359e640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 0x7fe424076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.733+0000 7fe42259c640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241094a0 0x7fe424075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe422d9d640 1 -- 172.21.15.92:0/1900096744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 msgr2=0x7fe424076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe422d9d640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 0x7fe424076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe422d9d640 1 -- 172.21.15.92:0/1900096744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241094a0 msgr2=0x7fe424075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe422d9d640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241094a0 0x7fe424075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe422d9d640 1 -- 172.21.15.92:0/1900096744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4241a8740 con 0x7fe424103b60 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe42359e640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 0x7fe424076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe422d9d640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424075710 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe414005940 tx=0x7fe4140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe40bfff640 1 -- 172.21.15.92:0/1900096744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41403f030 con 0x7fe424103b60 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241a89d0 con 0x7fe424103b60 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe40bfff640 1 -- 172.21.15.92:0/1900096744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4140044c0 con 0x7fe424103b60 2024-09-21T02:09:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe40bfff640 1 -- 172.21.15.92:0/1900096744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414035070 con 0x7fe424103b60 2024-09-21T02:09:47.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.734+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241a8f00 con 0x7fe424103b60 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 Processor -- start 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 -- start start 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe40bfff640 1 -- 172.21.15.92:0/1900096744 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe414042030 con 0x7fe424103b60 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c749070 0x55f21caf0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 0x55f21caf0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c728390 0x55f21caf10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f21ca66e50 con 0x55f21c7201b0 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f21ca56280 con 0x55f21c728390 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe422d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 0x55f21caf0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe42f736740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f21ca5a090 con 0x55f21c749070 2024-09-21T02:09:47.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe422d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 0x55f21caf0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59976/0 (socket says 172.21.15.92:59976) 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.737+0000 7fe422d9d640 1 -- 172.21.15.92:0/1139346375 learned_addr learned my addr 172.21.15.92:0/1139346375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe42259c640 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c728390 0x55f21caf10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe42359e640 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c749070 0x55f21caf0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe422d9d640 1 -- 172.21.15.92:0/1139346375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c749070 msgr2=0x55f21caf0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe422d9d640 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c749070 0x55f21caf0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe422d9d640 1 -- 172.21.15.92:0/1139346375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c728390 msgr2=0x55f21caf10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe422d9d640 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c728390 0x55f21caf10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe422d9d640 1 -- 172.21.15.92:0/1139346375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21c7b4650 con 0x55f21c7201b0 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe42259c640 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c728390 0x55f21caf10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe40bfff640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe40c0b6510 0x7fe40c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe422d9d640 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 0x55f21caf0b90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe41410dc10 tx=0x7fe41410dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f768a51d90c1a2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:47.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe42359e640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe40c0b6510 0x7fe40c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe4097fa640 1 -- 172.21.15.92:0/1139346375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414116070 con 0x55f21c7201b0 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe40bfff640 1 -- 172.21.15.92:0/1900096744 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe414107660 con 0x7fe424103b60 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe4097fa640 1 -- 172.21.15.92:0/1139346375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4140cd390 con 0x55f21c7201b0 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.738+0000 7fe4097fa640 1 -- 172.21.15.92:0/1139346375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4140c9720 con 0x55f21c7201b0 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 -- 172.21.15.92:0/1139346375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 msgr2=0x55f21caf0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 0x55f21caf0b90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe41410dc10 tx=0x7fe41410dd00 comp rx=0 tx=0).stop 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42359e640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe40c0b6510 0x7fe40c0b8a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009fd0 tx=0x7fe41c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 -- 172.21.15.92:0/1139346375 shutdown_connections 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c728390 0x55f21caf10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c7201b0 0x55f21caf0b90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 --2- 172.21.15.92:0/1139346375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c749070 0x55f21caf0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 -- 172.21.15.92:0/1139346375 >> 172.21.15.92:0/1139346375 conn(0x55f21c73d460 msgr2=0x55f21caee540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 -- 172.21.15.92:0/1139346375 shutdown_connections 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.739+0000 7fe42f736740 1 -- 172.21.15.92:0/1139346375 wait complete. 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 -- start start 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 Processor -- start 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21cafbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f21c8b7700 con 0x55f21c728390 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f21c9f1110 con 0x55f21c749070 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f21cafebf0 con 0x55f21c7201b0 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42f736740 10 client.0 _ll_register_callbacks cb 0x55f21ca0a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59980/0 (socket says 172.21.15.92:59980) 2024-09-21T02:09:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42259c640 1 -- 172.21.15.92:0/419067789 learned_addr learned my addr 172.21.15.92:0/419067789 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.740+0000 7fe42359e640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe422d9d640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21cafbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42259c640 1 -- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 msgr2=0x55f21cafbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42259c640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21cafbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42259c640 1 -- 172.21.15.92:0/419067789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 msgr2=0x55f21cb07e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42259c640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42259c640 1 -- 172.21.15.92:0/419067789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21ca1ec40 con 0x55f21c728390 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42359e640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:47.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe422d9d640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21cafbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:47.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42259c640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafc270 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe418012700 tx=0x7fe418012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecfbf2a76e3d94b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:47.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe406ff5640 1 -- 172.21.15.92:0/419067789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41800f920 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe406ff5640 1 -- 172.21.15.92:0/419067789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe41800fac0 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4140cd390 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe406ff5640 1 -- 172.21.15.92:0/419067789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe41800fcc0 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42f736740 10 client.16977 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.741+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f21cb0a890 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe406ff5640 1 -- 172.21.15.92:0/419067789 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4140cd390 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe406ff5640 1 -- 172.21.15.92:0/419067789 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe418026420 con 0x55f21c728390 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe42f736740 10 client.16977 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe42f736740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe42f736740 20 client.16977 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe42f736740 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c7079f0 0x55f21cb0dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe42f736740 4 client.16977 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:47.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f21ca68da0 con 0x55f21c7079f0 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.742+0000 7fe422d9d640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c7079f0 0x55f21cb0dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.743+0000 7fe406ff5640 10 client.16977 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.743+0000 7fe422d9d640 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c7079f0 0x55f21cb0dd80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb2913eabf3a57d server_cookie=801c4b6982c717f3 in_seq=0 out_seq=0 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.743+0000 7fe406ff5640 1 -- 172.21.15.92:0/419067789 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f21ca68da0 con 0x55f21c7079f0 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.743+0000 7fe406ff5640 10 client.16977 handle_command_reply: tid=0 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 client.16977 shutdown 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 msgr2=0x55f21cafc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafc270 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe418012700 tx=0x7fe418012bc0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 shutdown_connections 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c7079f0 0x55f21cb0dd80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafc270 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe406ff5640 0 client.16977 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 --2- 172.21.15.92:0/419067789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21cafbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 >> 172.21.15.92:0/419067789 conn(0x55f21c73d460 msgr2=0x55f21caf1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 shutdown_connections 2024-09-21T02:09:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 1 -- 172.21.15.92:0/419067789 wait complete. 2024-09-21T02:09:47.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.744+0000 7fe42f736740 20 client.16977 trim_cache size 0 max 16384 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 Processor -- start 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 -- start start 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caf2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241b16c0 con 0x55f21c728390 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241b3200 con 0x55f21c749070 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.761+0000 7fe42f736740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241b1d90 con 0x55f21c7201b0 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59988/0 (socket says 172.21.15.92:59988) 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 -- 172.21.15.92:0/2470531105 learned_addr learned my addr 172.21.15.92:0/2470531105 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe422d9d640 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caf2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42359e640 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 -- 172.21.15.92:0/2470531105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 msgr2=0x55f21caf2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caf2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 -- 172.21.15.92:0/2470531105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 msgr2=0x55f21cb07d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 -- 172.21.15.92:0/2470531105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21cafcff0 con 0x55f21c728390 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42359e640 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe42259c640 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafdc10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe41800f7c0 tx=0x7fe41800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0754968e682984 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe408ff9640 1 -- 172.21.15.92:0/2470531105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41800f920 con 0x55f21c728390 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe408ff9640 1 -- 172.21.15.92:0/2470531105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41802b020 con 0x55f21c728390 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.762+0000 7fe408ff9640 1 -- 172.21.15.92:0/2470531105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180052a0 con 0x55f21c728390 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 -- 172.21.15.92:0/2470531105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 msgr2=0x55f21cafdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafdc10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe41800f7c0 tx=0x7fe41800f7f0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 -- 172.21.15.92:0/2470531105 shutdown_connections 2024-09-21T02:09:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb07d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cafdc10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 --2- 172.21.15.92:0/2470531105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caf2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 -- 172.21.15.92:0/2470531105 >> 172.21.15.92:0/2470531105 conn(0x55f21c73d460 msgr2=0x55f21caf5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 -- 172.21.15.92:0/2470531105 shutdown_connections 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.763+0000 7fe42f736740 1 -- 172.21.15.92:0/2470531105 wait complete. 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 -- start start 2024-09-21T02:09:47.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 Processor -- start 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cb003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb00910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241b2b30 con 0x55f21c728390 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241b2460 con 0x55f21c749070 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4241af6a0 con 0x55f21c7201b0 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cb003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42f736740 10 client.0 _ll_register_callbacks cb 0x7fe418009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cb003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60002/0 (socket says 172.21.15.92:60002) 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 -- 172.21.15.92:0/2212602325 learned_addr learned my addr 172.21.15.92:0/2212602325 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe422d9d640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 -- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 msgr2=0x55f21caffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 -- 172.21.15.92:0/2212602325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 msgr2=0x55f21cb00910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42359e640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb00910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb00910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.764+0000 7fe42259c640 1 -- 172.21.15.92:0/2212602325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21cb77840 con 0x55f21c728390 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42259c640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cb003d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe418026a30 tx=0x7fe418026650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b642add0ce5d568 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe406ff5640 1 -- 172.21.15.92:0/2212602325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418026d40 con 0x55f21c728390 2024-09-21T02:09:47.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe406ff5640 1 -- 172.21.15.92:0/2212602325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe418018040 con 0x55f21c728390 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41802b020 con 0x55f21c728390 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe406ff5640 1 -- 172.21.15.92:0/2212602325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4180051e0 con 0x55f21c728390 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 10 client.16983 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f21cb78ad0 con 0x55f21c728390 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe406ff5640 1 -- 172.21.15.92:0/2212602325 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41802b020 con 0x55f21c728390 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe406ff5640 1 -- 172.21.15.92:0/2212602325 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe418010430 con 0x55f21c728390 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 10 client.16983 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 20 client.16983 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c6dcda0 0x55f21cb7b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 4 client.16983 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.765+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f21ca64850 con 0x55f21c6dcda0 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe422d9d640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c6dcda0 0x55f21cb7b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe406ff5640 10 client.16983 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe422d9d640 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c6dcda0 0x55f21cb7b1c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef8685531356ed4 server_cookie=c6613964187c1ee2 in_seq=0 out_seq=0 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe406ff5640 1 -- 172.21.15.92:0/2212602325 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55f21ca64850 con 0x55f21c6dcda0 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe406ff5640 10 client.16983 handle_command_reply: tid=0 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe42f736740 1 client.16983 shutdown 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 msgr2=0x55f21cb003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe42f736740 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cb003d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe418026a30 tx=0x7fe418026650 comp rx=0 tx=0).stop 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 shutdown_connections 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.766+0000 7fe42f736740 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f21c6dcda0 0x55f21cb7b1c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f21c749070 0x55f21cb00910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f21c728390 0x55f21cb003d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 1 --2- 172.21.15.92:0/2212602325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f21c7201b0 0x55f21caffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe406ff5640 0 client.16983 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 >> 172.21.15.92:0/2212602325 conn(0x55f21c73d460 msgr2=0x55f21cafd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 shutdown_connections 2024-09-21T02:09:47.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 1 -- 172.21.15.92:0/2212602325 wait complete. 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe42f736740 20 client.16983 trim_cache size 0 max 16384 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84", 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.774 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.775 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:09:47.776 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:09:47.779 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe40c0b6510 msgr2=0x7fe40c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe40c0b6510 0x7fe40c0b8a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009fd0 tx=0x7fe41c009500 comp rx=0 tx=0).stop 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 msgr2=0x7fe424075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424075710 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe414005940 tx=0x7fe4140040c0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 shutdown_connections 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe40c0b6510 0x7fe40c0b8a20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe424109df0 0x7fe424076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241094a0 0x7fe424075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 --2- 172.21.15.92:0/1900096744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe424103b60 0x7fe424075710 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 >> 172.21.15.92:0/1900096744 conn(0x7fe42407b850 msgr2=0x7fe424107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 shutdown_connections 2024-09-21T02:09:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:09:47.767+0000 7fe4291c2640 1 -- 172.21.15.92:0/1900096744 wait complete. 2024-09-21T02:09:48.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:48 smithi092 ceph-mon[31190]: pgmap v505: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:09:48.514 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:48 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:09:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:48 smithi136 ceph-mon[34754]: pgmap v505: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:09:48.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:48 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:09:48.541 INFO:tasks.cephfs.filesystem:scrub status for tag:07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84 - {'path': '/', 'tag': '07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84', 'options': 'recursive,force'} 2024-09-21T02:09:48.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:48 smithi099 ceph-mon[34404]: pgmap v505: 97 pgs: 97 active+clean; 3.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:09:48.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:48 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:09:50.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:50 smithi136 ceph-mon[34754]: pgmap v506: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:09:50.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:50 smithi092 ceph-mon[31190]: pgmap v506: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:09:50.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:50 smithi099 ceph-mon[34404]: pgmap v506: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 23 op/s 2024-09-21T02:09:52.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:52 smithi136 ceph-mon[34754]: pgmap v507: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:52.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:52 smithi092 ceph-mon[31190]: pgmap v507: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:52.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:52 smithi099 ceph-mon[34404]: pgmap v507: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:09:54.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:54 smithi136 ceph-mon[34754]: pgmap v508: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:09:54.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:54 smithi092 ceph-mon[31190]: pgmap v508: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:09:54.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:54 smithi099 ceph-mon[34404]: pgmap v508: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:09:56.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:56 smithi136 ceph-mon[34754]: pgmap v509: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:09:56.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:56 smithi092 ceph-mon[31190]: pgmap v509: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:09:56.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:56 smithi099 ceph-mon[34404]: pgmap v509: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:09:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 -- 172.21.15.92:0/3139360161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 msgr2=0x7f6d5410db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 --2- 172.21.15.92:0/3139360161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5410db10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02a410 comp rx=0 tx=0).stop 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 -- 172.21.15.92:0/3139360161 shutdown_connections 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 --2- 172.21.15.92:0/3139360161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5410db10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 --2- 172.21.15.92:0/3139360161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d54075c70 0x7f6d54076080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 --2- 172.21.15.92:0/3139360161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d54077070 0x7f6d54075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 -- 172.21.15.92:0/3139360161 >> 172.21.15.92:0/3139360161 conn(0x7f6d540ff020 msgr2=0x7f6d541014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 -- 172.21.15.92:0/3139360161 shutdown_connections 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.518+0000 7f6d59362640 1 -- 172.21.15.92:0/3139360161 wait complete. 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 Processor -- start 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 -- start start 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d54075c70 0x7f6d5419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d54077070 0x7f6d5419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54111510 con 0x7f6d5410b6d0 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54111390 con 0x7f6d54075c70 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d59362640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54111690 con 0x7f6d54077070 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35116/0 (socket says 172.21.15.92:35116) 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d537fe640 1 -- 172.21.15.92:0/2969117700 learned_addr learned my addr 172.21.15.92:0/2969117700 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.519+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d54077070 0x7f6d5419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d54075c70 0x7f6d5419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d537fe640 1 -- 172.21.15.92:0/2969117700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d54077070 msgr2=0x7f6d5419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d537fe640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d54077070 0x7f6d5419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d537fe640 1 -- 172.21.15.92:0/2969117700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d54075c70 msgr2=0x7f6d5419c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d537fe640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d54075c70 0x7f6d5419c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d537fe640 1 -- 172.21.15.92:0/2969117700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5419db70 con 0x7f6d5410b6d0 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d537fe640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5419d350 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009230 tx=0x7f6d4c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d4bfff640 1 -- 172.21.15.92:0/2969117700 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c03f030 con 0x7f6d5410b6d0 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5419de00 con 0x7f6d5410b6d0 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d4bfff640 1 -- 172.21.15.92:0/2969117700 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d4c003aa0 con 0x7f6d5410b6d0 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d4bfff640 1 -- 172.21.15.92:0/2969117700 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c035070 con 0x7f6d5410b6d0 2024-09-21T02:09:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.520+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d541a4550 con 0x7f6d5410b6d0 2024-09-21T02:09:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.523+0000 7f6d602cf740 1 Processor -- start 2024-09-21T02:09:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.523+0000 7f6d4bfff640 1 -- 172.21.15.92:0/2969117700 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f6d4c042030 con 0x7f6d5410b6d0 2024-09-21T02:09:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.523+0000 7f6d602cf740 1 -- start start 2024-09-21T02:09:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f736e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f7374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f738010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d602cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b57f6a96f0 con 0x55b57f38f1b0 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d602cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b57f698b20 con 0x55b57f3662f0 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d602cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b57f69c930 con 0x55b57f36e4d0 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f736e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f736e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35120/0 (socket says 172.21.15.92:35120) 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 -- 172.21.15.92:0/2818353286 learned_addr learned my addr 172.21.15.92:0/2818353286 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f7374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f738010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 -- 172.21.15.92:0/2818353286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 msgr2=0x55b57f738010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f738010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 -- 172.21.15.92:0/2818353286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 msgr2=0x55b57f7374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f7374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d527fc640 1 -- 172.21.15.92:0/2818353286 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57f65e530 con 0x55b57f38f1b0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.524+0000 7f6d4bfff640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d400b6510 0x7f6d400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.525+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f736e50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00bbe0 tx=0x7f6d3c0042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8862baf6f238c7d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.525+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d400b6510 0x7f6d400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.525+0000 7f6d4bfff640 1 -- 172.21.15.92:0/2969117700 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6d4c107840 con 0x7f6d5410b6d0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.525+0000 7f6d497fa640 1 -- 172.21.15.92:0/2818353286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c005020 con 0x55b57f38f1b0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.525+0000 7f6d497fa640 1 -- 172.21.15.92:0/2818353286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3c005670 con 0x55b57f38f1b0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.525+0000 7f6d497fa640 1 -- 172.21.15.92:0/2818353286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0106c0 con 0x55b57f38f1b0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 -- 172.21.15.92:0/2818353286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 msgr2=0x55b57f736e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f736e50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00bbe0 tx=0x7f6d3c0042b0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d400b6510 0x7f6d400b8a20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400a300 tx=0x7f6d4400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 -- 172.21.15.92:0/2818353286 shutdown_connections 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f738010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f7374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2818353286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f736e50 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 -- 172.21.15.92:0/2818353286 >> 172.21.15.92:0/2818353286 conn(0x55b57f383300 msgr2=0x55b57f7328e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 -- 172.21.15.92:0/2818353286 shutdown_connections 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 -- 172.21.15.92:0/2818353286 wait complete. 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 -- start start 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.526+0000 7f6d602cf740 1 Processor -- start 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f740560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f740c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f741790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b57f5d0ab0 con 0x55b57f38f1b0 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b57f63e730 con 0x55b57f36e4d0 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b57f738ad0 con 0x55b57f3662f0 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 10 client.0 _ll_register_callbacks cb 0x55b57f6926b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f741790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f741790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35126/0 (socket says 172.21.15.92:35126) 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 -- 172.21.15.92:0/1465911117 learned_addr learned my addr 172.21.15.92:0/1465911117 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f740560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d527fc640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f740c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 -- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 msgr2=0x55b57f740560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f740560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 -- 172.21.15.92:0/1465911117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 msgr2=0x55b57f740c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f740c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 -- 172.21.15.92:0/1465911117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57f3fa790 con 0x55b57f38f1b0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d537fe640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f741790 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cd760 tx=0x7f6d4c003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aedd3d67c27635f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/1465911117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c049070 con 0x55b57f38f1b0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/1465911117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6d4c0d7140 con 0x55b57f38f1b0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.527+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3c005670 con 0x55b57f38f1b0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/1465911117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d4c0cd1d0 con 0x55b57f38f1b0 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 10 client.16995 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b57f473620 con 0x55b57f38f1b0 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/1465911117 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3c005670 con 0x55b57f38f1b0 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/1465911117 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d4c0cece0 con 0x55b57f38f1b0 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 10 client.16995 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 20 client.16995 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f34dd80 0x55b57f74dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 4 client.16995 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b57f6ab640 con 0x55b57f34dd80 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.528+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f34dd80 0x55b57f74dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.529+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f34dd80 0x55b57f74dcb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ba89afc86249c5 server_cookie=faf0cbec41f8cb67 in_seq=0 out_seq=0 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.529+0000 7f6d3a7fc640 10 client.16995 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/1465911117 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b57f6ab640 con 0x55b57f34dd80 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d3a7fc640 10 client.16995 handle_command_reply: tid=0 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 client.16995 shutdown 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 msgr2=0x55b57f741790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f741790 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cd760 tx=0x7f6d4c003e80 comp rx=0 tx=0).stop 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 shutdown_connections 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f34dd80 0x55b57f74dcb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f38f1b0 0x55b57f741790 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d3a7fc640 0 client.16995 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f740c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 --2- 172.21.15.92:0/1465911117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f740560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 >> 172.21.15.92:0/1465911117 conn(0x55b57f383300 msgr2=0x55b57f730ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 shutdown_connections 2024-09-21T02:09:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 1 -- 172.21.15.92:0/1465911117 wait complete. 2024-09-21T02:09:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.530+0000 7f6d602cf740 20 client.16995 trim_cache size 0 max 16384 2024-09-21T02:09:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.544+0000 7f6d602cf740 1 Processor -- start 2024-09-21T02:09:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.544+0000 7f6d602cf740 1 -- start start 2024-09-21T02:09:57.546 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f732290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f736df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f737330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d602cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541ad210 con 0x55b57f34dd80 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d602cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541aed50 con 0x55b57f3662f0 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d602cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541ad8e0 con 0x55b57f36e4d0 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f732290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f732290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35136/0 (socket says 172.21.15.92:35136) 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2572995562 learned_addr learned my addr 172.21.15.92:0/2572995562 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:57.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d537fe640 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f737330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2572995562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 msgr2=0x55b57f737330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f737330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f736df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2572995562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 msgr2=0x55b57f736df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f736df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2572995562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57f73e770 con 0x55b57f34dd80 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d537fe640 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f737330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f732290 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6d4401b950 tx=0x7f6d44019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aff4c60d1e2c8eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d48ff9640 1 -- 172.21.15.92:0/2572995562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44004b20 con 0x55b57f34dd80 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d48ff9640 1 -- 172.21.15.92:0/2572995562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d44004cc0 con 0x55b57f34dd80 2024-09-21T02:09:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.545+0000 7f6d48ff9640 1 -- 172.21.15.92:0/2572995562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44005330 con 0x55b57f34dd80 2024-09-21T02:09:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 -- 172.21.15.92:0/2572995562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 msgr2=0x55b57f732290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f732290 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6d4401b950 tx=0x7f6d44019000 comp rx=0 tx=0).stop 2024-09-21T02:09:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 -- 172.21.15.92:0/2572995562 shutdown_connections 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f36e4d0 0x55b57f737330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f3662f0 0x55b57f736df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2572995562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f732290 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 -- 172.21.15.92:0/2572995562 >> 172.21.15.92:0/2572995562 conn(0x55b57f383300 msgr2=0x55b57f737ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 -- 172.21.15.92:0/2572995562 shutdown_connections 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.546+0000 7f6d602cf740 1 -- 172.21.15.92:0/2572995562 wait complete. 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 -- start start 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 Processor -- start 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f748150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f746e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f747350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541ae680 con 0x55b57f34dd80 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541adfb0 con 0x55b57f36e4d0 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541ab1f0 con 0x55b57f3662f0 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d602cf740 10 client.0 _ll_register_callbacks cb 0x7f6d4c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f748150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f748150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35146/0 (socket says 172.21.15.92:35146) 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2405589608 learned_addr learned my addr 172.21.15.92:0/2405589608 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d537fe640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f747350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f746e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 msgr2=0x55b57f746e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f746e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2405589608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 msgr2=0x55b57f747350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f747350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d52ffd640 1 -- 172.21.15.92:0/2405589608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57f7478c0 con 0x55b57f34dd80 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.547+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f746e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d52ffd640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f748150 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6d44030040 tx=0x7f6d4400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaef6431e97c4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/2405589608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44032050 con 0x55b57f34dd80 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/2405589608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6d4400f990 con 0x55b57f34dd80 2024-09-21T02:09:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d44004cc0 con 0x55b57f34dd80 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/2405589608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d4400fb90 con 0x55b57f34dd80 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 10 client.17001 fetch_fsmap learned FSMap version 29 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b57f746750 con 0x55b57f34dd80 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/2405589608 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d44004cc0 con 0x55b57f34dd80 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/2405589608 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d440396b0 con 0x55b57f34dd80 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 10 client.17001 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 20 client.17001 populate_metadata read hostname 'smithi092' 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f38f1b0 0x55b57f7bae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 4 client.17001 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.548+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b57f6a70f0 con 0x55b57f38f1b0 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.549+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f38f1b0 0x55b57f7bae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:09:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.549+0000 7f6d3a7fc640 10 client.17001 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.549+0000 7f6d527fc640 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f38f1b0 0x55b57f7bae30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6174b52ae4a5907 server_cookie=6be2839418d140b in_seq=0 out_seq=0 2024-09-21T02:09:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d3a7fc640 1 -- 172.21.15.92:0/2405589608 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b57f6a70f0 con 0x55b57f38f1b0 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d3a7fc640 10 client.17001 handle_command_reply: tid=0 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 client.17001 shutdown 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 msgr2=0x55b57f748150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f748150 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6d44030040 tx=0x7f6d4400f5a0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 shutdown_connections 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b57f38f1b0 0x55b57f7bae30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b57f36e4d0 0x55b57f747350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d3a7fc640 0 client.17001 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b57f3662f0 0x55b57f746e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 --2- 172.21.15.92:0/2405589608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b57f34dd80 0x55b57f748150 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 >> 172.21.15.92:0/2405589608 conn(0x55b57f383300 msgr2=0x55b57f737ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 shutdown_connections 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.563+0000 7f6d602cf740 1 -- 172.21.15.92:0/2405589608 wait complete. 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d602cf740 20 client.17001 trim_cache size 0 max 16384 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:09:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:09:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d400b6510 msgr2=0x7f6d400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d400b6510 0x7f6d400b8a20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400a300 tx=0x7f6d4400a590 comp rx=0 tx=0).stop 2024-09-21T02:09:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 msgr2=0x7f6d5419d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:09:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5419d350 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009230 tx=0x7f6d4c031a50 comp rx=0 tx=0).stop 2024-09-21T02:09:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 shutdown_connections 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d400b6510 0x7f6d400b8a20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5410b6d0 0x7f6d5419d350 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d54077070 0x7f6d5419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 --2- 172.21.15.92:0/2969117700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d54075c70 0x7f6d5419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 >> 172.21.15.92:0/2969117700 conn(0x7f6d540ff020 msgr2=0x7f6d541014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 shutdown_connections 2024-09-21T02:09:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:09:57.564+0000 7f6d59362640 1 -- 172.21.15.92:0/2969117700 wait complete. 2024-09-21T02:09:58.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:09:58 smithi136 ceph-mon[34754]: pgmap v510: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:09:58.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:09:58 smithi092 ceph-mon[31190]: pgmap v510: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:09:58.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:09:58 smithi099 ceph-mon[34404]: pgmap v510: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:00.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:00 smithi136 ceph-mon[34754]: pgmap v511: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:10:00.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:00 smithi136 ceph-mon[34754]: overall HEALTH_OK 2024-09-21T02:10:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:00 smithi092 ceph-mon[31190]: pgmap v511: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:10:00.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:00 smithi092 ceph-mon[31190]: overall HEALTH_OK 2024-09-21T02:10:00.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:00 smithi099 ceph-mon[34404]: pgmap v511: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:10:00.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:00 smithi099 ceph-mon[34404]: overall HEALTH_OK 2024-09-21T02:10:02.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:02 smithi136 ceph-mon[34754]: pgmap v512: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:02.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:02 smithi092 ceph-mon[31190]: pgmap v512: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:02.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:02 smithi099 ceph-mon[34404]: pgmap v512: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:04.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:04 smithi136 ceph-mon[34754]: pgmap v513: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:04.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:04 smithi092 ceph-mon[31190]: pgmap v513: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:04.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:04 smithi099 ceph-mon[34404]: pgmap v513: 97 pgs: 97 active+clean; 3.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:06.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:06 smithi136 ceph-mon[34754]: pgmap v514: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:10:06.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:06 smithi092 ceph-mon[31190]: pgmap v514: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:10:06.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:06 smithi099 ceph-mon[34404]: pgmap v514: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:10:08.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:08 smithi136 ceph-mon[34754]: pgmap v515: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:08.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:08 smithi092 ceph-mon[31190]: pgmap v515: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:08.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:08 smithi099 ceph-mon[34404]: pgmap v515: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:10.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:10 smithi136 ceph-mon[34754]: pgmap v516: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:10.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:10 smithi092 ceph-mon[31190]: pgmap v516: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:10.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:10 smithi099 ceph-mon[34404]: pgmap v516: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 -- 172.21.15.92:0/2759956526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 msgr2=0x7f3e24106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 --2- 172.21.15.92:0/2759956526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e24106310 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009900 tx=0x7f3e1402a410 comp rx=0 tx=0).stop 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 -- 172.21.15.92:0/2759956526 shutdown_connections 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 --2- 172.21.15.92:0/2759956526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24109410 0x7f3e2410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 --2- 172.21.15.92:0/2759956526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 0x7f3e24108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 --2- 172.21.15.92:0/2759956526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e24106310 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 -- 172.21.15.92:0/2759956526 >> 172.21.15.92:0/2759956526 conn(0x7f3e240fdf30 msgr2=0x7f3e241003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 -- 172.21.15.92:0/2759956526 shutdown_connections 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.669+0000 7f3e29d28640 1 -- 172.21.15.92:0/2759956526 wait complete. 2024-09-21T02:10:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 Processor -- start 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 -- start start 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e2419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 0x7f3e2419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24109410 0x7f3e2419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410f3c0 con 0x7f3e2406c4b0 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410f240 con 0x7f3e24109410 2024-09-21T02:10:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.670+0000 7f3e29d28640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410f540 con 0x7f3e24106850 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e2419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e2419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35474/0 (socket says 172.21.15.92:35474) 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 -- 172.21.15.92:0/1682075303 learned_addr learned my addr 172.21.15.92:0/1682075303 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 0x7f3e2419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 -- 172.21.15.92:0/1682075303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 msgr2=0x7f3e2419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 0x7f3e2419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e23fff640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24109410 0x7f3e2419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 -- 172.21.15.92:0/1682075303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24109410 msgr2=0x7f3e2419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24109410 0x7f3e2419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 -- 172.21.15.92:0/1682075303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419ba50 con 0x7f3e2406c4b0 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 0x7f3e2419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e237fe640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e2419a010 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e14004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e20ff9640 1 -- 172.21.15.92:0/1682075303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1403f030 con 0x7f3e2406c4b0 2024-09-21T02:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e20ff9640 1 -- 172.21.15.92:0/1682075303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e14003cb0 con 0x7f3e2406c4b0 2024-09-21T02:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419bce0 con 0x7f3e2406c4b0 2024-09-21T02:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e20ff9640 1 -- 172.21.15.92:0/1682075303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14035070 con 0x7f3e2406c4b0 2024-09-21T02:10:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.671+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241a2430 con 0x7f3e2406c4b0 2024-09-21T02:10:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 Processor -- start 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 -- start start 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e20ff9640 1 -- 172.21.15.92:0/1682075303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f3e14042030 con 0x7f3e2406c4b0 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1040e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f10414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1042010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b1f0fb36f0 con 0x55b1f0c991b0 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1f0fa2b20 con 0x55b1f0c784d0 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e30c94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b1f0fa6930 con 0x55b1f0c702f0 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1040e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1040e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35490/0 (socket says 172.21.15.92:35490) 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 -- 172.21.15.92:0/599922406 learned_addr learned my addr 172.21.15.92:0/599922406 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e237fe640 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f10414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 -- 172.21.15.92:0/599922406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 msgr2=0x55b1f10414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f10414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 -- 172.21.15.92:0/599922406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 msgr2=0x55b1f1042010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.674+0000 7f3e23fff640 1 -- 172.21.15.92:0/599922406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f0f68530 con 0x55b1f0c991b0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e237fe640 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f10414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e20ff9640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3e0c0b6510 0x7f3e0c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e23fff640 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1040e50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c007d90 tx=0x7f3e1c00c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb0e2646cfba32e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e237fe640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3e0c0b6510 0x7f3e0c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e09ffb640 1 -- 172.21.15.92:0/599922406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c015020 con 0x55b1f0c991b0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e09ffb640 1 -- 172.21.15.92:0/599922406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c011ae0 con 0x55b1f0c991b0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e09ffb640 1 -- 172.21.15.92:0/599922406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c0109d0 con 0x55b1f0c991b0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.675+0000 7f3e20ff9640 1 -- 172.21.15.92:0/1682075303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3e14107a60 con 0x7f3e2406c4b0 2024-09-21T02:10:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 -- 172.21.15.92:0/599922406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 msgr2=0x55b1f1040e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1040e50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c007d90 tx=0x7f3e1c00c560 comp rx=0 tx=0).stop 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e237fe640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3e0c0b6510 0x7f3e0c0b8a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3e140ca1a0 tx=0x7f3e1410c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 -- 172.21.15.92:0/599922406 shutdown_connections 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1042010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f10414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 --2- 172.21.15.92:0/599922406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1040e50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 -- 172.21.15.92:0/599922406 >> 172.21.15.92:0/599922406 conn(0x55b1f0c8d300 msgr2=0x55b1f103c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 -- 172.21.15.92:0/599922406 shutdown_connections 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 -- 172.21.15.92:0/599922406 wait complete. 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.676+0000 7f3e30c94740 1 -- start start 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 Processor -- start 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f104a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f104ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f104b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b1f0edaab0 con 0x55b1f0c702f0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1f0f48730 con 0x55b1f0c991b0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b1f1042b30 con 0x55b1f0c784d0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e30c94740 10 client.0 _ll_register_callbacks cb 0x55b1f0f9c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f104a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f104a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35506/0 (socket says 172.21.15.92:35506) 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 -- 172.21.15.92:0/802116358 learned_addr learned my addr 172.21.15.92:0/802116358 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f104b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f104ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 -- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 msgr2=0x55b1f104ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f104ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 -- 172.21.15.92:0/802116358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 msgr2=0x55b1f104b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f104b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 -- 172.21.15.92:0/802116358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f0d04790 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e23fff640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f104a5a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0072a0 tx=0x7f3e1c0071d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27e81b7d4f67210 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.677+0000 7f3e02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e02ffd640 1 -- 172.21.15.92:0/802116358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c015050 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e02ffd640 1 -- 172.21.15.92:0/802116358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3e1c010e60 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1c011ae0 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e02ffd640 1 -- 172.21.15.92:0/802116358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c02f070 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 10 client.17013 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1f0d7d620 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e02ffd640 1 -- 172.21.15.92:0/802116358 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c011ae0 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e02ffd640 1 -- 172.21.15.92:0/802116358 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e1c02b4b0 con 0x55b1f0c702f0 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 10 client.17013 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 20 client.17013 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c57d80 0x55b1f1057950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 4 client.17013 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.678+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1f0fb5640 con 0x55b1f0c57d80 2024-09-21T02:10:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.679+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c57d80 0x55b1f1057950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.679+0000 7f3e02ffd640 10 client.17013 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.679+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c57d80 0x55b1f1057950 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de20b560724843bb server_cookie=d06985dabe02e6ed in_seq=0 out_seq=0 2024-09-21T02:10:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e02ffd640 1 -- 172.21.15.92:0/802116358 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b1f0fb5640 con 0x55b1f0c57d80 2024-09-21T02:10:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e02ffd640 10 client.17013 handle_command_reply: tid=0 2024-09-21T02:10:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 client.17013 shutdown 2024-09-21T02:10:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 msgr2=0x55b1f104a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f104a5a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0072a0 tx=0x7f3e1c0071d0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 shutdown_connections 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c57d80 0x55b1f1057950 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f104b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f104ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e02ffd640 0 client.17013 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 --2- 172.21.15.92:0/802116358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f104a5a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 >> 172.21.15.92:0/802116358 conn(0x55b1f0c8d300 msgr2=0x55b1f103aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 shutdown_connections 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 1 -- 172.21.15.92:0/802116358 wait complete. 2024-09-21T02:10:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.680+0000 7f3e30c94740 20 client.17013 trim_cache size 0 max 16384 2024-09-21T02:10:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 Processor -- start 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 -- start start 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f1041500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f1044fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f10454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241ab150 con 0x55b1f0c702f0 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241acc90 con 0x55b1f0c991b0 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e30c94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241ab820 con 0x55b1f0c784d0 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f1041500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f1041500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35514/0 (socket says 172.21.15.92:35514) 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 -- 172.21.15.92:0/3464505262 learned_addr learned my addr 172.21.15.92:0/3464505262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 -- 172.21.15.92:0/3464505262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 msgr2=0x55b1f1044fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f1044fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 -- 172.21.15.92:0/3464505262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 msgr2=0x55b1f10454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f10454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.695+0000 7f3e23fff640 1 -- 172.21.15.92:0/3464505262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f103d240 con 0x55b1f0c702f0 2024-09-21T02:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f1041500 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00c5a0 tx=0x7f3e1c0109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2836cbf2aa34b825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e097fa640 1 -- 172.21.15.92:0/3464505262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c039070 con 0x55b1f0c702f0 2024-09-21T02:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e097fa640 1 -- 172.21.15.92:0/3464505262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c011ab0 con 0x55b1f0c702f0 2024-09-21T02:10:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e097fa640 1 -- 172.21.15.92:0/3464505262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01e070 con 0x55b1f0c702f0 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 -- 172.21.15.92:0/3464505262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 msgr2=0x55b1f1041500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f1041500 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00c5a0 tx=0x7f3e1c0109d0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 -- 172.21.15.92:0/3464505262 shutdown_connections 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c991b0 0x55b1f10454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c784d0 0x55b1f1044fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3464505262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c702f0 0x55b1f1041500 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 -- 172.21.15.92:0/3464505262 >> 172.21.15.92:0/3464505262 conn(0x55b1f0c8d300 msgr2=0x55b1f1040df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.696+0000 7f3e30c94740 1 -- 172.21.15.92:0/3464505262 shutdown_connections 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 -- 172.21.15.92:0/3464505262 wait complete. 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 -- start start 2024-09-21T02:10:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 Processor -- start 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f1052230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1050ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1051450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241ac5c0 con 0x55b1f0c991b0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241abef0 con 0x55b1f0c784d0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241a9130 con 0x55b1f0c702f0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e30c94740 10 client.0 _ll_register_callbacks cb 0x7f3e1c007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1051450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1051450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35518/0 (socket says 172.21.15.92:35518) 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.697+0000 7f3e23fff640 1 -- 172.21.15.92:0/3543805121 learned_addr learned my addr 172.21.15.92:0/3543805121 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1050ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f1052230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 -- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 msgr2=0x55b1f1052230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f1052230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 -- 172.21.15.92:0/3543805121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 msgr2=0x55b1f1050ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1050ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 -- 172.21.15.92:0/3543805121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f1051be0 con 0x55b1f0c991b0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1050ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e23fff640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1051450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c037040 tx=0x7f3e1c011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ffd5227006997e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e02ffd640 1 -- 172.21.15.92:0/3543805121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01e070 con 0x55b1f0c991b0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e02ffd640 1 -- 172.21.15.92:0/3543805121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3e1c02b110 con 0x55b1f0c991b0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1c011ab0 con 0x55b1f0c991b0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e02ffd640 1 -- 172.21.15.92:0/3543805121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c039040 con 0x55b1f0c991b0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e30c94740 10 client.17019 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1f10c22c0 con 0x55b1f0c991b0 2024-09-21T02:10:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.698+0000 7f3e02ffd640 1 -- 172.21.15.92:0/3543805121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c011ab0 con 0x55b1f0c991b0 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e02ffd640 1 -- 172.21.15.92:0/3543805121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e1c0349f0 con 0x55b1f0c991b0 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e30c94740 10 client.17019 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e30c94740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e30c94740 20 client.17019 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c2d1c0 0x55b1f10c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e30c94740 4 client.17019 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1f0fb10f0 con 0x55b1f0c2d1c0 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c2d1c0 0x55b1f10c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e22ffd640 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c2d1c0 0x55b1f10c4c10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ffbbc9a3071d3c server_cookie=6fa359255bd7b019 in_seq=0 out_seq=0 2024-09-21T02:10:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.699+0000 7f3e02ffd640 10 client.17019 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e02ffd640 1 -- 172.21.15.92:0/3543805121 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1f0fb10f0 con 0x55b1f0c2d1c0 2024-09-21T02:10:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e02ffd640 10 client.17019 handle_command_reply: tid=0 2024-09-21T02:10:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 client.17019 shutdown 2024-09-21T02:10:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 msgr2=0x55b1f1051450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1051450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c037040 tx=0x7f3e1c011a40 comp rx=0 tx=0).stop 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 shutdown_connections 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b1f0c2d1c0 0x55b1f10c4c10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b1f0c991b0 0x55b1f1051450 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1f0c784d0 0x55b1f1050ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 --2- 172.21.15.92:0/3543805121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b1f0c702f0 0x55b1f1052230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e02ffd640 0 client.17019 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.714+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 >> 172.21.15.92:0/3543805121 conn(0x55b1f0c8d300 msgr2=0x55b1f1044fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 shutdown_connections 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e30c94740 1 -- 172.21.15.92:0/3543805121 wait complete. 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e30c94740 20 client.17019 trim_cache size 0 max 16384 2024-09-21T02:10:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3e0c0b6510 msgr2=0x7f3e0c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3e0c0b6510 0x7f3e0c0b8a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3e140ca1a0 tx=0x7f3e1410c040 comp rx=0 tx=0).stop 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 msgr2=0x7f3e2419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e2419a010 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e14004340 comp rx=0 tx=0).stop 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 shutdown_connections 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3e0c0b6510 0x7f3e0c0b8a20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24109410 0x7f3e2419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e24106850 0x7f3e2419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 --2- 172.21.15.92:0/1682075303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e2406c4b0 0x7f3e2419a010 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 >> 172.21.15.92:0/1682075303 conn(0x7f3e240fdf30 msgr2=0x7f3e24104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 shutdown_connections 2024-09-21T02:10:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:10.715+0000 7f3e29d28640 1 -- 172.21.15.92:0/1682075303 wait complete. 2024-09-21T02:10:12.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:12 smithi136 ceph-mon[34754]: pgmap v517: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:12.555 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:12 smithi092 ceph-mon[31190]: pgmap v517: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:12.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:12 smithi099 ceph-mon[34404]: pgmap v517: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:14.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:14 smithi136 ceph-mon[34754]: pgmap v518: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:14.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:14 smithi092 ceph-mon[31190]: pgmap v518: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:14.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:14 smithi099 ceph-mon[34404]: pgmap v518: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:16 smithi136 ceph-mon[34754]: pgmap v519: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:10:16.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:16 smithi092 ceph-mon[31190]: pgmap v519: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:10:16.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:16 smithi099 ceph-mon[34404]: pgmap v519: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:10:18.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:18 smithi136 ceph-mon[34754]: pgmap v520: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:18.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:10:18.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:18 smithi092 ceph-mon[31190]: pgmap v520: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:18.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:18 smithi099 ceph-mon[34404]: pgmap v520: 97 pgs: 97 active+clean; 4.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:18.795 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 -- 172.21.15.92:0/4200200295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70581083a0 msgr2=0x7f70581087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 --2- 172.21.15.92:0/4200200295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70581083a0 0x7f70581087b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02a410 comp rx=0 tx=0).stop 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 -- 172.21.15.92:0/4200200295 shutdown_connections 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 --2- 172.21.15.92:0/4200200295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7058108cf0 0x7f705810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 --2- 172.21.15.92:0/4200200295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70581083a0 0x7f70581087b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 --2- 172.21.15.92:0/4200200295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7058102a60 0x7f7058102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 -- 172.21.15.92:0/4200200295 >> 172.21.15.92:0/4200200295 conn(0x7f7058079730 msgr2=0x7f7058079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 -- 172.21.15.92:0/4200200295 shutdown_connections 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.290+0000 7f705e3da640 1 -- 172.21.15.92:0/4200200295 wait complete. 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 Processor -- start 2024-09-21T02:10:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 -- start start 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 0x7f705819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70581083a0 0x7f705819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7058108cf0 0x7f70581a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7058112960 con 0x7f7058102a60 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.291+0000 7f705e3da640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70581127e0 con 0x7f7058108cf0 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f705e3da640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7058112ae0 con 0x7f70581083a0 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 0x7f705819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 0x7f705819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50920/0 (socket says 172.21.15.92:50920) 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 -- 172.21.15.92:0/257450533 learned_addr learned my addr 172.21.15.92:0/257450533 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f70577fe640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70581083a0 0x7f705819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f705c977640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7058108cf0 0x7f70581a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 -- 172.21.15.92:0/257450533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70581083a0 msgr2=0x7f705819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70581083a0 0x7f705819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 -- 172.21.15.92:0/257450533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7058108cf0 msgr2=0x7f70581a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7058108cf0 0x7f70581a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 -- 172.21.15.92:0/257450533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70581a2520 con 0x7f7058102a60 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f7057fff640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 0x7f705819c670 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f704800c920 tx=0x7f704800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f70557fa640 1 -- 172.21.15.92:0/257450533 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048007c70 con 0x7f7058102a60 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f70557fa640 1 -- 172.21.15.92:0/257450533 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7048007e10 con 0x7f7058102a60 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70581a2810 con 0x7f7058102a60 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f70557fa640 1 -- 172.21.15.92:0/257450533 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048018820 con 0x7f7058102a60 2024-09-21T02:10:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.292+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70581a9030 con 0x7f7058102a60 2024-09-21T02:10:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.295+0000 7f70557fa640 1 -- 172.21.15.92:0/257450533 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f70480189c0 con 0x7f7058102a60 2024-09-21T02:10:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.295+0000 7f706494e740 1 Processor -- start 2024-09-21T02:10:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.295+0000 7f706494e740 1 -- start start 2024-09-21T02:10:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.295+0000 7f706494e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d348c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.295+0000 7f706494e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30bc1b0 0x5636d348c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f706494e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 0x5636d348ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f706494e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636d3402e50 con 0x5636d30c4390 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f706494e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5636d33f2280 con 0x5636d30bc1b0 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f706494e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636d33f6090 con 0x5636d30e5070 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70557fa640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70400b65c0 0x7f70400b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 0x5636d348ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 0x5636d348ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50932/0 (socket says 172.21.15.92:50932) 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 -- 172.21.15.92:0/1655036915 learned_addr learned my addr 172.21.15.92:0/1655036915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f705c977640 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d348c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f705c977640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70400b65c0 0x7f70400b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70557fa640 1 -- 172.21.15.92:0/257450533 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f70480e7470 con 0x7f7058102a60 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f7057fff640 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30bc1b0 0x5636d348c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 -- 172.21.15.92:0/1655036915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 msgr2=0x5636d348c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d348c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 -- 172.21.15.92:0/1655036915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30bc1b0 msgr2=0x5636d348c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30bc1b0 0x5636d348c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 -- 172.21.15.92:0/1655036915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d3150650 con 0x5636d30c4390 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f705c977640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70400b65c0 0x7f70400b8ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7050005fd0 tx=0x7f70500099c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.296+0000 7f70577fe640 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 0x5636d348ccf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9158581eb8a374f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.297+0000 7f703e7fc640 1 -- 172.21.15.92:0/1655036915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c040070 con 0x5636d30c4390 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.297+0000 7f703e7fc640 1 -- 172.21.15.92:0/1655036915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c005db0 con 0x5636d30c4390 2024-09-21T02:10:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.297+0000 7f703e7fc640 1 -- 172.21.15.92:0/1655036915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c035040 con 0x5636d30c4390 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.297+0000 7f706494e740 1 -- 172.21.15.92:0/1655036915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 msgr2=0x5636d348ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.297+0000 7f706494e740 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 0x5636d348ccf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c032df0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 -- 172.21.15.92:0/1655036915 shutdown_connections 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30c4390 0x5636d348ccf0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30bc1b0 0x5636d348c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 --2- 172.21.15.92:0/1655036915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d348c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 -- 172.21.15.92:0/1655036915 >> 172.21.15.92:0/1655036915 conn(0x5636d30d9460 msgr2=0x5636d348a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 -- 172.21.15.92:0/1655036915 shutdown_connections 2024-09-21T02:10:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 -- 172.21.15.92:0/1655036915 wait complete. 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 -- start start 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.298+0000 7f706494e740 1 Processor -- start 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3497940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30c4390 0x5636d3497e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30e5070 0x5636d34a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636d3253700 con 0x5636d30bc1b0 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5636d338d110 con 0x5636d30e5070 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636d349a9f0 con 0x5636d30c4390 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3497940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f706494e740 10 client.0 _ll_register_callbacks cb 0x5636d33a6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3497940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50940/0 (socket says 172.21.15.92:50940) 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 -- 172.21.15.92:0/2081966753 learned_addr learned my addr 172.21.15.92:0/2081966753 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f70577fe640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30c4390 0x5636d3497e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f705c977640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30e5070 0x5636d34a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 -- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30c4390 msgr2=0x5636d3497e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30c4390 0x5636d3497e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 -- 172.21.15.92:0/2081966753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30e5070 msgr2=0x5636d34a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30e5070 0x5636d34a3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 -- 172.21.15.92:0/2081966753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d33bac40 con 0x5636d30bc1b0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f7057fff640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3497940 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7048000c00 tx=0x7f704800fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac3993fb3f92b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.299+0000 7f70377fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f70377fe640 1 -- 172.21.15.92:0/2081966753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048098030 con 0x5636d30bc1b0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f70377fe640 1 -- 172.21.15.92:0/2081966753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f70480a8ed0 con 0x5636d30bc1b0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c005db0 con 0x5636d30bc1b0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f70377fe640 1 -- 172.21.15.92:0/2081966753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70480ad090 con 0x5636d30bc1b0 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 10 client.17031 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636d34a67c0 con 0x5636d30bc1b0 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f70377fe640 1 -- 172.21.15.92:0/2081966753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c005db0 con 0x5636d30bc1b0 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f70377fe640 1 -- 172.21.15.92:0/2081966753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f70480b31f0 con 0x5636d30bc1b0 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 10 client.17031 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 20 client.17031 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d30a39f0 0x5636d34a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 4 client.17031 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.300+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636d3404da0 con 0x5636d30a39f0 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.301+0000 7f70577fe640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d30a39f0 0x5636d34a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.301+0000 7f70377fe640 10 client.17031 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:19.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.301+0000 7f70577fe640 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d30a39f0 0x5636d34a9c70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d612529b4551a6 server_cookie=f150280945ac38e9 in_seq=0 out_seq=0 2024-09-21T02:10:19.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f70377fe640 1 -- 172.21.15.92:0/2081966753 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636d3404da0 con 0x5636d30a39f0 2024-09-21T02:10:19.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f70377fe640 10 client.17031 handle_command_reply: tid=0 2024-09-21T02:10:19.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 client.17031 shutdown 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 msgr2=0x5636d3497940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3497940 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7048000c00 tx=0x7f704800fcc0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 shutdown_connections 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d30a39f0 0x5636d34a9c70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30e5070 0x5636d34a3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30c4390 0x5636d3497e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 --2- 172.21.15.92:0/2081966753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3497940 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 >> 172.21.15.92:0/2081966753 conn(0x5636d30d9460 msgr2=0x5636d348f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f70377fe640 0 client.17031 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 shutdown_connections 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 1 -- 172.21.15.92:0/2081966753 wait complete. 2024-09-21T02:10:19.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.302+0000 7f706494e740 20 client.17031 trim_cache size 0 max 16384 2024-09-21T02:10:19.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.319+0000 7f706494e740 1 Processor -- start 2024-09-21T02:10:19.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.319+0000 7f706494e740 1 -- start start 2024-09-21T02:10:19.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f706494e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3489710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f706494e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d3489c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f706494e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d3494b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f706494e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70581b1510 con 0x5636d30bc1b0 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f706494e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70581b3050 con 0x5636d30c4390 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f706494e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70581b1be0 con 0x5636d30e5070 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3489710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3489710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50946/0 (socket says 172.21.15.92:50946) 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 -- 172.21.15.92:0/1478975249 learned_addr learned my addr 172.21.15.92:0/1478975249 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f70577fe640 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d3489c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f705c977640 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d3494b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 -- 172.21.15.92:0/1478975249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 msgr2=0x5636d3494b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d3494b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 -- 172.21.15.92:0/1478975249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 msgr2=0x5636d3489c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d3489c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 -- 172.21.15.92:0/1478975249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d348e560 con 0x5636d30bc1b0 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f705c977640 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d3494b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f7057fff640 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3489710 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f70480ad280 tx=0x7f70480e7740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1021d5857a0b5b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f703dffb640 1 -- 172.21.15.92:0/1478975249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048098040 con 0x5636d30bc1b0 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f703dffb640 1 -- 172.21.15.92:0/1478975249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70480adc80 con 0x5636d30bc1b0 2024-09-21T02:10:19.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.320+0000 7f703dffb640 1 -- 172.21.15.92:0/1478975249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70480b68e0 con 0x5636d30bc1b0 2024-09-21T02:10:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 -- 172.21.15.92:0/1478975249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 msgr2=0x5636d3489710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3489710 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f70480ad280 tx=0x7f70480e7740 comp rx=0 tx=0).stop 2024-09-21T02:10:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 -- 172.21.15.92:0/1478975249 shutdown_connections 2024-09-21T02:10:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30e5070 0x5636d3494b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d3489c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 --2- 172.21.15.92:0/1478975249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30bc1b0 0x5636d3489710 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 -- 172.21.15.92:0/1478975249 >> 172.21.15.92:0/1478975249 conn(0x5636d30d9460 msgr2=0x5636d34976f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 -- 172.21.15.92:0/1478975249 shutdown_connections 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.321+0000 7f706494e740 1 -- 172.21.15.92:0/1478975249 wait complete. 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 -- start start 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 Processor -- start 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30bc1b0 0x5636d349bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d349c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 0x5636d349c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70581b2980 con 0x5636d30e5070 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70581b22b0 con 0x5636d30c4390 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70581af4f0 con 0x5636d30bc1b0 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f706494e740 10 client.0 _ll_register_callbacks cb 0x7f704800a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f705c977640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 0x5636d349c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f705c977640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 0x5636d349c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50958/0 (socket says 172.21.15.92:50958) 2024-09-21T02:10:19.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f705c977640 1 -- 172.21.15.92:0/3781928915 learned_addr learned my addr 172.21.15.92:0/3781928915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f7057fff640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30bc1b0 0x5636d349bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.322+0000 7f70577fe640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d349c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f705c977640 1 -- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30bc1b0 msgr2=0x5636d349bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f705c977640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30bc1b0 0x5636d349bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f705c977640 1 -- 172.21.15.92:0/3781928915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 msgr2=0x5636d349c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f705c977640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d349c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f705c977640 1 -- 172.21.15.92:0/3781928915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d3513630 con 0x5636d30e5070 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f705c977640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 0x5636d349c600 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7050010ea0 tx=0x7f7050009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3fc27c28f44f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f70377fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f70377fe640 1 -- 172.21.15.92:0/3781928915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050004b20 con 0x5636d30e5070 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f70377fe640 1 -- 172.21.15.92:0/3781928915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f705001b040 con 0x5636d30e5070 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70480adc80 con 0x5636d30e5070 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f70377fe640 1 -- 172.21.15.92:0/3781928915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7050004cc0 con 0x5636d30e5070 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f706494e740 10 client.17037 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636d3514790 con 0x5636d30e5070 2024-09-21T02:10:19.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.323+0000 7f70377fe640 1 -- 172.21.15.92:0/3781928915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70480adc80 con 0x5636d30e5070 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f70377fe640 1 -- 172.21.15.92:0/3781928915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f705001f070 con 0x5636d30e5070 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f706494e740 10 client.17037 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f706494e740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f706494e740 20 client.17037 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f706494e740 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d3078da0 0x5636d3517000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f706494e740 4 client.17037 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636d3400850 con 0x5636d3078da0 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.324+0000 7f7057fff640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d3078da0 0x5636d3517000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f70377fe640 10 client.17037 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f7057fff640 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d3078da0 0x5636d3517000 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e52a2e50d712ac server_cookie=a0eb6d347cf220ee in_seq=0 out_seq=0 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f70377fe640 1 -- 172.21.15.92:0/3781928915 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x5636d3400850 con 0x5636d3078da0 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f70377fe640 10 client.17037 handle_command_reply: tid=0 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 client.17037 shutdown 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 msgr2=0x5636d349c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 0x5636d349c600 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7050010ea0 tx=0x7f7050009720 comp rx=0 tx=0).stop 2024-09-21T02:10:19.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 shutdown_connections 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5636d3078da0 0x5636d3517000 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636d30e5070 0x5636d349c600 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636d30c4390 0x5636d349c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 --2- 172.21.15.92:0/3781928915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636d30bc1b0 0x5636d349bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 >> 172.21.15.92:0/3781928915 conn(0x5636d30d9460 msgr2=0x5636d348d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f70377fe640 0 client.17037 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 shutdown_connections 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 1 -- 172.21.15.92:0/3781928915 wait complete. 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.325+0000 7f706494e740 20 client.17037 trim_cache size 0 max 16384 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:10:19.330 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.331 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.332 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.333 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:10:19.335 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70400b65c0 msgr2=0x7f70400b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70400b65c0 0x7f70400b8ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7050005fd0 tx=0x7f70500099c0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 msgr2=0x7f705819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 0x7f705819c670 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f704800c920 tx=0x7f704800cde0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 shutdown_connections 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f70400b65c0 0x7f70400b8ad0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7058108cf0 0x7f70581a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70581083a0 0x7f705819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 --2- 172.21.15.92:0/257450533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7058102a60 0x7f705819c670 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 >> 172.21.15.92:0/257450533 conn(0x7f7058079730 msgr2=0x7f705806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 shutdown_connections 2024-09-21T02:10:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:19.326+0000 7f705e3da640 1 -- 172.21.15.92:0/257450533 wait complete. 2024-09-21T02:10:20.180 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:10:20.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:10:20.467 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:20.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:20 smithi136 ceph-mon[34754]: pgmap v521: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:20.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:20 smithi092 ceph-mon[31190]: pgmap v521: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:20.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:20 smithi099 ceph-mon[34404]: pgmap v521: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.945+0000 7f0b10dd7640 1 -- 172.21.15.92:0/253143440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 msgr2=0x7f0b0c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.945+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/253143440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c110320 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009ca0 tx=0x7f0b0402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.945+0000 7f0b10dd7640 1 -- 172.21.15.92:0/253143440 shutdown_connections 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.945+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/253143440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c110320 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.945+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/253143440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105320 0x7f0b0c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.945+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/253143440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b0c1049d0 0x7f0b0c104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.946+0000 7f0b10dd7640 1 -- 172.21.15.92:0/253143440 >> 172.21.15.92:0/253143440 conn(0x7f0b0c079740 msgr2=0x7f0b0c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.946+0000 7f0b10dd7640 1 -- 172.21.15.92:0/253143440 shutdown_connections 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.946+0000 7f0b10dd7640 1 -- 172.21.15.92:0/253143440 wait complete. 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.946+0000 7f0b10dd7640 1 Processor -- start 2024-09-21T02:10:20.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.946+0000 7f0b10dd7640 1 -- start start 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b10dd7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b0c1049d0 0x7f0b0c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b10dd7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105320 0x7f0b0c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b10dd7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b10dd7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c113ab0 con 0x7f0b0c10de30 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b10dd7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c113930 con 0x7f0b0c105320 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b10dd7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c113c30 con 0x7f0b0c1049d0 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50970/0 (socket says 172.21.15.92:50970) 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 -- 172.21.15.92:0/3693435394 learned_addr learned my addr 172.21.15.92:0/3693435394 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0a59c640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b0c1049d0 0x7f0b0c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b09d9b640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105320 0x7f0b0c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 -- 172.21.15.92:0/3693435394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b0c1049d0 msgr2=0x7f0b0c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b0c1049d0 0x7f0b0c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 -- 172.21.15.92:0/3693435394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105320 msgr2=0x7f0b0c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105320 0x7f0b0c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 -- 172.21.15.92:0/3693435394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1a0470 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.947+0000 7f0b0ad9d640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c19fd40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005cc0 tx=0x7f0b0402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.948+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04003bf0 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.948+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b04003d90 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.948+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1a0700 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.948+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0402fbc0 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.948+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a6ec0 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.949+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c06c240 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.950+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f0b0402fd60 con 0x7f0b0c10de30 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.951+0000 7f0afb7fe640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0af40b6510 0x7f0af40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.951+0000 7f0b0a59c640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0af40b6510 0x7f0af40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.951+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0b04108900 con 0x7f0b0c10de30 2024-09-21T02:10:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.951+0000 7f0b0a59c640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0af40b6510 0x7f0af40b8a20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c06c940 tx=0x7f0afc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:20.954+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b040cf170 con 0x7f0b0c10de30 2024-09-21T02:10:21.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.169+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b0c104de0 con 0x7f0b0c10de30 2024-09-21T02:10:21.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.170+0000 7f0afb7fe640 1 -- 172.21.15.92:0/3693435394 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f0b040d3550 con 0x7f0b0c10de30 2024-09-21T02:10:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:10:21.171 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:10:21.172 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:10:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.172+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0af40b6510 msgr2=0x7f0af40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.172+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0af40b6510 0x7f0af40b8a20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c06c940 tx=0x7f0afc00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.172+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 msgr2=0x7f0b0c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.172+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c19fd40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005cc0 tx=0x7f0b0402f380 comp rx=0 tx=0).stop 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 shutdown_connections 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0af40b6510 0x7f0af40b8a20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b0c10de30 0x7f0b0c19fd40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105320 0x7f0b0c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 --2- 172.21.15.92:0/3693435394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b0c1049d0 0x7f0b0c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 >> 172.21.15.92:0/3693435394 conn(0x7f0b0c079740 msgr2=0x7f0b0c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 shutdown_connections 2024-09-21T02:10:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:21.173+0000 7f0b10dd7640 1 -- 172.21.15.92:0/3693435394 wait complete. 2024-09-21T02:10:21.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:21 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3693435394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:10:21.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:21 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3693435394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:10:21.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:21 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3693435394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:10:21.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:10:22.054 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:22.456 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:22 smithi092 ceph-mon[31190]: pgmap v522: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:22.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:22 smithi136 ceph-mon[34754]: pgmap v522: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 -- 172.21.15.92:0/1747904225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c81094a0 msgr2=0x7ff5c81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/1747904225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c81094a0 0x7ff5c81098b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009900 tx=0x7ff5b002a410 comp rx=0 tx=0).stop 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 -- 172.21.15.92:0/1747904225 shutdown_connections 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/1747904225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8109df0 0x7ff5c8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/1747904225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c81094a0 0x7ff5c81098b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/1747904225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5c8103b60 0x7ff5c8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 -- 172.21.15.92:0/1747904225 >> 172.21.15.92:0/1747904225 conn(0x7ff5c807b850 msgr2=0x7ff5c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 -- 172.21.15.92:0/1747904225 shutdown_connections 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.546+0000 7ff5ce95e640 1 -- 172.21.15.92:0/1747904225 wait complete. 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 Processor -- start 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 -- start start 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8103b60 0x7ff5c819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5c81094a0 0x7ff5c819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 0x7ff5c819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113ab0 con 0x7ff5c8109df0 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113930 con 0x7ff5c8103b60 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ce95e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8113c30 con 0x7ff5c81094a0 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ccefb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 0x7ff5c819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ccefb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 0x7ff5c819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50978/0 (socket says 172.21.15.92:50978) 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.547+0000 7ff5ccefb640 1 -- 172.21.15.92:0/3405853683 learned_addr learned my addr 172.21.15.92:0/3405853683 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:22.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5c81094a0 0x7ff5c819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8103b60 0x7ff5c819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ccefb640 1 -- 172.21.15.92:0/3405853683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5c81094a0 msgr2=0x7ff5c819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ccefb640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5c81094a0 0x7ff5c819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ccefb640 1 -- 172.21.15.92:0/3405853683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8103b60 msgr2=0x7ff5c819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ccefb640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8103b60 0x7ff5c819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ccefb640 1 -- 172.21.15.92:0/3405853683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c81a03f0 con 0x7ff5c8109df0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8103b60 0x7ff5c819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ccefb640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 0x7ff5c819fcc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009fd0 tx=0x7ff5c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5c5ffb640 1 -- 172.21.15.92:0/3405853683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0017070 con 0x7ff5c8109df0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5c5ffb640 1 -- 172.21.15.92:0/3405853683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c0003e20 con 0x7ff5c8109df0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5c5ffb640 1 -- 172.21.15.92:0/3405853683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0010460 con 0x7ff5c8109df0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c81a06e0 con 0x7ff5c8109df0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.548+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c81a6e40 con 0x7ff5c8109df0 2024-09-21T02:10:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 Processor -- start 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5c5ffb640 1 -- 172.21.15.92:0/3405853683 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff5c0003970 con 0x7ff5c8109df0 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 -- start start 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d971be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d972120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55618d8e7ea0 con 0x55618d5a9390 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55618d8d72d0 con 0x55618d5a11b0 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55618d8db0e0 con 0x55618d5ca070 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.551+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d972120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d972120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50988/0 (socket says 172.21.15.92:50988) 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 -- 172.21.15.92:0/3888879055 learned_addr learned my addr 172.21.15.92:0/3888879055 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d971be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5c5ffb640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5b40b6590 0x7ff5b40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5ccefb640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5b40b6590 0x7ff5b40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 -- 172.21.15.92:0/3888879055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 msgr2=0x55618d9716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 -- 172.21.15.92:0/3888879055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 msgr2=0x55618d971be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d971be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 -- 172.21.15.92:0/3888879055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618d635650 con 0x55618d5a9390 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5c5ffb640 1 -- 172.21.15.92:0/3405853683 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff5c00e7010 con 0x7ff5c8109df0 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d972120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00057c0 tx=0x7ff5b00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fa74a6781a493f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5be7fc640 1 -- 172.21.15.92:0/3888879055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0035040 con 0x55618d5a9390 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5be7fc640 1 -- 172.21.15.92:0/3888879055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b0004030 con 0x55618d5a9390 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.552+0000 7ff5be7fc640 1 -- 172.21.15.92:0/3888879055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00038e0 con 0x55618d5a9390 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3888879055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 msgr2=0x55618d972120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5ccefb640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5b40b6590 0x7ff5b40b8aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00a97f0 tx=0x7ff5c0004020 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d972120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00057c0 tx=0x7ff5b00058b0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3888879055 shutdown_connections 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d972120 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d971be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3888879055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3888879055 >> 172.21.15.92:0/3888879055 conn(0x55618d5be460 msgr2=0x55618d96f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3888879055 shutdown_connections 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.553+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3888879055 wait complete. 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 -- start start 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 Processor -- start 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5a11b0 0x55618d97cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 0x55618d989140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55618d738700 con 0x55618d5ca070 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55618d872160 con 0x55618d5a9390 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55618d97fe20 con 0x55618d5a11b0 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5d4ed2740 10 client.0 _ll_register_callbacks cb 0x55618d88b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 0x55618d989140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 0x55618d989140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50990/0 (socket says 172.21.15.92:50990) 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.554+0000 7ff5bffff640 1 -- 172.21.15.92:0/4287905317 learned_addr learned my addr 172.21.15.92:0/4287905317 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5a11b0 0x55618d97cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 -- 172.21.15.92:0/4287905317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5a11b0 msgr2=0x55618d97cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5a11b0 0x55618d97cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 -- 172.21.15.92:0/4287905317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 msgr2=0x55618d97d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 -- 172.21.15.92:0/4287905317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618d89fc90 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5bffff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 0x55618d989140 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0031440 tx=0x7ff5b0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd3b00948483c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5a77fe640 1 -- 172.21.15.92:0/4287905317 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0035050 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b0004030 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5a77fe640 1 -- 172.21.15.92:0/4287905317 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff5b0003660 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5a77fe640 1 -- 172.21.15.92:0/4287905317 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5b0003860 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5d4ed2740 10 client.17055 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55618d98bb40 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.555+0000 7ff5a77fe640 1 -- 172.21.15.92:0/4287905317 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b0004030 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5a77fe640 1 -- 172.21.15.92:0/4287905317 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5b003a040 con 0x55618d5ca070 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5d4ed2740 10 client.17055 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5d4ed2740 10 client.17055 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:10:22.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5d4ed2740 20 client.17055 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d5889f0 0x55618d98eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5d4ed2740 4 client.17055 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618d8e9df0 con 0x55618d5889f0 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d5889f0 0x55618d98eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5a77fe640 10 client.17055 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.556+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d5889f0 0x55618d98eea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88cdd266de2bdea5 server_cookie=6b062e9d99a5c80d in_seq=0 out_seq=0 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5a77fe640 1 -- 172.21.15.92:0/4287905317 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55618d8e9df0 con 0x55618d5889f0 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5a77fe640 10 client.17055 handle_command_reply: tid=0 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 client.17055 shutdown 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 msgr2=0x55618d989140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 0x55618d989140 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0031440 tx=0x7ff5b0005c90 comp rx=0 tx=0).stop 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 shutdown_connections 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d5889f0 0x55618d98eea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5ca070 0x55618d989140 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/4287905317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5a11b0 0x55618d97cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 >> 172.21.15.92:0/4287905317 conn(0x55618d5be460 msgr2=0x55618d974830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5a77fe640 0 client.17055 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 shutdown_connections 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/4287905317 wait complete. 2024-09-21T02:10:22.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.557+0000 7ff5d4ed2740 20 client.17055 trim_cache size 0 max 16384 2024-09-21T02:10:22.578 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 Processor -- start 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 -- start start 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 0x55618d973090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d96ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81af380 con 0x55618d5a11b0 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81b0ec0 con 0x55618d5a9390 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81afa50 con 0x55618d5ca070 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 0x55618d973090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 0x55618d973090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51004/0 (socket says 172.21.15.92:51004) 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 -- 172.21.15.92:0/701722761 learned_addr learned my addr 172.21.15.92:0/701722761 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 -- 172.21.15.92:0/701722761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 msgr2=0x55618d96ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d96ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 -- 172.21.15.92:0/701722761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 msgr2=0x55618d97e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 -- 172.21.15.92:0/701722761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618d97dbe0 con 0x55618d5a11b0 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.571+0000 7ff5bffff640 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 0x55618d973090 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0031740 tx=0x7ff5b0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2d9c64a837a32c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5bdffb640 1 -- 172.21.15.92:0/701722761 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0035040 con 0x55618d5a11b0 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5bdffb640 1 -- 172.21.15.92:0/701722761 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b005c020 con 0x55618d5a11b0 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5bdffb640 1 -- 172.21.15.92:0/701722761 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0003a00 con 0x55618d5a11b0 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/701722761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 msgr2=0x55618d973090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 0x55618d973090 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0031740 tx=0x7ff5b0005e70 comp rx=0 tx=0).stop 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/701722761 shutdown_connections 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d96ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a9390 0x55618d97e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/701722761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a11b0 0x55618d973090 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.572+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/701722761 >> 172.21.15.92:0/701722761 conn(0x55618d5be460 msgr2=0x55618d97e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/701722761 shutdown_connections 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/701722761 wait complete. 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 -- start start 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 Processor -- start 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d98f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d98fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81b07f0 con 0x55618d5a9390 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81b0120 con 0x55618d5a11b0 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c81ad360 con 0x55618d5ca070 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.573+0000 7ff5d4ed2740 10 client.0 _ll_register_callbacks cb 0x7ff5b8009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d98fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d98fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51016/0 (socket says 172.21.15.92:51016) 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 -- 172.21.15.92:0/3447430133 learned_addr learned my addr 172.21.15.92:0/3447430133 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d98f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 -- 172.21.15.92:0/3447430133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 msgr2=0x55618d9f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 -- 172.21.15.92:0/3447430133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 msgr2=0x55618d98f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d98f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 -- 172.21.15.92:0/3447430133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618d9f7c20 con 0x55618d5a9390 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5c7fff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d98fc40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8005fd0 tx=0x7ff5b8012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914cbbb6443f06d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5a77fe640 1 -- 172.21.15.92:0/3447430133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8011910 con 0x55618d5a9390 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5a77fe640 1 -- 172.21.15.92:0/3447430133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff5b8011ab0 con 0x55618d5a9390 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b005c020 con 0x55618d5a9390 2024-09-21T02:10:22.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5a77fe640 1 -- 172.21.15.92:0/3447430133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5b8011cb0 con 0x55618d5a9390 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5d4ed2740 10 client.17061 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.574+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55618d9f8ea0 con 0x55618d5a9390 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5a77fe640 1 -- 172.21.15.92:0/3447430133 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b005c020 con 0x55618d5a9390 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5a77fe640 1 -- 172.21.15.92:0/3447430133 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5b800f040 con 0x55618d5a9390 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5d4ed2740 10 client.17061 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5d4ed2740 10 client.17061 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5d4ed2740 20 client.17061 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d55dda0 0x55618d9fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5d4ed2740 4 client.17061 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55618d8e58a0 con 0x55618d55dda0 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d55dda0 0x55618d9fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5a77fe640 10 client.17061 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:10:22.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.575+0000 7ff5bffff640 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d55dda0 0x55618d9fb690 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7f73412ed1e3482 server_cookie=c4cf3bee6f95b8cf in_seq=0 out_seq=0 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5a77fe640 1 -- 172.21.15.92:0/3447430133 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55618d8e58a0 con 0x55618d55dda0 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5a77fe640 10 client.17061 handle_command_reply: tid=0 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 client.17061 shutdown 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 msgr2=0x55618d98fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d98fc40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8005fd0 tx=0x7ff5b8012a90 comp rx=0 tx=0).stop 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 shutdown_connections 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55618d55dda0 0x55618d9fb690 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55618d5ca070 0x55618d9f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55618d5a9390 0x55618d98fc40 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5a77fe640 0 client.17061 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 --2- 172.21.15.92:0/3447430133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55618d5a11b0 0x55618d98f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 >> 172.21.15.92:0/3447430133 conn(0x55618d5be460 msgr2=0x55618d972e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 shutdown_connections 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 1 -- 172.21.15.92:0/3447430133 wait complete. 2024-09-21T02:10:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.576+0000 7ff5d4ed2740 20 client.17061 trim_cache size 0 max 16384 2024-09-21T02:10:22.585 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:10:22.586 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:22 smithi099 ceph-mon[34404]: pgmap v522: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5b40b6590 msgr2=0x7ff5b40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5b40b6590 0x7ff5b40b8aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00a97f0 tx=0x7ff5c0004020 comp rx=0 tx=0).stop 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 msgr2=0x7ff5c819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 0x7ff5c819fcc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009fd0 tx=0x7ff5c000b260 comp rx=0 tx=0).stop 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 shutdown_connections 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff5b40b6590 0x7ff5b40b8aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8109df0 0x7ff5c819fcc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5c81094a0 0x7ff5c819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 --2- 172.21.15.92:0/3405853683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c8103b60 0x7ff5c819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:22.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 >> 172.21.15.92:0/3405853683 conn(0x7ff5c807b850 msgr2=0x7ff5c8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:22.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 shutdown_connections 2024-09-21T02:10:22.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:22.577+0000 7ff5ce95e640 1 -- 172.21.15.92:0/3405853683 wait complete. 2024-09-21T02:10:23.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:10:23.509 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 -- 172.21.15.92:0/1485880802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 msgr2=0x7fe03810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 --2- 172.21.15.92:0/1485880802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03810b920 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe030009ca0 tx=0x7fe03002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 -- 172.21.15.92:0/1485880802 shutdown_connections 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 --2- 172.21.15.92:0/1485880802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03810b920 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 --2- 172.21.15.92:0/1485880802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe038106b20 0x7fe038108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 --2- 172.21.15.92:0/1485880802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe038101a90 0x7fe038101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 -- 172.21.15.92:0/1485880802 >> 172.21.15.92:0/1485880802 conn(0x7fe0380fd690 msgr2=0x7fe0380ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 -- 172.21.15.92:0/1485880802 shutdown_connections 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.816+0000 7fe03e389640 1 -- 172.21.15.92:0/1485880802 wait complete. 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 Processor -- start 2024-09-21T02:10:23.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 -- start start 2024-09-21T02:10:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe038101a90 0x7fe038199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe038106b20 0x7fe03819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810f300 con 0x7fe038109480 2024-09-21T02:10:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810f180 con 0x7fe038101a90 2024-09-21T02:10:23.820 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03e389640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810f480 con 0x7fe038106b20 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03c926640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.817+0000 7fe03c926640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03819b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34586/0 (socket says 172.21.15.92:34586) 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 -- 172.21.15.92:0/2825562743 learned_addr learned my addr 172.21.15.92:0/2825562743 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe0377fe640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe038106b20 0x7fe03819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe037fff640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe038101a90 0x7fe038199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 -- 172.21.15.92:0/2825562743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe038106b20 msgr2=0x7fe03819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe038106b20 0x7fe03819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 -- 172.21.15.92:0/2825562743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe038101a90 msgr2=0x7fe038199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe038101a90 0x7fe038199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 -- 172.21.15.92:0/2825562743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03819b8a0 con 0x7fe038109480 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03c926640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03819b110 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097b0 tx=0x7fe030003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe0357fa640 1 -- 172.21.15.92:0/2825562743 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030004240 con 0x7fe038109480 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe0357fa640 1 -- 172.21.15.92:0/2825562743 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03002f380 con 0x7fe038109480 2024-09-21T02:10:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03819bb30 con 0x7fe038109480 2024-09-21T02:10:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe0357fa640 1 -- 172.21.15.92:0/2825562743 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03002fc20 con 0x7fe038109480 2024-09-21T02:10:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.818+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0381a2130 con 0x7fe038109480 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.821+0000 7fe0453d1740 1 Processor -- start 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0357fa640 1 -- 172.21.15.92:0/2825562743 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe0300043e0 con 0x7fe038109480 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 -- start start 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b301b0 0x5628e6ed7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ed84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b0f4d0 0x5628e6ed9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5628e6e4a6f0 con 0x5628e6b072f0 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5628e6e39b20 con 0x5628e6b301b0 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0453d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628e6e3d930 con 0x5628e6b0f4d0 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ed84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.824 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ed84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34588/0 (socket says 172.21.15.92:34588) 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 -- 172.21.15.92:0/3962778663 learned_addr learned my addr 172.21.15.92:0/3962778663 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe0377fe640 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b301b0 0x5628e6ed7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 -- 172.21.15.92:0/3962778663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b0f4d0 msgr2=0x5628e6ed9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b0f4d0 0x5628e6ed9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 -- 172.21.15.92:0/3962778663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b301b0 msgr2=0x5628e6ed7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b301b0 0x5628e6ed7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 -- 172.21.15.92:0/3962778663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e6dff530 con 0x5628e6b072f0 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.822+0000 7fe037fff640 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ed84c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ad00 tx=0x7fe02800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68593c6bd3ea700 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.823+0000 7fe0357fa640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0240b6510 0x7fe0240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.823+0000 7fe02e7fc640 1 -- 172.21.15.92:0/3962778663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02801d070 con 0x5628e6b072f0 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.823+0000 7fe02e7fc640 1 -- 172.21.15.92:0/3962778663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe028018af0 con 0x5628e6b072f0 2024-09-21T02:10:23.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.823+0000 7fe02e7fc640 1 -- 172.21.15.92:0/3962778663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028010b50 con 0x5628e6b072f0 2024-09-21T02:10:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.823+0000 7fe0377fe640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0240b6510 0x7fe0240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.823+0000 7fe0357fa640 1 -- 172.21.15.92:0/2825562743 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe030108a00 con 0x7fe038109480 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 -- 172.21.15.92:0/3962778663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 msgr2=0x5628e6ed84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0377fe640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0240b6510 0x7fe0240b8a20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe020005d00 tx=0x7fe02000b860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ed84c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ad00 tx=0x7fe02800f790 comp rx=0 tx=0).stop 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 -- 172.21.15.92:0/3962778663 shutdown_connections 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b0f4d0 0x5628e6ed9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ed84c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 --2- 172.21.15.92:0/3962778663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b301b0 0x5628e6ed7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 -- 172.21.15.92:0/3962778663 >> 172.21.15.92:0/3962778663 conn(0x5628e6b24300 msgr2=0x5628e6ed38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 -- 172.21.15.92:0/3962778663 shutdown_connections 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 -- 172.21.15.92:0/3962778663 wait complete. 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 -- start start 2024-09-21T02:10:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 Processor -- start 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b072f0 0x5628e6ee15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.824+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b301b0 0x5628e6ee27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5628e6d71ab0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5628e6ddf730 con 0x5628e6b072f0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5628e6ed9b30 con 0x5628e6b301b0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 10 client.0 _ll_register_callbacks cb 0x5628e6e336b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34590/0 (socket says 172.21.15.92:34590) 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 -- 172.21.15.92:0/2218412598 learned_addr learned my addr 172.21.15.92:0/2218412598 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe03c926640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b301b0 0x5628e6ee27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe037fff640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b072f0 0x5628e6ee15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 -- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b301b0 msgr2=0x5628e6ee27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b301b0 0x5628e6ee27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 -- 172.21.15.92:0/2218412598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b072f0 msgr2=0x5628e6ee15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b072f0 0x5628e6ee15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 -- 172.21.15.92:0/2218412598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e6b9b790 con 0x5628e6b0f4d0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe037fff640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b072f0 0x5628e6ee15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe03c926640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b301b0 0x5628e6ee27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0377fe640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee1c70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe020012700 tx=0x7fe0200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20da93ff5917054 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0177fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0177fe640 1 -- 172.21.15.92:0/2218412598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02001f8f0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0177fe640 1 -- 172.21.15.92:0/2218412598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe02001fa90 con 0x5628e6b0f4d0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe028018af0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0177fe640 1 -- 172.21.15.92:0/2218412598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe02002d070 con 0x5628e6b0f4d0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 10 client.17073 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.825+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628e6c14620 con 0x5628e6b0f4d0 2024-09-21T02:10:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0177fe640 1 -- 172.21.15.92:0/2218412598 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe028018af0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0177fe640 1 -- 172.21.15.92:0/2218412598 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0200313d0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0453d1740 10 client.17073 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0453d1740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0453d1740 20 client.17073 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0453d1740 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6aeed80 0x5628e6eeeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0453d1740 4 client.17073 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e6e4c640 con 0x5628e6aeed80 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.826+0000 7fe037fff640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6aeed80 0x5628e6eeeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.827+0000 7fe0177fe640 10 client.17073 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.827+0000 7fe037fff640 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6aeed80 0x5628e6eeeeb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13008f65c78afaa1 server_cookie=59e6048eb0d3e126 in_seq=0 out_seq=0 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.827+0000 7fe0177fe640 1 -- 172.21.15.92:0/2218412598 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628e6e4c640 con 0x5628e6aeed80 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.827+0000 7fe0177fe640 10 client.17073 handle_command_reply: tid=0 2024-09-21T02:10:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.827+0000 7fe0453d1740 1 client.17073 shutdown 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 msgr2=0x5628e6ee1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee1c70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe020012700 tx=0x7fe0200046c0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 shutdown_connections 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6aeed80 0x5628e6eeeeb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b301b0 0x5628e6ee27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee1c70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0177fe640 0 client.17073 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 --2- 172.21.15.92:0/2218412598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b072f0 0x5628e6ee15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 >> 172.21.15.92:0/2218412598 conn(0x5628e6b24300 msgr2=0x5628e6ed1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 shutdown_connections 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 1 -- 172.21.15.92:0/2218412598 wait complete. 2024-09-21T02:10:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.828+0000 7fe0453d1740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 Processor -- start 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 -- start start 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6aeed80 0x5628e6ee07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b072f0 0x5628e6ece5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6edfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381aafc0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381acb00 con 0x5628e6aeed80 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe0453d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381ab690 con 0x5628e6b072f0 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe03c926640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6edfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe03c926640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6edfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34596/0 (socket says 172.21.15.92:34596) 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.842+0000 7fe03c926640 1 -- 172.21.15.92:0/681305312 learned_addr learned my addr 172.21.15.92:0/681305312 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe037fff640 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6aeed80 0x5628e6ee07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe0377fe640 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b072f0 0x5628e6ece5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe03c926640 1 -- 172.21.15.92:0/681305312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b072f0 msgr2=0x5628e6ece5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe03c926640 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b072f0 0x5628e6ece5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe03c926640 1 -- 172.21.15.92:0/681305312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6aeed80 msgr2=0x5628e6ee07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe03c926640 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6aeed80 0x5628e6ee07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe03c926640 1 -- 172.21.15.92:0/681305312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e6ee2b50 con 0x5628e6b0f4d0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe03c926640 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6edfac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe0301064e0 tx=0x7fe03002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2f480b8d8a19f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe02dffb640 1 -- 172.21.15.92:0/681305312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300ca250 con 0x5628e6b0f4d0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe02dffb640 1 -- 172.21.15.92:0/681305312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe030036070 con 0x5628e6b0f4d0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.843+0000 7fe02dffb640 1 -- 172.21.15.92:0/681305312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300b60c0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 -- 172.21.15.92:0/681305312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 msgr2=0x5628e6edfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6edfac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe0301064e0 tx=0x7fe03002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 -- 172.21.15.92:0/681305312 shutdown_connections 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b0f4d0 0x5628e6edfac0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6b072f0 0x5628e6ece5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 --2- 172.21.15.92:0/681305312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6aeed80 0x5628e6ee07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 -- 172.21.15.92:0/681305312 >> 172.21.15.92:0/681305312 conn(0x5628e6b24300 msgr2=0x5628e6ed3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 -- 172.21.15.92:0/681305312 shutdown_connections 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 -- 172.21.15.92:0/681305312 wait complete. 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.844+0000 7fe0453d1740 1 -- start start 2024-09-21T02:10:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 Processor -- start 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6aeed80 0x5628e6ee6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ee6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381ac430 con 0x5628e6b072f0 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381abd60 con 0x5628e6b0f4d0 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381a8fa0 con 0x5628e6aeed80 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0453d1740 10 client.0 _ll_register_callbacks cb 0x7fe020011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe037fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6aeed80 0x5628e6ee6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe037fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6aeed80 0x5628e6ee6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:49856/0 (socket says 172.21.15.92:49856) 2024-09-21T02:10:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe037fff640 1 -- 172.21.15.92:0/447692637 learned_addr learned my addr 172.21.15.92:0/447692637 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe03c926640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0377fe640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ee6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe03c926640 1 -- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6aeed80 msgr2=0x5628e6ee6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe03c926640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6aeed80 0x5628e6ee6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe03c926640 1 -- 172.21.15.92:0/447692637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 msgr2=0x5628e6ee6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe03c926640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ee6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe03c926640 1 -- 172.21.15.92:0/447692637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e6ee82a0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.845+0000 7fe0377fe640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ee6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe03c926640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee7b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe030031900 tx=0x7fe0300cf270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe940359d483dd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0177fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0177fe640 1 -- 172.21.15.92:0/447692637 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300ce680 con 0x5628e6b0f4d0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe030036070 con 0x5628e6b0f4d0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0177fe640 1 -- 172.21.15.92:0/447692637 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe03002fdc0 con 0x5628e6b0f4d0 2024-09-21T02:10:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0177fe640 1 -- 172.21.15.92:0/447692637 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0300f6100 con 0x5628e6b0f4d0 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 10 client.26177 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628e6ee9650 con 0x5628e6b0f4d0 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0177fe640 1 -- 172.21.15.92:0/447692637 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe030036070 con 0x5628e6b0f4d0 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0177fe640 1 -- 172.21.15.92:0/447692637 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0300cad50 con 0x5628e6b0f4d0 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 10 client.26177 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 20 client.26177 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6b301b0 0x5628e6f5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 4 client.26177 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.846+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628e6e480f0 con 0x5628e6b301b0 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.847+0000 7fe037fff640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6b301b0 0x5628e6f5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.847+0000 7fe0177fe640 10 client.26177 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.847+0000 7fe037fff640 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6b301b0 0x5628e6f5ba20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adad53bb65f5e60a server_cookie=e69fce8ef0e8a109 in_seq=0 out_seq=0 2024-09-21T02:10:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0177fe640 1 -- 172.21.15.92:0/447692637 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628e6e480f0 con 0x5628e6b301b0 2024-09-21T02:10:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0177fe640 10 client.26177 handle_command_reply: tid=0 2024-09-21T02:10:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 client.26177 shutdown 2024-09-21T02:10:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b0f4d0 msgr2=0x5628e6ee7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee7b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe030031900 tx=0x7fe0300cf270 comp rx=0 tx=0).stop 2024-09-21T02:10:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 shutdown_connections 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5628e6b301b0 0x5628e6f5ba20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628e6b0f4d0 0x5628e6ee7b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5628e6b072f0 0x5628e6ee6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 --2- 172.21.15.92:0/447692637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628e6aeed80 0x5628e6ee6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0177fe640 0 client.26177 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 >> 172.21.15.92:0/447692637 conn(0x5628e6b24300 msgr2=0x5628e6ed3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 shutdown_connections 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 1 -- 172.21.15.92:0/447692637 wait complete. 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.863+0000 7fe0453d1740 20 client.26177 trim_cache size 0 max 16384 2024-09-21T02:10:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:10:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:10:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:10:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:10:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0240b6510 msgr2=0x7fe0240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0240b6510 0x7fe0240b8a20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe020005d00 tx=0x7fe02000b860 comp rx=0 tx=0).stop 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 msgr2=0x7fe03819b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03819b110 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097b0 tx=0x7fe030003c20 comp rx=0 tx=0).stop 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 shutdown_connections 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe0240b6510 0x7fe0240b8a20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe038109480 0x7fe03819b110 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe038106b20 0x7fe03819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 --2- 172.21.15.92:0/2825562743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe038101a90 0x7fe038199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 >> 172.21.15.92:0/2825562743 conn(0x7fe0380fd690 msgr2=0x7fe0381078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 shutdown_connections 2024-09-21T02:10:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:23.864+0000 7fe03e389640 1 -- 172.21.15.92:0/2825562743 wait complete. 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 -- 172.21.15.92:0/2993766882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc102ae0 msgr2=0x7f71dc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 --2- 172.21.15.92:0/2993766882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc102ae0 0x7f71dc102ef0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009ca0 tx=0x7f71d402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 -- 172.21.15.92:0/2993766882 shutdown_connections 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 --2- 172.21.15.92:0/2993766882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71dc108d20 0x7f71dc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 --2- 172.21.15.92:0/2993766882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71dc1083d0 0x7f71dc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 --2- 172.21.15.92:0/2993766882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc102ae0 0x7f71dc102ef0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 -- 172.21.15.92:0/2993766882 >> 172.21.15.92:0/2993766882 conn(0x7f71dc079790 msgr2=0x7f71dc079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 -- 172.21.15.92:0/2993766882 shutdown_connections 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.955+0000 7f71e0de8640 1 -- 172.21.15.92:0/2993766882 wait complete. 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.956+0000 7f71e0de8640 1 Processor -- start 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.956+0000 7f71e0de8640 1 -- start start 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71dc102ae0 0x7f71dc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71dc1083d0 0x7f71dc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 0x7f71dc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc113b50 con 0x7f71dc108d20 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1139d0 con 0x7f71dc102ae0 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc113cd0 con 0x7f71dc1083d0 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 0x7f71dc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 0x7f71dc19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34624/0 (socket says 172.21.15.92:34624) 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 -- 172.21.15.92:0/1034739219 learned_addr learned my addr 172.21.15.92:0/1034739219 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71dc1083d0 0x7f71dc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71da59c640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71dc102ae0 0x7f71dc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 -- 172.21.15.92:0/1034739219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71dc1083d0 msgr2=0x7f71dc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71dc1083d0 0x7f71dc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 -- 172.21.15.92:0/1034739219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71dc102ae0 msgr2=0x7f71dc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71dc102ae0 0x7f71dc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 -- 172.21.15.92:0/1034739219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc1a04d0 con 0x7f71dc108d20 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71dad9d640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 0x7f71dc19fda0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009fd0 tx=0x7f71cc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71d37fe640 1 -- 172.21.15.92:0/1034739219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc011830 con 0x7f71dc108d20 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71d37fe640 1 -- 172.21.15.92:0/1034739219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71cc016070 con 0x7f71dc108d20 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a07c0 con 0x7f71dc108d20 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.957+0000 7f71d37fe640 1 -- 172.21.15.92:0/1034739219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc010de0 con 0x7f71dc108d20 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.958+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a6f20 con 0x7f71dc108d20 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.960+0000 7f71e735c740 1 Processor -- start 2024-09-21T02:10:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71d37fe640 1 -- 172.21.15.92:0/1034739219 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f71cc0119d0 con 0x7f71dc108d20 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 -- start start 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbfd070 0x56348bfa46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfa4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbdc390 0x56348bfa5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56348bf1aea0 con 0x56348bbd41b0 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56348bf0a2d0 con 0x56348bbfd070 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71e735c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56348bf0e0e0 con 0x56348bbdc390 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfa4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfa4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34632/0 (socket says 172.21.15.92:34632) 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 -- 172.21.15.92:0/3021748610 learned_addr learned my addr 172.21.15.92:0/3021748610 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbdc390 0x56348bfa5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbfd070 0x56348bfa46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71d37fe640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c80b6510 0x7f71c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 -- 172.21.15.92:0/3021748610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbdc390 msgr2=0x56348bfa5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbdc390 0x56348bfa5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 -- 172.21.15.92:0/3021748610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbfd070 msgr2=0x56348bfa46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbfd070 0x56348bfa46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71da59c640 1 -- 172.21.15.92:0/3021748610 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348bc68650 con 0x56348bbd41b0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.961+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbdc390 0x56348bfa5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.962+0000 7f71dad9d640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c80b6510 0x7f71c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.962+0000 7f71da59c640 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfa4be0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f71d4035000 tx=0x7f71d4009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ce122354dbd005 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.962+0000 7f71d37fe640 1 -- 172.21.15.92:0/1034739219 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f71cc0e6890 con 0x7f71dc108d20 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.962+0000 7f71d0ff9640 1 -- 172.21.15.92:0/3021748610 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4031450 con 0x56348bbd41b0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.962+0000 7f71d0ff9640 1 -- 172.21.15.92:0/3021748610 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4036030 con 0x56348bbd41b0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.962+0000 7f71d0ff9640 1 -- 172.21.15.92:0/3021748610 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4004320 con 0x56348bbd41b0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 -- 172.21.15.92:0/3021748610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 msgr2=0x56348bfa4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71dad9d640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c80b6510 0x7f71c80b8a20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0ac6a0 tx=0x7f71cc0d0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfa4be0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f71d4035000 tx=0x7f71d4009940 comp rx=0 tx=0).stop 2024-09-21T02:10:23.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 -- 172.21.15.92:0/3021748610 shutdown_connections 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbdc390 0x56348bfa5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfa4be0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 --2- 172.21.15.92:0/3021748610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbfd070 0x56348bfa46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 -- 172.21.15.92:0/3021748610 >> 172.21.15.92:0/3021748610 conn(0x56348bbf1460 msgr2=0x56348bfa2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 -- 172.21.15.92:0/3021748610 shutdown_connections 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 -- 172.21.15.92:0/3021748610 wait complete. 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.963+0000 7f71e735c740 1 -- start start 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 Processor -- start 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfb45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbfd070 0x56348bfb4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56348bd6b700 con 0x56348bbd41b0 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56348bea5160 con 0x56348bbdc390 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56348bfb7000 con 0x56348bbfd070 2024-09-21T02:10:23.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71e735c740 10 client.0 _ll_register_callbacks cb 0x56348bebe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34646/0 (socket says 172.21.15.92:34646) 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 -- 172.21.15.92:0/1063843179 learned_addr learned my addr 172.21.15.92:0/1063843179 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71da59c640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfb45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 -- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbfd070 msgr2=0x56348bfb4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbfd070 0x56348bfb4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 -- 172.21.15.92:0/1063843179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 msgr2=0x56348bfb45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfb45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71d9d9b640 1 -- 172.21.15.92:0/1063843179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348bed2c90 con 0x56348bbd41b0 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.964+0000 7f71da59c640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfb45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb40a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f71c400bbb0 tx=0x7f71c40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fb926478a27625 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1063843179 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4005020 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1063843179 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f71c40051c0 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4036030 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1063843179 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71c40053c0 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 10 client.17088 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56348bfc2c50 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1063843179 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4036030 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1063843179 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71c4019070 con 0x56348bbd41b0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 10 client.17088 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 10 client.17088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 20 client.17088 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbbb9f0 0x56348bfc6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 4 client.17088 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.965+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56348bf1cdf0 con 0x56348bbbb9f0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.966+0000 7f71da59c640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbbb9f0 0x56348bfc6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.966+0000 7f71c1ffb640 10 client.17088 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.966+0000 7f71da59c640 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbbb9f0 0x56348bfc6180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e74fa47c9e3e3c4 server_cookie=150ce539af074803 in_seq=0 out_seq=0 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1063843179 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56348bf1cdf0 con 0x56348bbbb9f0 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71c1ffb640 10 client.17088 handle_command_reply: tid=0 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 client.17088 shutdown 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 msgr2=0x56348bfb40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb40a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f71c400bbb0 tx=0x7f71c40039c0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 shutdown_connections 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbfd070 0x56348bfb4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbbb9f0 0x56348bfc6180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71c1ffb640 0 client.17088 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfb45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 --2- 172.21.15.92:0/1063843179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb40a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.967+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 >> 172.21.15.92:0/1063843179 conn(0x56348bbf1460 msgr2=0x56348bfa6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.968+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 shutdown_connections 2024-09-21T02:10:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.968+0000 7f71e735c740 1 -- 172.21.15.92:0/1063843179 wait complete. 2024-09-21T02:10:23.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.968+0000 7f71e735c740 20 client.17088 trim_cache size 0 max 16384 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.980+0000 7f71e735c740 1 Processor -- start 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 -- start start 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfb5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbd41b0 0x56348bfa5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 0x56348bfb1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ad2c0 con 0x56348bbdc390 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1aee00 con 0x56348bbd41b0 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71e735c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ad990 con 0x56348bbbb9f0 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 0x56348bfb1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 0x56348bfb1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34662/0 (socket says 172.21.15.92:34662) 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 -- 172.21.15.92:0/3745310601 learned_addr learned my addr 172.21.15.92:0/3745310601 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfb5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71da59c640 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbd41b0 0x56348bfa5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 -- 172.21.15.92:0/3745310601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 msgr2=0x56348bfb5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfb5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 -- 172.21.15.92:0/3745310601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbd41b0 msgr2=0x56348bfa5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbd41b0 0x56348bfa5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.981+0000 7f71d9d9b640 1 -- 172.21.15.92:0/3745310601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348bfa54e0 con 0x56348bbdc390 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.982+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 0x56348bfb1290 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f71c4004540 tx=0x7f71c40043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa117e4bbcb3389d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.982+0000 7f71c3fff640 1 -- 172.21.15.92:0/3745310601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4029070 con 0x56348bbdc390 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.982+0000 7f71c3fff640 1 -- 172.21.15.92:0/3745310601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4011850 con 0x56348bbdc390 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.982+0000 7f71c3fff640 1 -- 172.21.15.92:0/3745310601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4031400 con 0x56348bbdc390 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 -- 172.21.15.92:0/3745310601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 msgr2=0x56348bfb1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 0x56348bfb1290 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f71c4004540 tx=0x7f71c40043d0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 -- 172.21.15.92:0/3745310601 shutdown_connections 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbdc390 0x56348bfb1290 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbd41b0 0x56348bfa5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- 172.21.15.92:0/3745310601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfb5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 -- 172.21.15.92:0/3745310601 >> 172.21.15.92:0/3745310601 conn(0x56348bbf1460 msgr2=0x56348bfa56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 -- 172.21.15.92:0/3745310601 shutdown_connections 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 -- 172.21.15.92:0/3745310601 wait complete. 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 -- start start 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 Processor -- start 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.983+0000 7f71e735c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71e735c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ae730 con 0x56348bbd41b0 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71e735c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ae060 con 0x56348bbdc390 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71e735c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1ab2a0 con 0x56348bbbb9f0 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71e735c740 10 client.0 _ll_register_callbacks cb 0x7f71c4004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:23.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34672/0 (socket says 172.21.15.92:34672) 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 -- 172.21.15.92:0/1615067351 learned_addr learned my addr 172.21.15.92:0/1615067351 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 -- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 msgr2=0x56348bfbca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfbca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 -- 172.21.15.92:0/1615067351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 msgr2=0x56348bfba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 -- 172.21.15.92:0/1615067351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348bfbab50 con 0x56348bbd41b0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71da59c640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb9f00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f71d402f550 tx=0x7f71d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c67c6a730a93cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1615067351 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4003ba0 con 0x56348bbd41b0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1615067351 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f71d404a070 con 0x56348bbd41b0 2024-09-21T02:10:23.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.984+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c4011850 con 0x56348bbd41b0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1615067351 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71d4003d40 con 0x56348bbd41b0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 10 client.17094 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56348c030a80 con 0x56348bbd41b0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1615067351 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4011850 con 0x56348bbd41b0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1615067351 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71d4043070 con 0x56348bbd41b0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 10 client.17094 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 10 client.17094 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 20 client.17094 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbfd070 0x56348c032fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 4 client.17094 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.985+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56348bf188a0 con 0x56348bbfd070 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbfd070 0x56348c032fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71d9d9b640 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbfd070 0x56348c032fe0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59e64e89d6c4b4b4 server_cookie=1eaafd5e5d14f9ed in_seq=0 out_seq=0 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71c1ffb640 10 client.17094 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71c1ffb640 1 -- 172.21.15.92:0/1615067351 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56348bf188a0 con 0x56348bbfd070 2024-09-21T02:10:23.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71c1ffb640 10 client.17094 handle_command_reply: tid=0 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 client.17094 shutdown 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 msgr2=0x56348bfb9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb9f00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f71d402f550 tx=0x7f71d4005fd0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 shutdown_connections 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56348bbfd070 0x56348c032fe0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56348bbdc390 0x56348bfba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71c1ffb640 0 client.17094 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56348bbd41b0 0x56348bfb9f00 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 --2- 172.21.15.92:0/1615067351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348bbbb9f0 0x56348bfbca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 >> 172.21.15.92:0/1615067351 conn(0x56348bbf1460 msgr2=0x56348bfb5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.986+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 shutdown_connections 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e735c740 1 -- 172.21.15.92:0/1615067351 wait complete. 2024-09-21T02:10:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e735c740 20 client.17094 trim_cache size 0 max 16384 2024-09-21T02:10:23.999 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:10:23.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c80b6510 msgr2=0x7f71c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c80b6510 0x7f71c80b8a20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0ac6a0 tx=0x7f71cc0d0040 comp rx=0 tx=0).stop 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 msgr2=0x7f71dc19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 0x7f71dc19fda0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009fd0 tx=0x7f71cc00b8e0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 shutdown_connections 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f71c80b6510 0x7f71c80b8a20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71dc108d20 0x7f71dc19fda0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71dc1083d0 0x7f71dc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 --2- 172.21.15.92:0/1034739219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71dc102ae0 0x7f71dc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 >> 172.21.15.92:0/1034739219 conn(0x7f71dc079790 msgr2=0x7f71dc106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 shutdown_connections 2024-09-21T02:10:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:23.987+0000 7f71e0de8640 1 -- 172.21.15.92:0/1034739219 wait complete. 2024-09-21T02:10:24.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:24 smithi092 ceph-mon[31190]: pgmap v523: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:10:24.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:24 smithi099 ceph-mon[34404]: pgmap v523: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:10:24.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:10:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:24 smithi136 ceph-mon[34754]: pgmap v523: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:10:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 -- 172.21.15.92:0/442603691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 msgr2=0x7f1630102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 --2- 172.21.15.92:0/442603691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f1630102f20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1628009900 tx=0x7f162802a410 comp rx=0 tx=0).stop 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 -- 172.21.15.92:0/442603691 shutdown_connections 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 --2- 172.21.15.92:0/442603691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1630108da0 0x7f163010f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 --2- 172.21.15.92:0/442603691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1630108450 0x7f1630108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 --2- 172.21.15.92:0/442603691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f1630102f20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 -- 172.21.15.92:0/442603691 >> 172.21.15.92:0/442603691 conn(0x7f16300796d0 msgr2=0x7f1630079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 -- 172.21.15.92:0/442603691 shutdown_connections 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.453+0000 7f1636b8e640 1 -- 172.21.15.92:0/442603691 wait complete. 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 Processor -- start 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 -- start start 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f163019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1630108450 0x7f163019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1630108da0 0x7f163019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1630112a60 con 0x7f1630102b10 2024-09-21T02:10:25.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16301128e0 con 0x7f1630108da0 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f1636b8e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1630112be0 con 0x7f1630108450 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.454+0000 7f163492a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f163019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f163019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34682/0 (socket says 172.21.15.92:34682) 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 -- 172.21.15.92:0/1545966490 learned_addr learned my addr 172.21.15.92:0/1545966490 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163512b640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1630108da0 0x7f163019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f162ffff640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1630108450 0x7f163019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 -- 172.21.15.92:0/1545966490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1630108450 msgr2=0x7f163019b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1630108450 0x7f163019b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 -- 172.21.15.92:0/1545966490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1630108da0 msgr2=0x7f163019be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1630108da0 0x7f163019be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 -- 172.21.15.92:0/1545966490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301a66a0 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f163492a640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f163019b410 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1628005940 tx=0x7f16280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f162dffb640 1 -- 172.21.15.92:0/1545966490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162803f030 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16301a6930 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f162dffb640 1 -- 172.21.15.92:0/1545966490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1628003cb0 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.455+0000 7f162dffb640 1 -- 172.21.15.92:0/1545966490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628035070 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.456+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301a6e60 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.458+0000 7f162dffb640 1 -- 172.21.15.92:0/1545966490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1628042030 con 0x7f1630102b10 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.458+0000 7f163d102740 1 Processor -- start 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.458+0000 7f163d102740 1 -- start start 2024-09-21T02:10:25.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163d102740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 0x55e6796842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163d102740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792b41b0 0x55e679684800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163d102740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792bc390 0x55e679684d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163d102740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e6795faea0 con 0x55e6792dd070 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163d102740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e6795ea2d0 con 0x55e6792bc390 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163d102740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e6795ee0e0 con 0x55e6792b41b0 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 0x55e6796842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 0x55e6796842c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34690/0 (socket says 172.21.15.92:34690) 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 -- 172.21.15.92:0/335019763 learned_addr learned my addr 172.21.15.92:0/335019763 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163492a640 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792b41b0 0x55e679684800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f162ffff640 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792bc390 0x55e679684d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f162dffb640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f161c0b65c0 0x7f161c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 -- 172.21.15.92:0/335019763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792b41b0 msgr2=0x55e679684800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792b41b0 0x55e679684800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 -- 172.21.15.92:0/335019763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792bc390 msgr2=0x55e679684d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792bc390 0x55e679684d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 -- 172.21.15.92:0/335019763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e679348650 con 0x55e6792dd070 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f162dffb640 1 -- 172.21.15.92:0/1545966490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f16281077f0 con 0x7f1630102b10 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f161c0b65c0 0x7f161c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.459+0000 7f163512b640 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 0x55e6796842c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f161800b6b0 tx=0x7f1618004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0504d5f6ee3a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.460+0000 7f1626ffd640 1 -- 172.21.15.92:0/335019763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618003d60 con 0x55e6792dd070 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.460+0000 7f1626ffd640 1 -- 172.21.15.92:0/335019763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1618015070 con 0x55e6792dd070 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.460+0000 7f1626ffd640 1 -- 172.21.15.92:0/335019763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161801bb00 con 0x55e6792dd070 2024-09-21T02:10:25.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 -- 172.21.15.92:0/335019763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 msgr2=0x55e6796842c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163512b640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f161c0b65c0 0x7f161c0b8ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f161801c000 tx=0x7f1618010810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 0x55e6796842c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f161800b6b0 tx=0x7f1618004610 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 -- 172.21.15.92:0/335019763 shutdown_connections 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792bc390 0x55e679684d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792b41b0 0x55e679684800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 --2- 172.21.15.92:0/335019763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792dd070 0x55e6796842c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 -- 172.21.15.92:0/335019763 >> 172.21.15.92:0/335019763 conn(0x55e6792d1460 msgr2=0x55e6796821b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 -- 172.21.15.92:0/335019763 shutdown_connections 2024-09-21T02:10:25.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 -- 172.21.15.92:0/335019763 wait complete. 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 -- start start 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.461+0000 7f163d102740 1 Processor -- start 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 0x55e67968f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792bc390 0x55e67968feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792dd070 0x55e6796903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e67944b700 con 0x55e6792b41b0 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e679585160 con 0x55e6792dd070 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e679692aa0 con 0x55e6792bc390 2024-09-21T02:10:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 10 client.0 _ll_register_callbacks cb 0x55e67959e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 0x55e67968f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 0x55e67968f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34704/0 (socket says 172.21.15.92:34704) 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 -- 172.21.15.92:0/4183627745 learned_addr learned my addr 172.21.15.92:0/4183627745 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f162ffff640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792bc390 0x55e67968feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163512b640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792dd070 0x55e6796903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 -- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792bc390 msgr2=0x55e67968feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792bc390 0x55e67968feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 -- 172.21.15.92:0/4183627745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792dd070 msgr2=0x55e6796903f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792dd070 0x55e6796903f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 -- 172.21.15.92:0/4183627745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6795b2c90 con 0x55e6792b41b0 2024-09-21T02:10:25.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163492a640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 0x55e67968f970 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1628006010 tx=0x7f16280045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7da58f9c0a81dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f160ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f160ffff640 1 -- 172.21.15.92:0/4183627745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162810d020 con 0x55e6792b41b0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f160ffff640 1 -- 172.21.15.92:0/4183627745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f162803bbf0 con 0x55e6792b41b0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.462+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1618015070 con 0x55e6792b41b0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f160ffff640 1 -- 172.21.15.92:0/4183627745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1628049070 con 0x55e6792b41b0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 10 client.17106 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e67969e700 con 0x55e6792b41b0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f160ffff640 1 -- 172.21.15.92:0/4183627745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1618015070 con 0x55e6792b41b0 2024-09-21T02:10:25.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f160ffff640 1 -- 172.21.15.92:0/4183627745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16280d2080 con 0x55e6792b41b0 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 10 client.17106 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 10 client.17106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 20 client.17106 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e67929b9f0 0x55e6796a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 4 client.17106 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6795fcdf0 con 0x55e67929b9f0 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.463+0000 7f162ffff640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e67929b9f0 0x55e6796a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.464+0000 7f162ffff640 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e67929b9f0 0x55e6796a1c40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6eca372180df17d server_cookie=72f29aabdf4877df in_seq=0 out_seq=0 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.464+0000 7f160ffff640 10 client.17106 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:10:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f160ffff640 1 -- 172.21.15.92:0/4183627745 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6795fcdf0 con 0x55e67929b9f0 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f160ffff640 10 client.17106 handle_command_reply: tid=0 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 client.17106 shutdown 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 msgr2=0x55e67968f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 0x55e67968f970 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1628006010 tx=0x7f16280045a0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 shutdown_connections 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e67929b9f0 0x55e6796a1c40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792dd070 0x55e6796903f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e6792bc390 0x55e67968feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f160ffff640 0 client.17106 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:10:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 --2- 172.21.15.92:0/4183627745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792b41b0 0x55e67968f970 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 >> 172.21.15.92:0/4183627745 conn(0x55e6792d1460 msgr2=0x55e6796874a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 shutdown_connections 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 1 -- 172.21.15.92:0/4183627745 wait complete. 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.465+0000 7f163d102740 20 client.17106 trim_cache size 0 max 16384 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 Processor -- start 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 -- start start 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679685cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e6796915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e679681760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16301af5f0 con 0x55e6792bc390 2024-09-21T02:10:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b1130 con 0x55e6792b41b0 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163d102740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f16301afcc0 con 0x55e67929b9f0 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163512b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e679681760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.478+0000 7f163512b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e679681760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34706/0 (socket says 172.21.15.92:34706) 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 -- 172.21.15.92:0/343517356 learned_addr learned my addr 172.21.15.92:0/343517356 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163492a640 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679685cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f162ffff640 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e6796915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 -- 172.21.15.92:0/343517356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 msgr2=0x55e679685cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679685cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 -- 172.21.15.92:0/343517356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 msgr2=0x55e6796915a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e6796915a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 -- 172.21.15.92:0/343517356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6796a2120 con 0x55e6792bc390 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f163512b640 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e679681760 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f16180036a0 tx=0x7f1618026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe750c81733fb914 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f16267fc640 1 -- 172.21.15.92:0/343517356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161802d050 con 0x55e6792bc390 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f16267fc640 1 -- 172.21.15.92:0/343517356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1618037070 con 0x55e6792bc390 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.479+0000 7f16267fc640 1 -- 172.21.15.92:0/343517356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618005040 con 0x55e6792bc390 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 -- 172.21.15.92:0/343517356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 msgr2=0x55e679681760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e679681760 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f16180036a0 tx=0x7f1618026000 comp rx=0 tx=0).stop 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 -- 172.21.15.92:0/343517356 shutdown_connections 2024-09-21T02:10:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e679681760 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e6796915a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 --2- 172.21.15.92:0/343517356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679685cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 -- 172.21.15.92:0/343517356 >> 172.21.15.92:0/343517356 conn(0x55e6792d1460 msgr2=0x55e679690e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 -- 172.21.15.92:0/343517356 shutdown_connections 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 -- 172.21.15.92:0/343517356 wait complete. 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 -- start start 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.480+0000 7f163d102740 1 Processor -- start 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679693e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e679694380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e6796948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b0a60 con 0x55e6792bc390 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b0390 con 0x55e6792b41b0 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f16301ad5d0 con 0x55e67929b9f0 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163d102740 10 client.0 _ll_register_callbacks cb 0x7f1628005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e6796948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e6796948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34722/0 (socket says 172.21.15.92:34722) 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 -- 172.21.15.92:0/345639514 learned_addr learned my addr 172.21.15.92:0/345639514 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163492a640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679693e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f162ffff640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e679694380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 -- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 msgr2=0x55e679693e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679693e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 -- 172.21.15.92:0/345639514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 msgr2=0x55e679694380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e679694380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163512b640 1 -- 172.21.15.92:0/345639514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e67970b730 con 0x55e6792bc390 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f162ffff640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e679694380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.481+0000 7f163492a640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679693e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163512b640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e6796948c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1618011f30 tx=0x7f1618011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefdfa97acaa0fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f160ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f160ffff640 1 -- 172.21.15.92:0/345639514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618003a00 con 0x55e6792bc390 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f160ffff640 1 -- 172.21.15.92:0/345639514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1618003ba0 con 0x55e6792bc390 2024-09-21T02:10:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1618037070 con 0x55e6792bc390 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f160ffff640 1 -- 172.21.15.92:0/345639514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f161804a070 con 0x55e6792bc390 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 10 client.17112 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e67970c990 con 0x55e6792bc390 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f160ffff640 1 -- 172.21.15.92:0/345639514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1618037070 con 0x55e6792bc390 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f160ffff640 1 -- 172.21.15.92:0/345639514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f161801b5a0 con 0x55e6792bc390 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 10 client.17112 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 10 client.17112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 20 client.17112 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.482+0000 7f163d102740 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e6792dd070 0x55e67970ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f163d102740 4 client.17112 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:10:25.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6795f88a0 con 0x55e6792dd070 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f163492a640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e6792dd070 0x55e67970ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f163492a640 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e6792dd070 0x55e67970ef50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d821ef1bd72de048 server_cookie=c92ac1f987a21d75 in_seq=0 out_seq=0 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f160ffff640 10 client.17112 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f160ffff640 1 -- 172.21.15.92:0/345639514 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6795f88a0 con 0x55e6792dd070 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f160ffff640 10 client.17112 handle_command_reply: tid=0 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.483+0000 7f163d102740 1 client.17112 shutdown 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 msgr2=0x55e6796948c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e6796948c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1618011f30 tx=0x7f1618011f60 comp rx=0 tx=0).stop 2024-09-21T02:10:25.505 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 shutdown_connections 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55e6792dd070 0x55e67970ef50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e6792bc390 0x55e6796948c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6792b41b0 0x55e679694380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 --2- 172.21.15.92:0/345639514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e67929b9f0 0x55e679693e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 >> 172.21.15.92:0/345639514 conn(0x55e6792d1460 msgr2=0x55e679685a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f160ffff640 0 client.17112 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 shutdown_connections 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 1 -- 172.21.15.92:0/345639514 wait complete. 2024-09-21T02:10:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f163d102740 20 client.17112 trim_cache size 0 max 16384 2024-09-21T02:10:25.507 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f161c0b65c0 msgr2=0x7f161c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f1636b8e640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f161c0b65c0 0x7f161c0b8ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f161801c000 tx=0x7f1618010810 comp rx=0 tx=0).stop 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 msgr2=0x7f163019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.484+0000 7f1636b8e640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f163019b410 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1628005940 tx=0x7f16280042f0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 shutdown_connections 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f161c0b65c0 0x7f161c0b8ad0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1630108da0 0x7f163019be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1630108450 0x7f163019b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 --2- 172.21.15.92:0/1545966490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1630102b10 0x7f163019b410 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 >> 172.21.15.92:0/1545966490 conn(0x7f16300796d0 msgr2=0x7f1630106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 shutdown_connections 2024-09-21T02:10:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:25.485+0000 7f1636b8e640 1 -- 172.21.15.92:0/1545966490 wait complete. 2024-09-21T02:10:26.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:10:26.472 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:26.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:26 smithi092 ceph-mon[31190]: pgmap v524: 97 pgs: 97 active+clean; 4.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:10:26.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:26 smithi099 ceph-mon[34404]: pgmap v524: 97 pgs: 97 active+clean; 4.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:10:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:26 smithi136 ceph-mon[34754]: pgmap v524: 97 pgs: 97 active+clean; 4.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:10:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 -- 172.21.15.92:0/1850539408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664109df0 msgr2=0x7f6664110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 --2- 172.21.15.92:0/1850539408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664109df0 0x7f6664110620 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f665c009ca0 tx=0x7f665c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 -- 172.21.15.92:0/1850539408 shutdown_connections 2024-09-21T02:10:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 --2- 172.21.15.92:0/1850539408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664109df0 0x7f6664110620 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 --2- 172.21.15.92:0/1850539408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66641094a0 0x7f66641098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 --2- 172.21.15.92:0/1850539408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6664103b60 0x7f6664103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 -- 172.21.15.92:0/1850539408 >> 172.21.15.92:0/1850539408 conn(0x7f666407b850 msgr2=0x7f666407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 -- 172.21.15.92:0/1850539408 shutdown_connections 2024-09-21T02:10:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 -- 172.21.15.92:0/1850539408 wait complete. 2024-09-21T02:10:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.961+0000 7f6669e37640 1 Processor -- start 2024-09-21T02:10:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 -- start start 2024-09-21T02:10:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 0x7f666419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66641094a0 0x7f666419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6664109df0 0x7f666419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6664113ab0 con 0x7f6664103b60 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6664113930 con 0x7f66641094a0 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6669e37640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6664113c30 con 0x7f6664109df0 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 0x7f666419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 0x7f666419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34748/0 (socket says 172.21.15.92:34748) 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 -- 172.21.15.92:0/606376834 learned_addr learned my addr 172.21.15.92:0/606376834 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6663fff640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6664109df0 0x7f666419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6662ffd640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66641094a0 0x7f666419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 -- 172.21.15.92:0/606376834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6664109df0 msgr2=0x7f666419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6664109df0 0x7f666419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 -- 172.21.15.92:0/606376834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66641094a0 msgr2=0x7f666419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66641094a0 0x7f666419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f66637fe640 1 -- 172.21.15.92:0/606376834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66641a0370 con 0x7f6664103b60 2024-09-21T02:10:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.962+0000 7f6662ffd640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66641094a0 0x7f666419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.963+0000 7f66637fe640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 0x7f666419a400 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f665400d950 tx=0x7f665400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.963+0000 7f6660ff9640 1 -- 172.21.15.92:0/606376834 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654017070 con 0x7f6664103b60 2024-09-21T02:10:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.963+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66641a0660 con 0x7f6664103b60 2024-09-21T02:10:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.963+0000 7f6660ff9640 1 -- 172.21.15.92:0/606376834 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6654009c30 con 0x7f6664103b60 2024-09-21T02:10:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.963+0000 7f6660ff9640 1 -- 172.21.15.92:0/606376834 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665400f620 con 0x7f6664103b60 2024-09-21T02:10:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.963+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66641a6dc0 con 0x7f6664103b60 2024-09-21T02:10:26.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.965+0000 7f66703ab740 1 Processor -- start 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.965+0000 7f66703ab740 1 -- start start 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66703ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c0d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6660ff9640 1 -- 172.21.15.92:0/606376834 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f665400f7c0 con 0x7f6664103b60 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66703ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66703ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66703ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563b4c04fea0 con 0x563b4bd11390 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66703ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563b4c03f2d0 con 0x563b4bd32070 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66703ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b4c0430e0 con 0x563b4bd091b0 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6662ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c0d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f66637fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6662ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0da120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34760/0 (socket says 172.21.15.92:34760) 2024-09-21T02:10:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6662ffd640 1 -- 172.21.15.92:0/16867526 learned_addr learned my addr 172.21.15.92:0/16867526 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 -- 172.21.15.92:0/16867526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 msgr2=0x563b4c0d9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0d9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 -- 172.21.15.92:0/16867526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 msgr2=0x563b4c0da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 -- 172.21.15.92:0/16867526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4bd9d650 con 0x563b4bd32070 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6662ffd640 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6663fff640 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c0d96a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f665c005cc0 tx=0x7f665c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e1c459f2c09a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.966+0000 7f6660ff9640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f664c0b6510 0x7f664c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.967+0000 7f66637fe640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f664c0b6510 0x7f664c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.967+0000 7f6649ffb640 1 -- 172.21.15.92:0/16867526 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c031cf0 con 0x563b4bd32070 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.967+0000 7f6660ff9640 1 -- 172.21.15.92:0/606376834 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f66540e7030 con 0x7f6664103b60 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.967+0000 7f6649ffb640 1 -- 172.21.15.92:0/16867526 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f665c036040 con 0x563b4bd32070 2024-09-21T02:10:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.967+0000 7f6649ffb640 1 -- 172.21.15.92:0/16867526 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0042a0 con 0x563b4bd32070 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 -- 172.21.15.92:0/16867526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 msgr2=0x563b4c0d96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66637fe640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f664c0b6510 0x7f664c0b8a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563b4c0d8320 tx=0x7f6654094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c0d96a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f665c005cc0 tx=0x7f665c009800 comp rx=0 tx=0).stop 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 -- 172.21.15.92:0/16867526 shutdown_connections 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 --2- 172.21.15.92:0/16867526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c0d96a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 -- 172.21.15.92:0/16867526 >> 172.21.15.92:0/16867526 conn(0x563b4bd26460 msgr2=0x563b4c0d7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 -- 172.21.15.92:0/16867526 shutdown_connections 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 -- 172.21.15.92:0/16867526 wait complete. 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 -- start start 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.968+0000 7f66703ab740 1 Processor -- start 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0e9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd32070 0x563b4c0f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563b4bea0700 con 0x563b4bd11390 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563b4bfda160 con 0x563b4bd091b0 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b4c0ec120 con 0x563b4bd32070 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f66703ab740 10 client.0 _ll_register_callbacks cb 0x563b4bff39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0e9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34772/0 (socket says 172.21.15.92:34772) 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 -- 172.21.15.92:0/3801597132 learned_addr learned my addr 172.21.15.92:0/3801597132 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6663fff640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6663fff640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd32070 0x563b4c0f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 -- 172.21.15.92:0/3801597132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd32070 msgr2=0x563b4c0f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd32070 0x563b4c0f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 -- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 msgr2=0x563b4c0e9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0e9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 -- 172.21.15.92:0/3801597132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4c007c90 con 0x563b4bd11390 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6663fff640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd32070 0x563b4c0f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6662ffd640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0e9680 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f66580113e0 tx=0x7f66580118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11f9aaddf4c832c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.969+0000 7f6642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f6642ffd640 1 -- 172.21.15.92:0/3801597132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658012980 con 0x563b4bd11390 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c036040 con 0x563b4bd11390 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f6642ffd640 1 -- 172.21.15.92:0/3801597132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6658012b20 con 0x563b4bd11390 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f6642ffd640 1 -- 172.21.15.92:0/3801597132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6658012d20 con 0x563b4bd11390 2024-09-21T02:10:26.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 10 client.17124 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b4c0f7ed0 con 0x563b4bd11390 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f6642ffd640 1 -- 172.21.15.92:0/3801597132 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f665c036040 con 0x563b4bd11390 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f6642ffd640 1 -- 172.21.15.92:0/3801597132 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f665800f040 con 0x563b4bd11390 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 10 client.17124 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 10 client.17124 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 20 client.17124 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcf09f0 0x563b4c0fb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 4 client.17124 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.970+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b4c051df0 con 0x563b4bcf09f0 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.971+0000 7f6663fff640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcf09f0 0x563b4c0fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.971+0000 7f6663fff640 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcf09f0 0x563b4c0fb400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65f588971de0717a server_cookie=f2f70182a13002a in_seq=0 out_seq=0 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.971+0000 7f6642ffd640 10 client.17124 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f6642ffd640 1 -- 172.21.15.92:0/3801597132 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b4c051df0 con 0x563b4bcf09f0 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f6642ffd640 10 client.17124 handle_command_reply: tid=0 2024-09-21T02:10:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 client.17124 shutdown 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 msgr2=0x563b4c0e9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0e9680 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f66580113e0 tx=0x7f66580118a0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 shutdown_connections 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd32070 0x563b4c0f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcf09f0 0x563b4c0fb400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0e9680 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 --2- 172.21.15.92:0/3801597132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0e9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f6642ffd640 0 client.17124 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 >> 172.21.15.92:0/3801597132 conn(0x563b4bd26460 msgr2=0x563b4c0d5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 shutdown_connections 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 1 -- 172.21.15.92:0/3801597132 wait complete. 2024-09-21T02:10:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.972+0000 7f66703ab740 20 client.17124 trim_cache size 0 max 16384 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 Processor -- start 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 -- start start 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd11390 0x563b4c0e6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 0x563b4c0e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66641ad200 con 0x563b4bd32070 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66641aed40 con 0x563b4bd091b0 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.985+0000 7f66703ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f66641ad8d0 con 0x563b4bd11390 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 0x563b4c0e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 0x563b4c0e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34780/0 (socket says 172.21.15.92:34780) 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 -- 172.21.15.92:0/1628121116 learned_addr learned my addr 172.21.15.92:0/1628121116 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6662ffd640 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd11390 0x563b4c0e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 -- 172.21.15.92:0/1628121116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd11390 msgr2=0x563b4c0e6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd11390 0x563b4c0e6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 -- 172.21.15.92:0/1628121116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 msgr2=0x563b4c0db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0db090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 -- 172.21.15.92:0/1628121116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4c0ebc70 con 0x563b4bd32070 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f6663fff640 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 0x563b4c0e6870 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f665c009a60 tx=0x7f665c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2119a8ebb31424e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f66497fa640 1 -- 172.21.15.92:0/1628121116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c04a040 con 0x563b4bd32070 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f66497fa640 1 -- 172.21.15.92:0/1628121116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f665c036030 con 0x563b4bd32070 2024-09-21T02:10:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.986+0000 7f66497fa640 1 -- 172.21.15.92:0/1628121116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0043c0 con 0x563b4bd32070 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 -- 172.21.15.92:0/1628121116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 msgr2=0x563b4c0e6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 0x563b4c0e6870 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f665c009a60 tx=0x7f665c005c30 comp rx=0 tx=0).stop 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 -- 172.21.15.92:0/1628121116 shutdown_connections 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd32070 0x563b4c0e6870 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd11390 0x563b4c0e6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 --2- 172.21.15.92:0/1628121116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd091b0 0x563b4c0db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 -- 172.21.15.92:0/1628121116 >> 172.21.15.92:0/1628121116 conn(0x563b4bd26460 msgr2=0x563b4c0dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 -- 172.21.15.92:0/1628121116 shutdown_connections 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 -- 172.21.15.92:0/1628121116 wait complete. 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.987+0000 7f66703ab740 1 -- start start 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 Processor -- start 2024-09-21T02:10:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0ef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66641ae670 con 0x563b4bd11390 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66641adfa0 con 0x563b4bd32070 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f66641ab1e0 con 0x563b4bd091b0 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f66703ab740 10 client.0 _ll_register_callbacks cb 0x7f6658009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6662ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6662ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0efd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34784/0 (socket says 172.21.15.92:34784) 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6662ffd640 1 -- 172.21.15.92:0/1920033215 learned_addr learned my addr 172.21.15.92:0/1920033215 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6663fff640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6663fff640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6662ffd640 1 -- 172.21.15.92:0/1920033215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 msgr2=0x563b4c0ef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6662ffd640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0ef810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.988+0000 7f6662ffd640 1 -- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 msgr2=0x563b4c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6662ffd640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6662ffd640 1 -- 172.21.15.92:0/1920033215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4c0dc4d0 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6663fff640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6662ffd640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0efd50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6658000c00 tx=0x7f66580052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e77e8c94f7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6642ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6642ffd640 1 -- 172.21.15.92:0/1920033215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665802b070 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6642ffd640 1 -- 172.21.15.92:0/1920033215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f665801a9d0 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c036030 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6642ffd640 1 -- 172.21.15.92:0/1920033215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6658011da0 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f66703ab740 10 client.17130 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b4c166230 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6642ffd640 1 -- 172.21.15.92:0/1920033215 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f665c036030 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.989+0000 7f6642ffd640 1 -- 172.21.15.92:0/1920033215 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6658004c50 con 0x563b4bd11390 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 10 client.17130 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 20 client.17130 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcc5da0 0x563b4c1689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 4 client.17130 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b4c04d8a0 con 0x563b4bcc5da0 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f6663fff640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcc5da0 0x563b4c1689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f6663fff640 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcc5da0 0x563b4c1689c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=434e1c38ed065d77 server_cookie=1b835929aa98a204 in_seq=0 out_seq=0 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f6642ffd640 10 client.17130 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f6642ffd640 1 -- 172.21.15.92:0/1920033215 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b4c04d8a0 con 0x563b4bcc5da0 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f6642ffd640 10 client.17130 handle_command_reply: tid=0 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.990+0000 7f66703ab740 1 client.17130 shutdown 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 msgr2=0x563b4c0efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0efd50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6658000c00 tx=0x7f66580052f0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 shutdown_connections 2024-09-21T02:10:26.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b4bd32070 0x563b4c164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563b4bcc5da0 0x563b4c1689c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563b4bd11390 0x563b4c0efd50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 --2- 172.21.15.92:0/1920033215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b4bd091b0 0x563b4c0ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 >> 172.21.15.92:0/1920033215 conn(0x563b4bd26460 msgr2=0x563b4c0ea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6642ffd640 0 client.17130 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 shutdown_connections 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 1 -- 172.21.15.92:0/1920033215 wait complete. 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f66703ab740 20 client.17130 trim_cache size 0 max 16384 2024-09-21T02:10:26.996 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f664c0b6510 msgr2=0x7f664c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f664c0b6510 0x7f664c0b8a20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563b4c0d8320 tx=0x7f6654094040 comp rx=0 tx=0).stop 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 msgr2=0x7f666419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 0x7f666419a400 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f665400d950 tx=0x7f665400de10 comp rx=0 tx=0).stop 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 shutdown_connections 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f664c0b6510 0x7f664c0b8a20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6664109df0 0x7f666419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66641094a0 0x7f666419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 --2- 172.21.15.92:0/606376834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6664103b60 0x7f666419a400 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.991+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 >> 172.21.15.92:0/606376834 conn(0x7f666407b850 msgr2=0x7f6664107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.992+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 shutdown_connections 2024-09-21T02:10:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:26.992+0000 7f6669e37640 1 -- 172.21.15.92:0/606376834 wait complete. 2024-09-21T02:10:27.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:10:28.022 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:28.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/210547555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 msgr2=0x7fcd1410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/210547555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1410f4a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009ca0 tx=0x7fcd0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/210547555 shutdown_connections 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/210547555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1410f4a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/210547555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd14102fa0 0x7fcd141033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/210547555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd14107630 0x7fcd14107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/210547555 >> 172.21.15.92:0/210547555 conn(0x7fcd14079e40 msgr2=0x7fcd1407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/210547555 shutdown_connections 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.493+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/210547555 wait complete. 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 Processor -- start 2024-09-21T02:10:28.513 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 -- start start 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd14102fa0 0x7fcd1419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd14107630 0x7fcd1419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14113b00 con 0x7fcd141038f0 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14113980 con 0x7fcd14102fa0 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.494+0000 7fcd1b5dd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14113c80 con 0x7fcd14107630 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34790/0 (socket says 172.21.15.92:34790) 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 -- 172.21.15.92:0/1191159720 learned_addr learned my addr 172.21.15.92:0/1191159720 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd14107630 0x7fcd1419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd19379640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd14102fa0 0x7fcd1419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 -- 172.21.15.92:0/1191159720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd14107630 msgr2=0x7fcd1419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd14107630 0x7fcd1419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 -- 172.21.15.92:0/1191159720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd14102fa0 msgr2=0x7fcd1419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd14102fa0 0x7fcd1419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 -- 172.21.15.92:0/1191159720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd141a03a0 con 0x7fcd141038f0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1419a9d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000d740 tx=0x7fcd1000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd067fc640 1 -- 172.21.15.92:0/1191159720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10017070 con 0x7fcd141038f0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd141a0690 con 0x7fcd141038f0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd067fc640 1 -- 172.21.15.92:0/1191159720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd1000de50 con 0x7fcd141038f0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd067fc640 1 -- 172.21.15.92:0/1191159720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10003c30 con 0x7fcd141038f0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.495+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd141a6df0 con 0x7fcd141038f0 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.498+0000 7fcd21b51740 1 Processor -- start 2024-09-21T02:10:28.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.498+0000 7fcd21b51740 1 -- start start 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd067fc640 1 -- 172.21.15.92:0/1191159720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fcd10003dd0 con 0x7fcd141038f0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf100be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf101120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd21b51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626cf076ea0 con 0x5626ced301b0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd21b51740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5626cf0662d0 con 0x5626ced59070 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd21b51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5626cf06a0e0 con 0x5626ced38390 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34802/0 (socket says 172.21.15.92:34802) 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 -- 172.21.15.92:0/491838355 learned_addr learned my addr 172.21.15.92:0/491838355 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd18b78640 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 -- 172.21.15.92:0/491838355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 msgr2=0x5626cf101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 -- 172.21.15.92:0/491838355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 msgr2=0x5626cf1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd19379640 1 -- 172.21.15.92:0/491838355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626cedc4650 con 0x5626ced301b0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.499+0000 7fcd18b78640 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fcd19379640 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf100be0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800bcb0 tx=0x7fcd0800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29b4f891a9c66f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fccff7fe640 1 -- 172.21.15.92:0/491838355 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08015070 con 0x5626ced301b0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fcd067fc640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcd000b6510 0x7fcd000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fccff7fe640 1 -- 172.21.15.92:0/491838355 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd0801a910 con 0x5626ced301b0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fccff7fe640 1 -- 172.21.15.92:0/491838355 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080118b0 con 0x5626ced301b0 2024-09-21T02:10:28.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcd000b6510 0x7fcd000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.500+0000 7fcd067fc640 1 -- 172.21.15.92:0/1191159720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcd100e6de0 con 0x7fcd141038f0 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 -- 172.21.15.92:0/491838355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 msgr2=0x5626cf100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcd000b6510 0x7fcd000b8a20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c005e30 tx=0x7fcd0c005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf100be0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800bcb0 tx=0x7fcd0800f730 comp rx=0 tx=0).stop 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 -- 172.21.15.92:0/491838355 shutdown_connections 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf100be0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 --2- 172.21.15.92:0/491838355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 -- 172.21.15.92:0/491838355 >> 172.21.15.92:0/491838355 conn(0x5626ced4d460 msgr2=0x5626cf0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 -- 172.21.15.92:0/491838355 shutdown_connections 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 -- 172.21.15.92:0/491838355 wait complete. 2024-09-21T02:10:28.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 -- start start 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.501+0000 7fcd21b51740 1 Processor -- start 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf10bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced38390 0x5626cf10c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced59070 0x5626cf1181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626ceec7700 con 0x5626ced301b0 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5626cf001160 con 0x5626ced38390 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5626cf10ee80 con 0x5626ced59070 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd21b51740 10 client.0 _ll_register_callbacks cb 0x5626cf01a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf10bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf10bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34816/0 (socket says 172.21.15.92:34816) 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 -- 172.21.15.92:0/1878577773 learned_addr learned my addr 172.21.15.92:0/1878577773 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced38390 0x5626cf10c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced59070 0x5626cf1181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 -- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced59070 msgr2=0x5626cf1181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced59070 0x5626cf1181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 -- 172.21.15.92:0/1878577773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced38390 msgr2=0x5626cf10c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced38390 0x5626cf10c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 -- 172.21.15.92:0/1878577773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cf02ec90 con 0x5626ced301b0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced38390 0x5626cf10c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fcd19379640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf10bd70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000c00 tx=0x7fcd0800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f92525f607ecb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fccfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fccfcff9640 1 -- 172.21.15.92:0/1878577773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08028060 con 0x5626ced301b0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.502+0000 7fccfcff9640 1 -- 172.21.15.92:0/1878577773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcd0801aac0 con 0x5626ced301b0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0801a910 con 0x5626ced301b0 2024-09-21T02:10:28.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fccfcff9640 1 -- 172.21.15.92:0/1878577773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd08015070 con 0x5626ced301b0 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 10 client.17142 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626cf11ab20 con 0x5626ced301b0 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fccfcff9640 1 -- 172.21.15.92:0/1878577773 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd0801a910 con 0x5626ced301b0 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fccfcff9640 1 -- 172.21.15.92:0/1878577773 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd08004850 con 0x5626ced301b0 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 10 client.17142 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 20 client.17142 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626ced179f0 0x5626cf11df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 4 client.17142 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626cf078df0 con 0x5626ced179f0 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.503+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626ced179f0 0x5626cf11df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.504+0000 7fccfcff9640 10 client.17142 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.504+0000 7fcd18b78640 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626ced179f0 0x5626cf11df30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63c56954dc2f3f5 server_cookie=e6ac2cb148a73ce9 in_seq=0 out_seq=0 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fccfcff9640 1 -- 172.21.15.92:0/1878577773 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5626cf078df0 con 0x5626ced179f0 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fccfcff9640 10 client.17142 handle_command_reply: tid=0 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 client.17142 shutdown 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 msgr2=0x5626cf10bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf10bd70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000c00 tx=0x7fcd0800bfc0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 shutdown_connections 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626ced179f0 0x5626cf11df30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced59070 0x5626cf1181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced38390 0x5626cf10c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 --2- 172.21.15.92:0/1878577773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced301b0 0x5626cf10bd70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 >> 172.21.15.92:0/1878577773 conn(0x5626ced4d460 msgr2=0x5626cf103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fccfcff9640 0 client.17142 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:28.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 shutdown_connections 2024-09-21T02:10:28.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 1 -- 172.21.15.92:0/1878577773 wait complete. 2024-09-21T02:10:28.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.505+0000 7fcd21b51740 20 client.17142 trim_cache size 0 max 16384 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 Processor -- start 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 -- start start 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced301b0 0x5626cf10ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 0x5626cf101de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf118140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141af520 con 0x5626ced38390 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141b1060 con 0x5626ced59070 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd21b51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141afbf0 con 0x5626ced301b0 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 0x5626cf101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 0x5626cf101de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34826/0 (socket says 172.21.15.92:34826) 2024-09-21T02:10:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 -- 172.21.15.92:0/3837152619 learned_addr learned my addr 172.21.15.92:0/3837152619 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd19379640 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced301b0 0x5626cf10ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf118140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 -- 172.21.15.92:0/3837152619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced301b0 msgr2=0x5626cf10ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced301b0 0x5626cf10ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 -- 172.21.15.92:0/3837152619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 msgr2=0x5626cf118140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf118140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd18b78640 1 -- 172.21.15.92:0/3837152619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ced17c20 con 0x5626ced38390 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.519+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf118140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.520+0000 7fcd18b78640 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 0x5626cf101de0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009610 tx=0x7fcd100e4790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8dbea5278934f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.520+0000 7fccfeffd640 1 -- 172.21.15.92:0/3837152619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10020040 con 0x5626ced38390 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.520+0000 7fccfeffd640 1 -- 172.21.15.92:0/3837152619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd100b1a30 con 0x5626ced38390 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.520+0000 7fccfeffd640 1 -- 172.21.15.92:0/3837152619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100a9350 con 0x5626ced38390 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 -- 172.21.15.92:0/3837152619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 msgr2=0x5626cf101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 0x5626cf101de0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009610 tx=0x7fcd100e4790 comp rx=0 tx=0).stop 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 -- 172.21.15.92:0/3837152619 shutdown_connections 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced59070 0x5626cf118140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced38390 0x5626cf101de0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 --2- 172.21.15.92:0/3837152619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced301b0 0x5626cf10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 -- 172.21.15.92:0/3837152619 >> 172.21.15.92:0/3837152619 conn(0x5626ced4d460 msgr2=0x5626cf1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 -- 172.21.15.92:0/3837152619 shutdown_connections 2024-09-21T02:10:28.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 -- 172.21.15.92:0/3837152619 wait complete. 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 -- start start 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.521+0000 7fcd21b51740 1 Processor -- start 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced301b0 0x5626cf121930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf11ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 0x5626cf11f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141b0990 con 0x5626ced59070 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141b02c0 con 0x5626ced301b0 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcd141ad500 con 0x5626ced38390 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd21b51740 10 client.0 _ll_register_callbacks cb 0x7fcd0c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 0x5626cf11f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 0x5626cf11f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34840/0 (socket says 172.21.15.92:34840) 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 -- 172.21.15.92:0/4179363960 learned_addr learned my addr 172.21.15.92:0/4179363960 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19379640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced301b0 0x5626cf121930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd18b78640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf11ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 -- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 msgr2=0x5626cf11ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf11ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 -- 172.21.15.92:0/4179363960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced301b0 msgr2=0x5626cf121930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced301b0 0x5626cf121930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 -- 172.21.15.92:0/4179363960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cf18bac0 con 0x5626ced59070 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fcd19b7a640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 0x5626cf11f530 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c031fd0 tx=0x7fcd0c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5cce232e5346c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.522+0000 7fccfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:28.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fccfcff9640 1 -- 172.21.15.92:0/4179363960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c02fa60 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fccfcff9640 1 -- 172.21.15.92:0/4179363960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcd0c042030 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd100b1a30 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fccfcff9640 1 -- 172.21.15.92:0/4179363960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd0c02fc00 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 10 client.17148 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626cf18cb60 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fccfcff9640 1 -- 172.21.15.92:0/4179363960 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd100b1a30 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fccfcff9640 1 -- 172.21.15.92:0/4179363960 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd0c053420 con 0x5626ced59070 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 10 client.17148 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 20 client.17148 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626cececda0 0x5626cf18f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:28.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 4 client.17148 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.523+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626cf0748a0 con 0x5626cececda0 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.524+0000 7fcd19379640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626cececda0 0x5626cf18f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.524+0000 7fcd19379640 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626cececda0 0x5626cf18f350 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9babc9a2e7d91d57 server_cookie=4abe27e06960bba9 in_seq=0 out_seq=0 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.524+0000 7fccfcff9640 10 client.17148 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.524+0000 7fccfcff9640 1 -- 172.21.15.92:0/4179363960 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626cf0748a0 con 0x5626cececda0 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.524+0000 7fccfcff9640 10 client.17148 handle_command_reply: tid=0 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.524+0000 7fcd21b51740 1 client.17148 shutdown 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 msgr2=0x5626cf11f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 0x5626cf11f530 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c031fd0 tx=0x7fcd0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 shutdown_connections 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5626cececda0 0x5626cf18f350 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626ced59070 0x5626cf11f530 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626ced38390 0x5626cf11ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fccfcff9640 0 client.17148 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 --2- 172.21.15.92:0/4179363960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626ced301b0 0x5626cf121930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 >> 172.21.15.92:0/4179363960 conn(0x5626ced4d460 msgr2=0x5626cf10d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 shutdown_connections 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 1 -- 172.21.15.92:0/4179363960 wait complete. 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd21b51740 20 client.17148 trim_cache size 0 max 16384 2024-09-21T02:10:28.536 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcd000b6510 msgr2=0x7fcd000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcd000b6510 0x7fcd000b8a20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c005e30 tx=0x7fcd0c005c30 comp rx=0 tx=0).stop 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 msgr2=0x7fcd1419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1419a9d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000d740 tx=0x7fcd1000dc00 comp rx=0 tx=0).stop 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 shutdown_connections 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcd000b6510 0x7fcd000b8a20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd14107630 0x7fcd1419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd141038f0 0x7fcd1419a9d0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 --2- 172.21.15.92:0/1191159720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd14102fa0 0x7fcd1419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:28.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.525+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 >> 172.21.15.92:0/1191159720 conn(0x7fcd14079e40 msgr2=0x7fcd14104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:28.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.526+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 shutdown_connections 2024-09-21T02:10:28.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:28.526+0000 7fcd1b5dd640 1 -- 172.21.15.92:0/1191159720 wait complete. 2024-09-21T02:10:28.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:28 smithi136 ceph-mon[34754]: pgmap v525: 97 pgs: 97 active+clean; 4.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:28.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:28 smithi092 ceph-mon[31190]: pgmap v525: 97 pgs: 97 active+clean; 4.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:28 smithi099 ceph-mon[34404]: pgmap v525: 97 pgs: 97 active+clean; 4.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:30.247 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:10:30.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:10:30.511 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:30.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:30 smithi092 ceph-mon[31190]: pgmap v526: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:30.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:30 smithi136 ceph-mon[34754]: pgmap v526: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:30.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:30 smithi099 ceph-mon[34404]: pgmap v526: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 -- 172.21.15.92:0/3043146254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4105310 msgr2=0x7f0ae410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3043146254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4105310 0x7f0ae410d860 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009860 tx=0x7f0ae002a340 comp rx=0 tx=0).stop 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 -- 172.21.15.92:0/3043146254 shutdown_connections 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3043146254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae410de20 0x7f0ae4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3043146254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae4105310 0x7f0ae410d860 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3043146254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ae41049c0 0x7f0ae4104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 -- 172.21.15.92:0/3043146254 >> 172.21.15.92:0/3043146254 conn(0x7f0ae4079710 msgr2=0x7f0ae4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 -- 172.21.15.92:0/3043146254 shutdown_connections 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.988+0000 7f0aeb488640 1 -- 172.21.15.92:0/3043146254 wait complete. 2024-09-21T02:10:30.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.989+0000 7f0aeb488640 1 Processor -- start 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.989+0000 7f0aeb488640 1 -- start start 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.989+0000 7f0aeb488640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 0x7f0ae419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0aeb488640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ae4105310 0x7f0ae419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0aeb488640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae410de20 0x7f0ae419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0aeb488640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4113a40 con 0x7f0ae410de20 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0aeb488640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41138c0 con 0x7f0ae41049c0 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0aeb488640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4113bc0 con 0x7f0ae4105310 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9224640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 0x7f0ae419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9224640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 0x7f0ae419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:50658/0 (socket says 172.21.15.92:50658) 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9224640 1 -- 172.21.15.92:0/3116087184 learned_addr learned my addr 172.21.15.92:0/3116087184 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ae4105310 0x7f0ae419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9a25640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae410de20 0x7f0ae419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9a25640 1 -- 172.21.15.92:0/3116087184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ae4105310 msgr2=0x7f0ae419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9a25640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ae4105310 0x7f0ae419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9a25640 1 -- 172.21.15.92:0/3116087184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 msgr2=0x7f0ae419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9a25640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 0x7f0ae419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9a25640 1 -- 172.21.15.92:0/3116087184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae41a03e0 con 0x7f0ae410de20 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.990+0000 7f0ae9224640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 0x7f0ae419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.991+0000 7f0ae9a25640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae410de20 0x7f0ae419fcb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009fd0 tx=0x7f0adc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.991+0000 7f0ad67fc640 1 -- 172.21.15.92:0/3116087184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00ce20 con 0x7f0ae410de20 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.991+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae41a06d0 con 0x7f0ae410de20 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.991+0000 7f0ad67fc640 1 -- 172.21.15.92:0/3116087184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc005020 con 0x7f0ae410de20 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.991+0000 7f0ad67fc640 1 -- 172.21.15.92:0/3116087184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc010690 con 0x7f0ae410de20 2024-09-21T02:10:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.991+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41a6e30 con 0x7f0ae410de20 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 Processor -- start 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0ad67fc640 1 -- 172.21.15.92:0/3116087184 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f0adc010830 con 0x7f0ae410de20 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 -- start start 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdf3350 0x55e76019a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e76019a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e760110e60 con 0x55e75fdf3350 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e760100290 con 0x55e75fdca490 2024-09-21T02:10:30.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.994+0000 7f0af19fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e7601040a0 con 0x55e75fdd2670 2024-09-21T02:10:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e76019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae8a23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdf3350 0x55e76019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae8a23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdf3350 0x55e76019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34874/0 (socket says 172.21.15.92:34874) 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e76019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:50660/0 (socket says 172.21.15.92:50660) 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae8a23640 1 -- 172.21.15.92:0/3038093716 learned_addr learned my addr 172.21.15.92:0/3038093716 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 -- 172.21.15.92:0/3038093716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 msgr2=0x55e76019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 -- 172.21.15.92:0/3038093716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdf3350 msgr2=0x55e76019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdf3350 0x55e76019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ad67fc640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0ad00b6510 0x7f0ad00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 -- 172.21.15.92:0/3038093716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75fe5e930 con 0x55e75fdca490 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9a25640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0ad00b6510 0x7f0ad00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ae9224640 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e76019a970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8011510 tx=0x7f0ad80119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b106645871350ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0ad67fc640 1 -- 172.21.15.92:0/3116087184 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0adc0e79c0 con 0x7f0ae410de20 2024-09-21T02:10:30.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.995+0000 7f0acf7fe640 1 -- 172.21.15.92:0/3038093716 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad801e070 con 0x55e75fdca490 2024-09-21T02:10:30.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.996+0000 7f0acf7fe640 1 -- 172.21.15.92:0/3038093716 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80035f0 con 0x55e75fdca490 2024-09-21T02:10:30.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.996+0000 7f0acf7fe640 1 -- 172.21.15.92:0/3038093716 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800f7a0 con 0x55e75fdca490 2024-09-21T02:10:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.996+0000 7f0af19fc740 1 -- 172.21.15.92:0/3038093716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 msgr2=0x55e76019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.996+0000 7f0ae9a25640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0ad00b6510 0x7f0ad00b8a20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0adac0 tx=0x7f0adc095040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.996+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e76019a970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8011510 tx=0x7f0ad80119d0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.996+0000 7f0af19fc740 1 -- 172.21.15.92:0/3038093716 shutdown_connections 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e76019a970 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3038093716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdf3350 0x55e76019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- 172.21.15.92:0/3038093716 >> 172.21.15.92:0/3038093716 conn(0x55e75fde7740 msgr2=0x55e760198320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- 172.21.15.92:0/3038093716 shutdown_connections 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- 172.21.15.92:0/3038093716 wait complete. 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- start start 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 Processor -- start 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdca490 0x55e7601a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601a6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdf3350 0x55e7601b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e75ff61890 con 0x55e75fdd2670 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e76009ad40 con 0x55e75fdf3350 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.997+0000 7f0af19fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e7601ad020 con 0x55e75fdca490 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0af19fc740 10 client.0 _ll_register_callbacks cb 0x55e7600fec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601a6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34890/0 (socket says 172.21.15.92:34890) 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 -- 172.21.15.92:0/1247328412 learned_addr learned my addr 172.21.15.92:0/1247328412 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdf3350 0x55e7601b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdca490 0x55e7601a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 -- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdca490 msgr2=0x55e7601a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdca490 0x55e7601a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 -- 172.21.15.92:0/1247328412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdf3350 msgr2=0x55e7601b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdf3350 0x55e7601b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 -- 172.21.15.92:0/1247328412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7601b6950 con 0x55e75fdd2670 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0ae9224640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601a6600 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009710 tx=0x7f0ad80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d630e13aed8bac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0accff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0accff9640 1 -- 172.21.15.92:0/1247328412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad801e040 con 0x55e75fdd2670 2024-09-21T02:10:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad80035f0 con 0x55e75fdd2670 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0accff9640 1 -- 172.21.15.92:0/1247328412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0ad8005040 con 0x55e75fdd2670 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0accff9640 1 -- 172.21.15.92:0/1247328412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ad8009cd0 con 0x55e75fdd2670 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0af19fc740 10 client.17160 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.998+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7601b9280 con 0x55e75fdd2670 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0accff9640 1 -- 172.21.15.92:0/1247328412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80035f0 con 0x55e75fdd2670 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0accff9640 1 -- 172.21.15.92:0/1247328412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad8004b90 con 0x55e75fdd2670 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0af19fc740 10 client.17160 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0af19fc740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0af19fc740 20 client.17160 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0af19fc740 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdb1cd0 0x55e7601bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0af19fc740 4 client.17160 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e760112db0 con 0x55e75fdb1cd0 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:30.999+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdb1cd0 0x55e7601bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.000+0000 7f0accff9640 10 client.17160 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:31.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.000+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdb1cd0 0x55e7601bc640 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2405d544e5d25b8c server_cookie=573fb7a5d3666814 in_seq=0 out_seq=0 2024-09-21T02:10:31.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0accff9640 1 -- 172.21.15.92:0/1247328412 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e760112db0 con 0x55e75fdb1cd0 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0accff9640 10 client.17160 handle_command_reply: tid=0 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 client.17160 shutdown 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 msgr2=0x55e7601a6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601a6600 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009710 tx=0x7f0ad80039d0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 shutdown_connections 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdb1cd0 0x55e7601bc640 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdf3350 0x55e7601b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601a6600 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 --2- 172.21.15.92:0/1247328412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdca490 0x55e7601a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0accff9640 0 client.17160 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 >> 172.21.15.92:0/1247328412 conn(0x55e75fde7740 msgr2=0x55e76019d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 shutdown_connections 2024-09-21T02:10:31.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 1 -- 172.21.15.92:0/1247328412 wait complete. 2024-09-21T02:10:31.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.001+0000 7f0af19fc740 20 client.17160 trim_cache size 0 max 16384 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 Processor -- start 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 -- start start 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 0x55e7601a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41ad210 con 0x55e75fdb1cd0 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41aed50 con 0x55e75fdca490 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.018+0000 7f0af19fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41ad8e0 con 0x55e75fdd2670 2024-09-21T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 0x55e7601a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 0x55e7601a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34896/0 (socket says 172.21.15.92:34896) 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 -- 172.21.15.92:0/3981977011 learned_addr learned my addr 172.21.15.92:0/3981977011 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae9224640 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 -- 172.21.15.92:0/3981977011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 msgr2=0x55e76019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 -- 172.21.15.92:0/3981977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 msgr2=0x55e7601a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601a7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 -- 172.21.15.92:0/3981977011 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e760198860 con 0x55e75fdb1cd0 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 0x55e7601a55e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0030db0 tx=0x7f0ae002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db001c7e5cba4fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0aceffd640 1 -- 172.21.15.92:0/3981977011 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae004f030 con 0x55e75fdb1cd0 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0aceffd640 1 -- 172.21.15.92:0/3981977011 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae0030900 con 0x55e75fdb1cd0 2024-09-21T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.019+0000 7f0aceffd640 1 -- 172.21.15.92:0/3981977011 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0049070 con 0x55e75fdb1cd0 2024-09-21T02:10:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 -- 172.21.15.92:0/3981977011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 msgr2=0x55e7601a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 0x55e7601a55e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0030db0 tx=0x7f0ae002a340 comp rx=0 tx=0).stop 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 -- 172.21.15.92:0/3981977011 shutdown_connections 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdd2670 0x55e76019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 --2- 172.21.15.92:0/3981977011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdb1cd0 0x55e7601a55e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 -- 172.21.15.92:0/3981977011 >> 172.21.15.92:0/3981977011 conn(0x55e75fde7740 msgr2=0x55e7601a73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 -- 172.21.15.92:0/3981977011 shutdown_connections 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 -- 172.21.15.92:0/3981977011 wait complete. 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.020+0000 7f0af19fc740 1 -- start start 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 Processor -- start 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdb1cd0 0x55e7601b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41ae680 con 0x55e75fdd2670 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41adfb0 con 0x55e75fdca490 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41ab1f0 con 0x55e75fdb1cd0 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0af19fc740 10 client.0 _ll_register_callbacks cb 0x7f0ad801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0ae8a23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0ae8a23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34912/0 (socket says 172.21.15.92:34912) 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0ae8a23640 1 -- 172.21.15.92:0/173158303 learned_addr learned my addr 172.21.15.92:0/173158303 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0ae9224640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.021+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdb1cd0 0x55e7601b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae8a23640 1 -- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdb1cd0 msgr2=0x55e7601b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdb1cd0 0x55e7601b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae8a23640 1 -- 172.21.15.92:0/173158303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 msgr2=0x55e7601b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae8a23640 1 -- 172.21.15.92:0/173158303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7601b1d60 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae9224640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0ae8a23640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601b1820 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005c40 tx=0x7f0ae0004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fd313b816401a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0accff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0accff9640 1 -- 172.21.15.92:0/173158303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae004f040 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0accff9640 1 -- 172.21.15.92:0/173158303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0ae00037f0 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ae0030900 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0accff9640 1 -- 172.21.15.92:0/173158303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ae0037040 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0af19fc740 10 client.17166 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e760227ba0 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.022+0000 7f0accff9640 1 -- 172.21.15.92:0/173158303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae0030900 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0accff9640 1 -- 172.21.15.92:0/173158303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ae003d040 con 0x55e75fdd2670 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0af19fc740 10 client.17166 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0af19fc740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:31.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0af19fc740 20 client.17166 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:31.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0af19fc740 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdf3350 0x55e76022a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:31.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0af19fc740 4 client.17166 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:10:31.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e76010e860 con 0x55e75fdf3350 2024-09-21T02:10:31.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0ae9224640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdf3350 0x55e76022a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:31.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0accff9640 10 client.17166 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:31.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.023+0000 7f0ae9224640 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdf3350 0x55e76022a170 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea47bc766e4356f server_cookie=8f6c8a0c00066a21 in_seq=0 out_seq=0 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0accff9640 1 -- 172.21.15.92:0/173158303 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e76010e860 con 0x55e75fdf3350 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0accff9640 10 client.17166 handle_command_reply: tid=0 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 client.17166 shutdown 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 msgr2=0x55e7601b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601b1820 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005c40 tx=0x7f0ae0004160 comp rx=0 tx=0).stop 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 shutdown_connections 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e75fdf3350 0x55e76022a170 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e75fdd2670 0x55e7601b1820 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75fdca490 0x55e7601b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 --2- 172.21.15.92:0/173158303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e75fdb1cd0 0x55e7601b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0accff9640 0 client.17166 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.025+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 >> 172.21.15.92:0/173158303 conn(0x55e75fde7740 msgr2=0x55e76019be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 shutdown_connections 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0af19fc740 1 -- 172.21.15.92:0/173158303 wait complete. 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0af19fc740 20 client.17166 trim_cache size 0 max 16384 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "7ad514e7-6d04-4006-a154-862de6f99cc0", 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:10:31.030 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0ad00b6510 msgr2=0x7f0ad00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0ad00b6510 0x7f0ad00b8a20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0adac0 tx=0x7f0adc095040 comp rx=0 tx=0).stop 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae410de20 msgr2=0x7f0ae419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae410de20 0x7f0ae419fcb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009fd0 tx=0x7f0adc00ef10 comp rx=0 tx=0).stop 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 shutdown_connections 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0ad00b6510 0x7f0ad00b8a20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ae410de20 0x7f0ae419fcb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ae4105310 0x7f0ae419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 --2- 172.21.15.92:0/3116087184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ae41049c0 0x7f0ae419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:31.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 >> 172.21.15.92:0/3116087184 conn(0x7f0ae4079710 msgr2=0x7f0ae4106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:31.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 shutdown_connections 2024-09-21T02:10:31.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:31.026+0000 7f0aeb488640 1 -- 172.21.15.92:0/3116087184 wait complete. 2024-09-21T02:10:31.766 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:31 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:10:31.767 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:31 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:10:31.767 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:31 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:10:31.767 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:31 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:10:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:31 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:10:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:31 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:10:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:31 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:10:31.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:31 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:10:31.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:10:31.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:31 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:10:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:31 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:10:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:31 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:10:31.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:31 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:10:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:10:32.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.478+0000 7f0b64fff640 1 -- 172.21.15.92:0/3112366239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60102ae0 msgr2=0x7f0b60102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.478+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3112366239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60102ae0 0x7f0b60102ef0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009900 tx=0x7f0b5002a410 comp rx=0 tx=0).stop 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.478+0000 7f0b64fff640 1 -- 172.21.15.92:0/3112366239 shutdown_connections 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.478+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3112366239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b60108d20 0x7f0b6010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.478+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3112366239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b601083d0 0x7f0b601087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.478+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3112366239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60102ae0 0x7f0b60102ef0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.479+0000 7f0b64fff640 1 -- 172.21.15.92:0/3112366239 >> 172.21.15.92:0/3112366239 conn(0x7f0b60079760 msgr2=0x7f0b60079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.479+0000 7f0b64fff640 1 -- 172.21.15.92:0/3112366239 shutdown_connections 2024-09-21T02:10:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.479+0000 7f0b64fff640 1 -- 172.21.15.92:0/3112366239 wait complete. 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 Processor -- start 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 -- start start 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b60102ae0 0x7f0b6019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b601083d0 0x7f0b6019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 0x7f0b6019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60112990 con 0x7f0b60108d20 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60112810 con 0x7f0b60102ae0 2024-09-21T02:10:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b64fff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60112b10 con 0x7f0b601083d0 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b5ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 0x7f0b6019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b5ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 0x7f0b6019bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34920/0 (socket says 172.21.15.92:34920) 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b5ed9d640 1 -- 172.21.15.92:0/3310824562 learned_addr learned my addr 172.21.15.92:0/3310824562 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b601083d0 0x7f0b6019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.480+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b60102ae0 0x7f0b6019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5ed9d640 1 -- 172.21.15.92:0/3310824562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b601083d0 msgr2=0x7f0b6019b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b601083d0 0x7f0b6019b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5ed9d640 1 -- 172.21.15.92:0/3310824562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b60102ae0 msgr2=0x7f0b6019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b60102ae0 0x7f0b6019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5ed9d640 1 -- 172.21.15.92:0/3310824562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b601a6590 con 0x7f0b60108d20 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b60102ae0 0x7f0b6019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 0x7f0b6019bd80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800eaa0 tx=0x7f0b5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b57fff640 1 -- 172.21.15.92:0/3310824562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800cdb0 con 0x7f0b60108d20 2024-09-21T02:10:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b57fff640 1 -- 172.21.15.92:0/3310824562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b58005020 con 0x7f0b60108d20 2024-09-21T02:10:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b601a6880 con 0x7f0b60108d20 2024-09-21T02:10:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b57fff640 1 -- 172.21.15.92:0/3310824562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580116d0 con 0x7f0b60108d20 2024-09-21T02:10:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.481+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601a6d70 con 0x7f0b60108d20 2024-09-21T02:10:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 Processor -- start 2024-09-21T02:10:32.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b57fff640 1 -- 172.21.15.92:0/3310824562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f0b580051d0 con 0x7f0b60108d20 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 -- start start 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 0x563757a05650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a05b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56375797be50 con 0x56375765e070 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56375796b280 con 0x56375763d390 2024-09-21T02:10:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b6b573740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56375796f090 con 0x5637576351b0 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b55d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 0x563757a05650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b55d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 0x563757a05650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34934/0 (socket says 172.21.15.92:34934) 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b55d9b640 1 -- 172.21.15.92:0/361552827 learned_addr learned my addr 172.21.15.92:0/361552827 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.484+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a05b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 -- 172.21.15.92:0/361552827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 msgr2=0x563757a05b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a05b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 -- 172.21.15.92:0/361552827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 msgr2=0x563757a060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 -- 172.21.15.92:0/361552827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637576c9650 con 0x56375765e070 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b57fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0b4c0b6540 0x7f0b4c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 0x563757a05650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0b48013530 tx=0x7f0b4800a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6007b40bb396034 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0b4c0b6540 0x7f0b4c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b57fff640 1 -- 172.21.15.92:0/3310824562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0b580e7420 con 0x7f0b60108d20 2024-09-21T02:10:32.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b54d99640 1 -- 172.21.15.92:0/361552827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48019030 con 0x56375765e070 2024-09-21T02:10:32.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b54d99640 1 -- 172.21.15.92:0/361552827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b480110d0 con 0x56375765e070 2024-09-21T02:10:32.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.485+0000 7f0b54d99640 1 -- 172.21.15.92:0/361552827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48004d90 con 0x56375765e070 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 -- 172.21.15.92:0/361552827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 msgr2=0x563757a05650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0b4c0b6540 0x7f0b4c0b8a50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000c00 tx=0x7f0b50009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 0x563757a05650 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0b48013530 tx=0x7f0b4800a720 comp rx=0 tx=0).stop 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 -- 172.21.15.92:0/361552827 shutdown_connections 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a05b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 --2- 172.21.15.92:0/361552827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375765e070 0x563757a05650 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 -- 172.21.15.92:0/361552827 >> 172.21.15.92:0/361552827 conn(0x563757652460 msgr2=0x563757a03540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 -- 172.21.15.92:0/361552827 shutdown_connections 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.486+0000 7f0b6b573740 1 -- 172.21.15.92:0/361552827 wait complete. 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 -- start start 2024-09-21T02:10:32.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 Processor -- start 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 0x563757a15630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a21460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5637577cc700 con 0x56375763d390 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563757906110 con 0x56375765e070 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563757a18130 con 0x5637576351b0 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b6b573740 10 client.0 _ll_register_callbacks cb 0x56375791f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b55d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 0x563757a15630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b55d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 0x563757a15630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34944/0 (socket says 172.21.15.92:34944) 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.487+0000 7f0b55d9b640 1 -- 172.21.15.92:0/3393571315 learned_addr learned my addr 172.21.15.92:0/3393571315 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a21460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b55d9b640 1 -- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 msgr2=0x563757a150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b55d9b640 1 -- 172.21.15.92:0/3393571315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 msgr2=0x563757a21460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a21460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b55d9b640 1 -- 172.21.15.92:0/3393571315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563757933c40 con 0x56375763d390 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a21460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 0x563757a15630 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0b480049b0 tx=0x7f0b48005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575a56cafa8857e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b45ffb640 1 -- 172.21.15.92:0/3393571315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48019040 con 0x56375763d390 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b480110d0 con 0x56375763d390 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b45ffb640 1 -- 172.21.15.92:0/3393571315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0b480116b0 con 0x56375763d390 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b45ffb640 1 -- 172.21.15.92:0/3393571315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b4802d070 con 0x56375763d390 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b6b573740 10 client.17178 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:32.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.488+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563757a23e20 con 0x56375763d390 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b45ffb640 1 -- 172.21.15.92:0/3393571315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b480110d0 con 0x56375763d390 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b45ffb640 1 -- 172.21.15.92:0/3393571315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b48025020 con 0x56375763d390 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b6b573740 10 client.17178 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b6b573740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b6b573740 20 client.17178 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b6b573740 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56375761c9f0 0x563757a27290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b6b573740 4 client.17178 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56375797dda0 con 0x56375761c9f0 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.489+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56375761c9f0 0x563757a27290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.490+0000 7f0b45ffb640 10 client.17178 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.490+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56375761c9f0 0x563757a27290 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3cb5e306a215f5 server_cookie=6d372786cb2d3a4c in_seq=0 out_seq=0 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.490+0000 7f0b45ffb640 1 -- 172.21.15.92:0/3393571315 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56375797dda0 con 0x56375761c9f0 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.490+0000 7f0b45ffb640 10 client.17178 handle_command_reply: tid=0 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 client.17178 shutdown 2024-09-21T02:10:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 msgr2=0x563757a15630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 0x563757a15630 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0b480049b0 tx=0x7f0b48005250 comp rx=0 tx=0).stop 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 shutdown_connections 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56375761c9f0 0x563757a27290 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a21460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56375763d390 0x563757a15630 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b45ffb640 0 client.17178 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 --2- 172.21.15.92:0/3393571315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5637576351b0 0x563757a150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 >> 172.21.15.92:0/3393571315 conn(0x563757652460 msgr2=0x563757a01b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 shutdown_connections 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 1 -- 172.21.15.92:0/3393571315 wait complete. 2024-09-21T02:10:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.491+0000 7f0b6b573740 20 client.17178 trim_cache size 0 max 16384 2024-09-21T02:10:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 Processor -- start 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 -- start start 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a0af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375763d390 0x563757a21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ad2e0 con 0x5637576351b0 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601aee20 con 0x56375765e070 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b6b573740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ad9b0 con 0x56375763d390 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b5e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a0af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b5e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a0af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34956/0 (socket says 172.21.15.92:34956) 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.509+0000 7f0b5e59c640 1 -- 172.21.15.92:0/1279001814 learned_addr learned my addr 172.21.15.92:0/1279001814 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375763d390 0x563757a21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b5e59c640 1 -- 172.21.15.92:0/1279001814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375763d390 msgr2=0x563757a21360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375763d390 0x563757a21360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b5e59c640 1 -- 172.21.15.92:0/1279001814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 msgr2=0x563757a02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b5e59c640 1 -- 172.21.15.92:0/1279001814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563757a03f00 con 0x5637576351b0 2024-09-21T02:10:32.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a0af00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0b5002a930 tx=0x7f0b5003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0013f7ee410d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b47fff640 1 -- 172.21.15.92:0/1279001814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50035040 con 0x5637576351b0 2024-09-21T02:10:32.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b47fff640 1 -- 172.21.15.92:0/1279001814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b500054f0 con 0x5637576351b0 2024-09-21T02:10:32.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.510+0000 7f0b47fff640 1 -- 172.21.15.92:0/1279001814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50004bf0 con 0x5637576351b0 2024-09-21T02:10:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 -- 172.21.15.92:0/1279001814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 msgr2=0x563757a0af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a0af00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0b5002a930 tx=0x7f0b5003e000 comp rx=0 tx=0).stop 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 -- 172.21.15.92:0/1279001814 shutdown_connections 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375765e070 0x563757a02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375763d390 0x563757a21360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 --2- 172.21.15.92:0/1279001814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a0af00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 -- 172.21.15.92:0/1279001814 >> 172.21.15.92:0/1279001814 conn(0x563757652460 msgr2=0x563757a0a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 -- 172.21.15.92:0/1279001814 shutdown_connections 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 -- 172.21.15.92:0/1279001814 wait complete. 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 -- start start 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.511+0000 7f0b6b573740 1 Processor -- start 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a1dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375765e070 0x563757a195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ae750 con 0x5637576351b0 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ae080 con 0x56375763d390 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ab2c0 con 0x56375765e070 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b6b573740 10 client.0 _ll_register_callbacks cb 0x7f0b50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a1dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a1dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34960/0 (socket says 172.21.15.92:34960) 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 -- 172.21.15.92:0/4065894529 learned_addr learned my addr 172.21.15.92:0/4065894529 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375765e070 0x563757a195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 -- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375765e070 msgr2=0x563757a195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375765e070 0x563757a195e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 -- 172.21.15.92:0/4065894529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 msgr2=0x563757a19080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:32.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a19080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 -- 172.21.15.92:0/4065894529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563757a19d70 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5ed9d640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375765e070 0x563757a195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b5e59c640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a1dbd0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0b5002a450 tx=0x7f0b50004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713de548e83a5e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.512+0000 7f0b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b45ffb640 1 -- 172.21.15.92:0/4065894529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50035040 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b45ffb640 1 -- 172.21.15.92:0/4065894529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0b50058040 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b500054f0 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b45ffb640 1 -- 172.21.15.92:0/4065894529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b500038c0 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 10 client.17184 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563757a91a00 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b45ffb640 1 -- 172.21.15.92:0/4065894529 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b500054f0 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b45ffb640 1 -- 172.21.15.92:0/4065894529 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b5003cc40 con 0x5637576351b0 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 10 client.17184 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:32.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 20 client.17184 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637575f1da0 0x563757a941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 4 client.17184 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.513+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563757979850 con 0x5637575f1da0 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637575f1da0 0x563757a941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b45ffb640 10 client.17184 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b55d9b640 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637575f1da0 0x563757a941e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7169f76205061e server_cookie=8217b8a7be5df87 in_seq=0 out_seq=0 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b45ffb640 1 -- 172.21.15.92:0/4065894529 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x563757979850 con 0x5637575f1da0 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b45ffb640 10 client.17184 handle_command_reply: tid=0 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b6b573740 1 client.17184 shutdown 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 msgr2=0x563757a1dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.514+0000 7f0b6b573740 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a1dbd0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0b5002a450 tx=0x7f0b50004420 comp rx=0 tx=0).stop 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 shutdown_connections 2024-09-21T02:10:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5637575f1da0 0x563757a941e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56375765e070 0x563757a195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56375763d390 0x563757a19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b45ffb640 0 client.17184 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 --2- 172.21.15.92:0/4065894529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5637576351b0 0x563757a1dbd0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 >> 172.21.15.92:0/4065894529 conn(0x563757652460 msgr2=0x563757a16510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 shutdown_connections 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 1 -- 172.21.15.92:0/4065894529 wait complete. 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b6b573740 20 client.17184 trim_cache size 0 max 16384 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:10:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "7ad514e7-6d04-4006-a154-862de6f99cc0", 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:10:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:10:32.526 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:10:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0b4c0b6540 msgr2=0x7f0b4c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0b4c0b6540 0x7f0b4c0b8a50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000c00 tx=0x7f0b50009680 comp rx=0 tx=0).stop 2024-09-21T02:10:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 msgr2=0x7f0b6019bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 0x7f0b6019bd80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800eaa0 tx=0x7f0b5800ef60 comp rx=0 tx=0).stop 2024-09-21T02:10:32.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 shutdown_connections 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0b4c0b6540 0x7f0b4c0b8a50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b60108d20 0x7f0b6019bd80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b601083d0 0x7f0b6019b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 --2- 172.21.15.92:0/3310824562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b60102ae0 0x7f0b6019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.515+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 >> 172.21.15.92:0/3310824562 conn(0x7f0b60079760 msgr2=0x7f0b60101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.516+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 shutdown_connections 2024-09-21T02:10:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:10:32.516+0000 7f0b64fff640 1 -- 172.21.15.92:0/3310824562 wait complete. 2024-09-21T02:10:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:32 smithi136 ceph-mon[34754]: pgmap v527: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:10:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:32 smithi092 ceph-mon[31190]: pgmap v527: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:10:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:32 smithi099 ceph-mon[34404]: pgmap v527: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:10:33.286 INFO:tasks.cephfs.filesystem:scrub status for tag:7ad514e7-6d04-4006-a154-862de6f99cc0 - {'path': '/', 'tag': '7ad514e7-6d04-4006-a154-862de6f99cc0', 'options': 'recursive,force'} 2024-09-21T02:10:33.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:33 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:10:33.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:33 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:10:33.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:33 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:10:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:34 smithi136 ceph-mon[34754]: pgmap v528: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:10:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:34 smithi092 ceph-mon[31190]: pgmap v528: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:10:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:34 smithi099 ceph-mon[34404]: pgmap v528: 97 pgs: 97 active+clean; 4.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:10:36.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:36 smithi136 ceph-mon[34754]: pgmap v529: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:36.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:36 smithi092 ceph-mon[31190]: pgmap v529: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:36.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:36 smithi099 ceph-mon[34404]: pgmap v529: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 -- 172.21.15.92:0/945367966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5924103fe0 msgr2=0x7f592410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 --2- 172.21.15.92:0/945367966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5924103fe0 0x7f592410a980 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f591c009ca0 tx=0x7f591c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 -- 172.21.15.92:0/945367966 shutdown_connections 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 --2- 172.21.15.92:0/945367966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5924103fe0 0x7f592410a980 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 --2- 172.21.15.92:0/945367966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103640 0x7f5924103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 --2- 172.21.15.92:0/945367966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59241025c0 0x7f5924102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 -- 172.21.15.92:0/945367966 >> 172.21.15.92:0/945367966 conn(0x7f59240fdf00 msgr2=0x7f59241003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 -- 172.21.15.92:0/945367966 shutdown_connections 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.969+0000 7f592a20a640 1 -- 172.21.15.92:0/945367966 wait complete. 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.970+0000 7f592a20a640 1 Processor -- start 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.970+0000 7f592a20a640 1 -- start start 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 0x7f5924199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5924103640 0x7f592419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103fe0 0x7f592419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f592410e210 con 0x7f59241025c0 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f592410e090 con 0x7f5924103fe0 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f592410e390 con 0x7f5924103640 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 0x7f5924199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 0x7f5924199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48262/0 (socket says 172.21.15.92:48262) 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 -- 172.21.15.92:0/2069784921 learned_addr learned my addr 172.21.15.92:0/2069784921 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f5922ffd640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5924103640 0x7f592419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f5923fff640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103fe0 0x7f592419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 -- 172.21.15.92:0/2069784921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5924103640 msgr2=0x7f592419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5924103640 0x7f592419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 -- 172.21.15.92:0/2069784921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103fe0 msgr2=0x7f592419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103fe0 0x7f592419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 -- 172.21.15.92:0/2069784921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592419b970 con 0x7f59241025c0 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f5923fff640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103fe0 0x7f592419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f59237fe640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 0x7f5924199f30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f591400d770 tx=0x7f591400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f5920ff9640 1 -- 172.21.15.92:0/2069784921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914017070 con 0x7f59241025c0 2024-09-21T02:10:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f5920ff9640 1 -- 172.21.15.92:0/2069784921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59140054d0 con 0x7f59241025c0 2024-09-21T02:10:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592419bc60 con 0x7f59241025c0 2024-09-21T02:10:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f5920ff9640 1 -- 172.21.15.92:0/2069784921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914004c30 con 0x7f59241025c0 2024-09-21T02:10:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.971+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59241a2350 con 0x7f59241025c0 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 Processor -- start 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 -- start start 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f5920ff9640 1 -- 172.21.15.92:0/2069784921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5914004e10 con 0x7f59241025c0 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caaddd8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caaddd94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caaddda010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55caadd4b6f0 con 0x55caada082f0 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55caadd3ab20 con 0x55caada311b0 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f593124f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55caadd3e930 con 0x55caada104d0 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caaddd94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caaddd94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48266/0 (socket says 172.21.15.92:48266) 2024-09-21T02:10:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.974+0000 7f59237fe640 1 -- 172.21.15.92:0/217990485 learned_addr learned my addr 172.21.15.92:0/217990485 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5923fff640 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caaddd8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5922ffd640 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caaddda010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f59237fe640 1 -- 172.21.15.92:0/217990485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 msgr2=0x55caaddda010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f59237fe640 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caaddda010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f59237fe640 1 -- 172.21.15.92:0/217990485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 msgr2=0x55caaddd8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f59237fe640 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caaddd8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f59237fe640 1 -- 172.21.15.92:0/217990485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caadd00530 con 0x55caada082f0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5920ff9640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f590c0b6510 0x7f590c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f59237fe640 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caaddd94c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f59140eaf20 tx=0x7f59140ef570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf292606b092278 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5923fff640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f590c0b6510 0x7f590c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5909ffb640 1 -- 172.21.15.92:0/217990485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140a95d0 con 0x55caada082f0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5920ff9640 1 -- 172.21.15.92:0/2069784921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f59140e6e10 con 0x7f59241025c0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5909ffb640 1 -- 172.21.15.92:0/217990485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59140aca90 con 0x55caada082f0 2024-09-21T02:10:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.975+0000 7f5909ffb640 1 -- 172.21.15.92:0/217990485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140d9910 con 0x55caada082f0 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 -- 172.21.15.92:0/217990485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 msgr2=0x55caaddd94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f5923fff640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f590c0b6510 0x7f590c0b8a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f591c0094e0 tx=0x7f591c031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caaddd94c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f59140eaf20 tx=0x7f59140ef570 comp rx=0 tx=0).stop 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 -- 172.21.15.92:0/217990485 shutdown_connections 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caaddda010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caaddd94c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 --2- 172.21.15.92:0/217990485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caaddd8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 -- 172.21.15.92:0/217990485 >> 172.21.15.92:0/217990485 conn(0x55caada25300 msgr2=0x55caaddd48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 -- 172.21.15.92:0/217990485 shutdown_connections 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.976+0000 7f593124f740 1 -- 172.21.15.92:0/217990485 wait complete. 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 -- start start 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 Processor -- start 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada104d0 0x55caadde2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada311b0 0x55caadde37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55caadc72ab0 con 0x55caada082f0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55caadce0730 con 0x55caada104d0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55caadddab30 con 0x55caada311b0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f593124f740 10 client.0 _ll_register_callbacks cb 0x55caadd346b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48274/0 (socket says 172.21.15.92:48274) 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f59237fe640 1 -- 172.21.15.92:0/258223374 learned_addr learned my addr 172.21.15.92:0/258223374 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f5922ffd640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada104d0 0x55caadde2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.977+0000 7f5923fff640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada311b0 0x55caadde37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f59237fe640 1 -- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada311b0 msgr2=0x55caadde37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f59237fe640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada311b0 0x55caadde37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f59237fe640 1 -- 172.21.15.92:0/258223374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada104d0 msgr2=0x55caadde2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f59237fe640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada104d0 0x55caadde2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f59237fe640 1 -- 172.21.15.92:0/258223374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caada9c790 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5922ffd640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada104d0 0x55caadde2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f59237fe640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde25a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f59140e71c0 tx=0x7f5914009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167e97ebc85606f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5902ffd640 1 -- 172.21.15.92:0/258223374 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914020070 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5902ffd640 1 -- 172.21.15.92:0/258223374 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f59140eaaa0 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59140aca90 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5902ffd640 1 -- 172.21.15.92:0/258223374 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59140d6a50 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 10 client.17196 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55caadb15620 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5902ffd640 1 -- 172.21.15.92:0/258223374 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59140aca90 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f5902ffd640 1 -- 172.21.15.92:0/258223374 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5914005100 con 0x55caada082f0 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 10 client.17196 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 20 client.17196 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9efd80 0x55caaddef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 4 client.17196 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.978+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caadd4d640 con 0x55caad9efd80 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.979+0000 7f5922ffd640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9efd80 0x55caaddef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.979+0000 7f5902ffd640 10 client.17196 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.979+0000 7f5922ffd640 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9efd80 0x55caaddef8b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cde39cfcfd3ee74 server_cookie=26410bb057c91b69 in_seq=0 out_seq=0 2024-09-21T02:10:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f5902ffd640 1 -- 172.21.15.92:0/258223374 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55caadd4d640 con 0x55caad9efd80 2024-09-21T02:10:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f5902ffd640 10 client.17196 handle_command_reply: tid=0 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 client.17196 shutdown 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 msgr2=0x55caadde25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde25a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f59140e71c0 tx=0x7f5914009740 comp rx=0 tx=0).stop 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 shutdown_connections 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9efd80 0x55caaddef8b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada311b0 0x55caadde37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f5902ffd640 0 client.17196 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada104d0 0x55caadde2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 --2- 172.21.15.92:0/258223374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde25a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 >> 172.21.15.92:0/258223374 conn(0x55caada25300 msgr2=0x55caaddd9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 shutdown_connections 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 1 -- 172.21.15.92:0/258223374 wait complete. 2024-09-21T02:10:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.980+0000 7f593124f740 20 client.17196 trim_cache size 0 max 16384 2024-09-21T02:10:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.995+0000 7f593124f740 1 Processor -- start 2024-09-21T02:10:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.995+0000 7f593124f740 1 -- start start 2024-09-21T02:10:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.995+0000 7f593124f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada082f0 0x55caadde0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f593124f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f593124f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 0x55caaddd4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f593124f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59241ab050 con 0x55caada311b0 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f593124f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59241acb90 con 0x55caada082f0 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f593124f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59241ab720 con 0x55caada104d0 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 0x55caaddd4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 0x55caaddd4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48278/0 (socket says 172.21.15.92:48278) 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 -- 172.21.15.92:0/2370903920 learned_addr learned my addr 172.21.15.92:0/2370903920 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f59237fe640 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada082f0 0x55caadde0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5922ffd640 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 -- 172.21.15.92:0/2370903920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 msgr2=0x55caadde0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 -- 172.21.15.92:0/2370903920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada082f0 msgr2=0x55caadde0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada082f0 0x55caadde0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 -- 172.21.15.92:0/2370903920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caadde0730 con 0x55caada311b0 2024-09-21T02:10:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f5923fff640 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 0x55caaddd4220 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f591c031770 tx=0x7f591c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f589ec428dc820b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f59097fa640 1 -- 172.21.15.92:0/2370903920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c02f620 con 0x55caada311b0 2024-09-21T02:10:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f59097fa640 1 -- 172.21.15.92:0/2370903920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f591c036040 con 0x55caada311b0 2024-09-21T02:10:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.996+0000 7f59097fa640 1 -- 172.21.15.92:0/2370903920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c030a20 con 0x55caada311b0 2024-09-21T02:10:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 -- 172.21.15.92:0/2370903920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 msgr2=0x55caaddd4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 0x55caaddd4220 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f591c031770 tx=0x7f591c041000 comp rx=0 tx=0).stop 2024-09-21T02:10:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 -- 172.21.15.92:0/2370903920 shutdown_connections 2024-09-21T02:10:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada311b0 0x55caaddd4220 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 --2- 172.21.15.92:0/2370903920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada082f0 0x55caadde0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 -- 172.21.15.92:0/2370903920 >> 172.21.15.92:0/2370903920 conn(0x55caada25300 msgr2=0x55caaddd8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 -- 172.21.15.92:0/2370903920 shutdown_connections 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 -- 172.21.15.92:0/2370903920 wait complete. 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.997+0000 7f593124f740 1 -- start start 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 Processor -- start 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caadde65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59241ac4c0 con 0x55caada082f0 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59241abdf0 con 0x55caada311b0 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59241a9030 con 0x55caada104d0 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f593124f740 10 client.0 _ll_register_callbacks cb 0x7f5918005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48292/0 (socket says 172.21.15.92:48292) 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 -- 172.21.15.92:0/3932977471 learned_addr learned my addr 172.21.15.92:0/3932977471 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f5923fff640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caadde65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f5922ffd640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 -- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 msgr2=0x55caadde5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 -- 172.21.15.92:0/3932977471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 msgr2=0x55caadde65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caadde65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f59237fe640 1 -- 172.21.15.92:0/3932977471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caadde6d40 con 0x55caada082f0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.998+0000 7f5923fff640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caadde65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f59237fe640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde53b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f59140a9550 tx=0x7f59140effd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cb791449c9ec4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f5902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f5902ffd640 1 -- 172.21.15.92:0/3932977471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914020070 con 0x55caada082f0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f5902ffd640 1 -- 172.21.15.92:0/3932977471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f59140982f0 con 0x55caada082f0 2024-09-21T02:10:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591c036040 con 0x55caada082f0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f5902ffd640 1 -- 172.21.15.92:0/3932977471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59140d6870 con 0x55caada082f0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 10 client.17202 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55caaddea070 con 0x55caada082f0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f5902ffd640 1 -- 172.21.15.92:0/3932977471 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f591c036040 con 0x55caada082f0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f5902ffd640 1 -- 172.21.15.92:0/3932977471 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f59140a88d0 con 0x55caada082f0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 10 client.17202 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 20 client.17202 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9c51c0 0x55caade5c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 4 client.17202 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:36.999+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55caadd490f0 con 0x55caad9c51c0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.000+0000 7f5922ffd640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9c51c0 0x55caade5c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.000+0000 7f5902ffd640 10 client.17202 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.000+0000 7f5922ffd640 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9c51c0 0x55caade5c150 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0cf48f80e45b86c server_cookie=b96c826f201f8e1f in_seq=0 out_seq=0 2024-09-21T02:10:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.015+0000 7f5902ffd640 1 -- 172.21.15.92:0/3932977471 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55caadd490f0 con 0x55caad9c51c0 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.015+0000 7f5902ffd640 10 client.17202 handle_command_reply: tid=0 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.015+0000 7f593124f740 1 client.17202 shutdown 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 msgr2=0x55caadde53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde53b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f59140a9550 tx=0x7f59140effd0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 shutdown_connections 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55caad9c51c0 0x55caade5c150 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55caada311b0 0x55caadde65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f5902ffd640 0 client.17202 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caada104d0 0x55caadde5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 --2- 172.21.15.92:0/3932977471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55caada082f0 0x55caadde53b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 >> 172.21.15.92:0/3932977471 conn(0x55caada25300 msgr2=0x55caadde0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 shutdown_connections 2024-09-21T02:10:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 1 -- 172.21.15.92:0/3932977471 wait complete. 2024-09-21T02:10:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f593124f740 20 client.17202 trim_cache size 0 max 16384 2024-09-21T02:10:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:10:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:10:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:10:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f590c0b6510 msgr2=0x7f590c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f590c0b6510 0x7f590c0b8a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f591c0094e0 tx=0x7f591c031af0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 msgr2=0x7f5924199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 0x7f5924199f30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f591400d770 tx=0x7f591400dc30 comp rx=0 tx=0).stop 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 shutdown_connections 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f590c0b6510 0x7f590c0b8a20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5924103fe0 0x7f592419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5924103640 0x7f592419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 --2- 172.21.15.92:0/2069784921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59241025c0 0x7f5924199f30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.016+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 >> 172.21.15.92:0/2069784921 conn(0x7f59240fdf00 msgr2=0x7f592406ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.017+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 shutdown_connections 2024-09-21T02:10:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:37.017+0000 7f592a20a640 1 -- 172.21.15.92:0/2069784921 wait complete. 2024-09-21T02:10:38.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:38 smithi136 ceph-mon[34754]: pgmap v530: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:38.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:38 smithi092 ceph-mon[31190]: pgmap v530: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:38.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:38 smithi099 ceph-mon[34404]: pgmap v530: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:40.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:40 smithi136 ceph-mon[34754]: pgmap v531: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-09-21T02:10:40.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:40 smithi092 ceph-mon[31190]: pgmap v531: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-09-21T02:10:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:40 smithi099 ceph-mon[34404]: pgmap v531: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 24 op/s 2024-09-21T02:10:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:42 smithi136 ceph-mon[34754]: pgmap v532: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:10:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:10:42.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:10:42.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:10:42.784 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:10:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:42 smithi092 ceph-mon[31190]: pgmap v532: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:10:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:10:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:10:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:10:42.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:10:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:42 smithi099 ceph-mon[34404]: pgmap v532: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:10:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:10:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:10:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:10:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:10:44.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:44 smithi136 ceph-mon[34754]: pgmap v533: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:10:44.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:44 smithi092 ceph-mon[31190]: pgmap v533: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:10:44.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:44 smithi099 ceph-mon[34404]: pgmap v533: 97 pgs: 97 active+clean; 4.4 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:251/600: open fd = 3 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:252/600: open fd = 3 2024-09-21T02:10:46.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:253/600: open fd = 3 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:254/600: open fd = 3 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.222 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:255/600: open fd = 3 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:256/600: open fd = 3 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:257/600: open fd = 3 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.223 INFO:tasks.workunit.client.0.smithi092.stdout:258/600: open fd = 3 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:259/600: open fd = 3 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:260/600: open fd = 3 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.224 INFO:tasks.workunit.client.0.smithi092.stdout:261/600: open fd = 3 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:262/600: open fd = 3 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:263/600: open fd = 3 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.225 INFO:tasks.workunit.client.0.smithi092.stdout:264/600: open fd = 3 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:265/600: open fd = 3 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:266/600: open fd = 3 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:267/600: open fd = 3 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:268/600: open fd = 3 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:269/600: open fd = 3 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:270/600: open fd = 3 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:271/600: open fd = 3 2024-09-21T02:10:46.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:272/600: open fd = 3 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:273/600: open fd = 3 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.228 INFO:tasks.workunit.client.0.smithi092.stdout:274/600: open fd = 3 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:275/600: open fd = 3 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:276/600: open fd = 3 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.229 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:277/600: open fd = 3 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:278/600: open fd = 3 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.230 INFO:tasks.workunit.client.0.smithi092.stdout:279/600: open fd = 3 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:280/600: open fd = 3 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:281/600: open fd = 3 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:282/600: open fd = 3 2024-09-21T02:10:46.231 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:283/600: open fd = 3 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:284/600: open fd = 3 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.232 INFO:tasks.workunit.client.0.smithi092.stdout:285/600: open fd = 3 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:286/600: open fd = 3 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:287/600: open fd = 3 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:288/600: open fd = 3 2024-09-21T02:10:46.233 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:289/600: open fd = 3 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:290/600: open fd = 3 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.234 INFO:tasks.workunit.client.0.smithi092.stdout:291/600: open fd = 3 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:292/600: open fd = 3 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:293/600: open fd = 3 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.235 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:294/600: open fd = 3 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:295/600: open fd = 3 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:296/600: open fd = 3 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.236 INFO:tasks.workunit.client.0.smithi092.stdout:297/600: open fd = 3 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:298/600: open fd = 3 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:299/600: open fd = 3 2024-09-21T02:10:46.237 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:300/600: open fd = 3 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:301/600: open fd = 3 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.238 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:302/600: open fd = 3 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:303/600: open fd = 3 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.239 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:304/600: open fd = 3 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:305/600: open fd = 3 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.240 INFO:tasks.workunit.client.0.smithi092.stdout:306/600: open fd = 3 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:307/600: open fd = 3 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.241 INFO:tasks.workunit.client.0.smithi092.stdout:308/600: open fd = 3 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:309/600: open fd = 3 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.242 INFO:tasks.workunit.client.0.smithi092.stdout:310/600: open fd = 3 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:311/600: open fd = 3 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:312/600: open fd = 3 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.243 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:10:46.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:46 smithi136 ceph-mon[34754]: pgmap v534: 97 pgs: 97 active+clean; 4.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:46.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:46 smithi092 ceph-mon[31190]: pgmap v534: 97 pgs: 97 active+clean; 4.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:46.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:46 smithi099 ceph-mon[34404]: pgmap v534: 97 pgs: 97 active+clean; 4.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:10:48.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:48 smithi136 ceph-mon[34754]: pgmap v535: 97 pgs: 97 active+clean; 4.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:48 smithi092 ceph-mon[31190]: pgmap v535: 97 pgs: 97 active+clean; 4.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:48.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:48 smithi099 ceph-mon[34404]: pgmap v535: 97 pgs: 97 active+clean; 4.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.119+0000 7fa41cdba640 1 -- 172.21.15.92:0/2965694182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418109480 msgr2=0x7fa41810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.119+0000 7fa41cdba640 1 --2- 172.21.15.92:0/2965694182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418109480 0x7fa41810b920 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ca0 tx=0x7fa41002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 -- 172.21.15.92:0/2965694182 shutdown_connections 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 --2- 172.21.15.92:0/2965694182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418109480 0x7fa41810b920 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 --2- 172.21.15.92:0/2965694182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa418106b20 0x7fa418108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 --2- 172.21.15.92:0/2965694182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa418101a90 0x7fa418101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 -- 172.21.15.92:0/2965694182 >> 172.21.15.92:0/2965694182 conn(0x7fa4180fd6d0 msgr2=0x7fa4180ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 -- 172.21.15.92:0/2965694182 shutdown_connections 2024-09-21T02:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 -- 172.21.15.92:0/2965694182 wait complete. 2024-09-21T02:10:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 Processor -- start 2024-09-21T02:10:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.120+0000 7fa41cdba640 1 -- start start 2024-09-21T02:10:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa418101a90 0x7fa418199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 0x7fa41819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa418109480 0x7fa41819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa41810f320 con 0x7fa418106b20 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa41810f1a0 con 0x7fa418109480 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa41810f4a0 con 0x7fa418101a90 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 0x7fa41819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 0x7fa41819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37472/0 (socket says 172.21.15.92:37472) 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 -- 172.21.15.92:0/1358945382 learned_addr learned my addr 172.21.15.92:0/1358945382 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41659c640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa418101a90 0x7fa418199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa416d9d640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa418109480 0x7fa41819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 -- 172.21.15.92:0/1358945382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa418101a90 msgr2=0x7fa418199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa418101a90 0x7fa418199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 -- 172.21.15.92:0/1358945382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa418109480 msgr2=0x7fa41819b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa418109480 0x7fa41819b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 -- 172.21.15.92:0/1358945382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41819b960 con 0x7fa418106b20 2024-09-21T02:10:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa415d9b640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 0x7fa41819a590 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00d950 tx=0x7fa40c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa4077fe640 1 -- 172.21.15.92:0/1358945382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c017030 con 0x7fa418106b20 2024-09-21T02:10:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.121+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41819bc50 con 0x7fa418106b20 2024-09-21T02:10:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.122+0000 7fa4077fe640 1 -- 172.21.15.92:0/1358945382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40c004020 con 0x7fa418106b20 2024-09-21T02:10:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.122+0000 7fa4077fe640 1 -- 172.21.15.92:0/1358945382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0047a0 con 0x7fa418106b20 2024-09-21T02:10:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.122+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a2340 con 0x7fa418106b20 2024-09-21T02:10:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.124+0000 7fa423e06740 1 Processor -- start 2024-09-21T02:10:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.124+0000 7fa423e06740 1 -- start start 2024-09-21T02:10:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa423e06740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa423e06740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa423e06740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755b7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa423e06740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5619755286f0 con 0x5619751e52f0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa423e06740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561975517b20 con 0x56197520e1b0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa423e06740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56197551b930 con 0x5619751ed4d0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa4077fe640 1 -- 172.21.15.92:0/1358945382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa40c009a30 con 0x7fa418106b20 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37476/0 (socket says 172.21.15.92:37476) 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 -- 172.21.15.92:0/3868787448 learned_addr learned my addr 172.21.15.92:0/3868787448 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa415d9b640 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa416d9d640 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 -- 172.21.15.92:0/3868787448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 msgr2=0x5619755b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755b7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 -- 172.21.15.92:0/3868787448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 msgr2=0x5619755b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 -- 172.21.15.92:0/3868787448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619754dd530 con 0x5619751e52f0 2024-09-21T02:10:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa416d9d640 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa41659c640 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b64c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b760 tx=0x7fa40800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6108fa65a58a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.125+0000 7fa404ff9640 1 -- 172.21.15.92:0/3868787448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408003dd0 con 0x5619751e52f0 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa404ff9640 1 -- 172.21.15.92:0/3868787448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa408015070 con 0x5619751e52f0 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa404ff9640 1 -- 172.21.15.92:0/3868787448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408004d50 con 0x5619751e52f0 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa4077fe640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4000b6510 0x7fa4000b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa416d9d640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4000b6510 0x7fa4000b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa4077fe640 1 -- 172.21.15.92:0/1358945382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa40c0e6cd0 con 0x7fa418106b20 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa423e06740 1 -- 172.21.15.92:0/3868787448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 msgr2=0x5619755b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa423e06740 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b64c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b760 tx=0x7fa40800bc20 comp rx=0 tx=0).stop 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa416d9d640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4000b6510 0x7fa4000b8a20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa410031b50 tx=0x7fa4100099a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.126+0000 7fa423e06740 1 -- 172.21.15.92:0/3868787448 shutdown_connections 2024-09-21T02:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b64c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 --2- 172.21.15.92:0/3868787448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- 172.21.15.92:0/3868787448 >> 172.21.15.92:0/3868787448 conn(0x561975202300 msgr2=0x5619755b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- 172.21.15.92:0/3868787448 shutdown_connections 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- 172.21.15.92:0/3868787448 wait complete. 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- start start 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 Processor -- start 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755bfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56197544fab0 con 0x5619751e52f0 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5619754bd730 con 0x56197520e1b0 2024-09-21T02:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619755b7b30 con 0x5619751ed4d0 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa423e06740 10 client.0 _ll_register_callbacks cb 0x5619755116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa41659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa41659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755bf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37480/0 (socket says 172.21.15.92:37480) 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.127+0000 7fa41659c640 1 -- 172.21.15.92:0/4177707547 learned_addr learned my addr 172.21.15.92:0/4177707547 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa415d9b640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755bfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa416d9d640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa41659c640 1 -- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 msgr2=0x5619755bfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa41659c640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755bfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa41659c640 1 -- 172.21.15.92:0/4177707547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 msgr2=0x5619755c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa41659c640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa41659c640 1 -- 172.21.15.92:0/4177707547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561975279790 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa415d9b640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755bfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa41659c640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755bf640 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa408004310 tx=0x7fa408003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacab2e1f824cc3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa3fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa3fdffb640 1 -- 172.21.15.92:0/4177707547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408028060 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa3fdffb640 1 -- 172.21.15.92:0/4177707547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa408011850 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa408015070 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa3fdffb640 1 -- 172.21.15.92:0/4177707547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa408011a50 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa423e06740 10 client.17214 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619752f2620 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.128+0000 7fa3fdffb640 1 -- 172.21.15.92:0/4177707547 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa408015070 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa3fdffb640 1 -- 172.21.15.92:0/4177707547 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa408022070 con 0x5619751e52f0 2024-09-21T02:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa423e06740 10 client.17214 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa423e06740 10 client.17214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa423e06740 20 client.17214 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa423e06740 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5619751ccd80 0x5619755cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa423e06740 4 client.17214 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56197552a640 con 0x5619751ccd80 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.129+0000 7fa415d9b640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5619751ccd80 0x5619755cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa3fdffb640 10 client.17214 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa415d9b640 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5619751ccd80 0x5619755cc9f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c14ec50f2fb7986 server_cookie=314c2344b60ef52a in_seq=0 out_seq=0 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa3fdffb640 1 -- 172.21.15.92:0/4177707547 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56197552a640 con 0x5619751ccd80 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa3fdffb640 10 client.17214 handle_command_reply: tid=0 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 client.17214 shutdown 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 msgr2=0x5619755bf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755bf640 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa408004310 tx=0x7fa408003e00 comp rx=0 tx=0).stop 2024-09-21T02:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 shutdown_connections 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5619751ccd80 0x5619755cc9f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56197520e1b0 0x5619755c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ed4d0 0x5619755bfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa3fdffb640 0 client.17214 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 --2- 172.21.15.92:0/4177707547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755bf640 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.130+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 >> 172.21.15.92:0/4177707547 conn(0x561975202300 msgr2=0x5619755b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.131+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 shutdown_connections 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.131+0000 7fa423e06740 1 -- 172.21.15.92:0/4177707547 wait complete. 2024-09-21T02:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.131+0000 7fa423e06740 20 client.17214 trim_cache size 0 max 16384 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 Processor -- start 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 -- start start 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181aafd0 con 0x5619751e52f0 2024-09-21T02:10:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181acb10 con 0x5619751ed4d0 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa423e06740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181ab6a0 con 0x5619751ccd80 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37492/0 (socket says 172.21.15.92:37492) 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 -- 172.21.15.92:0/2725204431 learned_addr learned my addr 172.21.15.92:0/2725204431 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa416d9d640 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa41659c640 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 -- 172.21.15.92:0/2725204431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 msgr2=0x5619755b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 -- 172.21.15.92:0/2725204431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 msgr2=0x5619755b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa415d9b640 1 -- 172.21.15.92:0/2725204431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619755c0b70 con 0x5619751e52f0 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.145+0000 7fa416d9d640 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.146+0000 7fa415d9b640 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b20a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0acb40 tx=0x7fa40c0e4680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6225efabc8cad92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.146+0000 7fa3fffff640 1 -- 172.21.15.92:0/2725204431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0af020 con 0x5619751e52f0 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.146+0000 7fa3fffff640 1 -- 172.21.15.92:0/2725204431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40c0e8040 con 0x5619751e52f0 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.146+0000 7fa3fffff640 1 -- 172.21.15.92:0/2725204431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c009d20 con 0x5619751e52f0 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.146+0000 7fa423e06740 1 -- 172.21.15.92:0/2725204431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 msgr2=0x5619755b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.146+0000 7fa423e06740 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b20a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0acb40 tx=0x7fa40c0e4680 comp rx=0 tx=0).stop 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- 172.21.15.92:0/2725204431 shutdown_connections 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755b20a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 --2- 172.21.15.92:0/2725204431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- 172.21.15.92:0/2725204431 >> 172.21.15.92:0/2725204431 conn(0x561975202300 msgr2=0x5619755b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- 172.21.15.92:0/2725204431 shutdown_connections 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- 172.21.15.92:0/2725204431 wait complete. 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- start start 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 Processor -- start 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181ac440 con 0x5619751e52f0 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181abd70 con 0x5619751ed4d0 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181a8fb0 con 0x5619751ccd80 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa423e06740 10 client.0 _ll_register_callbacks cb 0x7fa408004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa415d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa415d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37504/0 (socket says 172.21.15.92:37504) 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa415d9b640 1 -- 172.21.15.92:0/1335469733 learned_addr learned my addr 172.21.15.92:0/1335469733 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa416d9d640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.147+0000 7fa41659c640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa415d9b640 1 -- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 msgr2=0x5619755c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa415d9b640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa415d9b640 1 -- 172.21.15.92:0/1335469733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 msgr2=0x5619755c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:10:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa415d9b640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa415d9b640 1 -- 172.21.15.92:0/1335469733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619755c4df0 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa41659c640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa415d9b640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755c66f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0e4800 tx=0x7fa40c094440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959aa2e6e011b0c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa3fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa3fdffb640 1 -- 172.21.15.92:0/1335469733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0af040 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa3fdffb640 1 -- 172.21.15.92:0/1335469733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa40c0041d0 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40c0e8040 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa3fdffb640 1 -- 172.21.15.92:0/1335469733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa40c0ad540 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 10 client.17220 fetch_fsmap learned FSMap version 29 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561975637690 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa3fdffb640 1 -- 172.21.15.92:0/1335469733 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40c0e8040 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa3fdffb640 1 -- 172.21.15.92:0/1335469733 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa40c0d40d0 con 0x5619751e52f0 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 10 client.17220 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:10:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 20 client.17220 populate_metadata read hostname 'smithi092' 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.148+0000 7fa423e06740 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56197520e1b0 0x561975639fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.149+0000 7fa423e06740 4 client.17220 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.149+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619755260f0 con 0x56197520e1b0 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.149+0000 7fa41659c640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56197520e1b0 0x561975639fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.149+0000 7fa3fdffb640 10 client.17220 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.149+0000 7fa41659c640 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56197520e1b0 0x561975639fd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f99a499a1a67fb6 server_cookie=1d2aefc9671dcc51 in_seq=0 out_seq=0 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa3fdffb640 1 -- 172.21.15.92:0/1335469733 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619755260f0 con 0x56197520e1b0 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa3fdffb640 10 client.17220 handle_command_reply: tid=0 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 client.17220 shutdown 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 msgr2=0x5619755c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755c66f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0e4800 tx=0x7fa40c094440 comp rx=0 tx=0).stop 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 shutdown_connections 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56197520e1b0 0x561975639fd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619751ed4d0 0x5619755c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5619751e52f0 0x5619755c66f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa3fdffb640 0 client.17220 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 --2- 172.21.15.92:0/1335469733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619751ccd80 0x5619755c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 >> 172.21.15.92:0/1335469733 conn(0x561975202300 msgr2=0x5619755d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 shutdown_connections 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 1 -- 172.21.15.92:0/1335469733 wait complete. 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa423e06740 20 client.17220 trim_cache size 0 max 16384 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:10:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:10:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4000b6510 msgr2=0x7fa4000b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa41cdba640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4000b6510 0x7fa4000b8a20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa410031b50 tx=0x7fa4100099a0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 msgr2=0x7fa41819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:10:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.165+0000 7fa41cdba640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 0x7fa41819a590 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00d950 tx=0x7fa40c00de10 comp rx=0 tx=0).stop 2024-09-21T02:10:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 shutdown_connections 2024-09-21T02:10:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa4000b6510 0x7fa4000b8a20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa418109480 0x7fa41819b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa418106b20 0x7fa41819a590 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 --2- 172.21.15.92:0/1358945382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa418101a90 0x7fa418199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:10:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 >> 172.21.15.92:0/1358945382 conn(0x7fa4180fd6d0 msgr2=0x7fa4181078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:10:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 shutdown_connections 2024-09-21T02:10:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:10:50.166+0000 7fa41cdba640 1 -- 172.21.15.92:0/1358945382 wait complete. 2024-09-21T02:10:50.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:50 smithi136 ceph-mon[34754]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:50.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:50 smithi092 ceph-mon[31190]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:50.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:50 smithi099 ceph-mon[34404]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:10:52.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:52 smithi136 ceph-mon[34754]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:52 smithi092 ceph-mon[31190]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:52.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:52 smithi099 ceph-mon[34404]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:10:54.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:54 smithi136 ceph-mon[34754]: pgmap v538: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-21T02:10:54.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:54 smithi092 ceph-mon[31190]: pgmap v538: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-21T02:10:54.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:54 smithi099 ceph-mon[34404]: pgmap v538: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-21T02:10:56.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:56 smithi136 ceph-mon[34754]: pgmap v539: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:10:56.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:56 smithi092 ceph-mon[31190]: pgmap v539: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:10:56.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:56 smithi099 ceph-mon[34404]: pgmap v539: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:10:58.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:10:58 smithi136 ceph-mon[34754]: pgmap v540: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:58.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:10:58 smithi092 ceph-mon[31190]: pgmap v540: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:10:58.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:10:58 smithi099 ceph-mon[34404]: pgmap v540: 97 pgs: 97 active+clean; 4.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:00.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:00 smithi136 ceph-mon[34754]: pgmap v541: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:11:00.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:00 smithi092 ceph-mon[31190]: pgmap v541: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:11:00.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:00 smithi099 ceph-mon[34404]: pgmap v541: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:11:02.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:02 smithi136 ceph-mon[34754]: pgmap v542: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:11:02.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:02 smithi092 ceph-mon[31190]: pgmap v542: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:11:02.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:02 smithi099 ceph-mon[34404]: pgmap v542: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:11:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 -- 172.21.15.92:0/3870410669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef8103fe0 msgr2=0x7f4ef810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/3870410669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef8103fe0 0x7f4ef810a980 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009ca0 tx=0x7f4ef002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 -- 172.21.15.92:0/3870410669 shutdown_connections 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/3870410669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef8103fe0 0x7f4ef810a980 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/3870410669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 0x7f4ef8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/3870410669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef81025c0 0x7f4ef8102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 -- 172.21.15.92:0/3870410669 >> 172.21.15.92:0/3870410669 conn(0x7f4ef80fdf00 msgr2=0x7f4ef81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 -- 172.21.15.92:0/3870410669 shutdown_connections 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.270+0000 7f4eff3cc640 1 -- 172.21.15.92:0/3870410669 wait complete. 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 Processor -- start 2024-09-21T02:11:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 -- start start 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 0x7f4ef806c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 0x7f4ef819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8103fe0 0x7f4ef81a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef810e210 con 0x7f4ef81025c0 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef810e090 con 0x7f4ef8103640 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4eff3cc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef810e390 con 0x7f4ef8103fe0 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4efd168640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 0x7f4ef806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4efd168640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 0x7f4ef806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41220/0 (socket says 172.21.15.92:41220) 2024-09-21T02:11:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.271+0000 7f4efd168640 1 -- 172.21.15.92:0/2143971634 learned_addr learned my addr 172.21.15.92:0/2143971634 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd969640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8103fe0 0x7f4ef81a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efc967640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 0x7f4ef819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd168640 1 -- 172.21.15.92:0/2143971634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8103fe0 msgr2=0x7f4ef81a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd168640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8103fe0 0x7f4ef81a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd168640 1 -- 172.21.15.92:0/2143971634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 msgr2=0x7f4ef819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd168640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 0x7f4ef819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd168640 1 -- 172.21.15.92:0/2143971634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef81a19a0 con 0x7f4ef81025c0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efc967640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 0x7f4ef819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4efd168640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 0x7f4ef806c210 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c970 tx=0x7f4ef400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4eee7fc640 1 -- 172.21.15.92:0/2143971634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4007bb0 con 0x7f4ef81025c0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4eee7fc640 1 -- 172.21.15.92:0/2143971634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef4007d50 con 0x7f4ef81025c0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4eee7fc640 1 -- 172.21.15.92:0/2143971634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400f450 con 0x7f4ef81025c0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef81a1c90 con 0x7f4ef81025c0 2024-09-21T02:11:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.272+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef81a2190 con 0x7f4ef81025c0 2024-09-21T02:11:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 Processor -- start 2024-09-21T02:11:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4eee7fc640 1 -- 172.21.15.92:0/2143971634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4ef4004f20 con 0x7f4ef81025c0 2024-09-21T02:11:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 -- start start 2024-09-21T02:11:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd3223e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e532f0 0x55efd32244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3225010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55efd31966f0 con 0x55efd2e7c1b0 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55efd3185b20 con 0x55efd2e532f0 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4f06414740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55efd3189930 con 0x55efd2e5b4d0 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4efd969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd3223e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4efd168640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e532f0 0x55efd32244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4efd969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd3223e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41234/0 (socket says 172.21.15.92:41234) 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4efd969640 1 -- 172.21.15.92:0/3824850982 learned_addr learned my addr 172.21.15.92:0/3824850982 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.275+0000 7f4efc967640 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3225010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd969640 1 -- 172.21.15.92:0/3824850982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 msgr2=0x55efd3225010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd969640 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3225010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd969640 1 -- 172.21.15.92:0/3824850982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e532f0 msgr2=0x55efd32244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd969640 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e532f0 0x55efd32244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd969640 1 -- 172.21.15.92:0/3824850982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd314b530 con 0x55efd2e7c1b0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4eee7fc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4ee40b6590 0x7f4ee40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd969640 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd3223e50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009930 tx=0x7f4ef0009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5068d80ae50b12b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4efd168640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4ee40b6590 0x7f4ee40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4ee37fe640 1 -- 172.21.15.92:0/3824850982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0031750 con 0x55efd2e7c1b0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4eee7fc640 1 -- 172.21.15.92:0/2143971634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4ef40e7040 con 0x7f4ef81025c0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4ee37fe640 1 -- 172.21.15.92:0/3824850982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef0036040 con 0x55efd2e7c1b0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.276+0000 7f4ee37fe640 1 -- 172.21.15.92:0/3824850982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef002f620 con 0x55efd2e7c1b0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 -- 172.21.15.92:0/3824850982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 msgr2=0x55efd3223e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4efd168640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4ee40b6590 0x7f4ee40b8aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40aa420 tx=0x7f4ef40eab20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd3223e50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009930 tx=0x7f4ef0009960 comp rx=0 tx=0).stop 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 -- 172.21.15.92:0/3824850982 shutdown_connections 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3225010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e532f0 0x55efd32244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 --2- 172.21.15.92:0/3824850982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd3223e50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 -- 172.21.15.92:0/3824850982 >> 172.21.15.92:0/3824850982 conn(0x55efd2e70300 msgr2=0x55efd321f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 -- 172.21.15.92:0/3824850982 shutdown_connections 2024-09-21T02:11:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.277+0000 7f4f06414740 1 -- 172.21.15.92:0/3824850982 wait complete. 2024-09-21T02:11:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 -- start start 2024-09-21T02:11:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 Processor -- start 2024-09-21T02:11:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd322d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e5b4d0 0x55efd322dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd322e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55efd30bdab0 con 0x55efd2e7c1b0 2024-09-21T02:11:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55efd312b730 con 0x55efd2e5b4d0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55efd3225ad0 con 0x55efd2e532f0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4f06414740 10 client.0 _ll_register_callbacks cb 0x55efd317f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4efd969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd322e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4efd969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd322e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41238/0 (socket says 172.21.15.92:41238) 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4efd969640 1 -- 172.21.15.92:0/623716643 learned_addr learned my addr 172.21.15.92:0/623716643 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.278+0000 7f4efd969640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd322d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efc967640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e5b4d0 0x55efd322dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efd969640 1 -- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 msgr2=0x55efd322d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efd969640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd322d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efd969640 1 -- 172.21.15.92:0/623716643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e5b4d0 msgr2=0x55efd322dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efd969640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e5b4d0 0x55efd322dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efd969640 1 -- 172.21.15.92:0/623716643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd2ee7790 con 0x55efd2e7c1b0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4efd969640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd322e770 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0004400 tx=0x7f4ef003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e9c92a9986416e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4ee0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/623716643 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0059060 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/623716643 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4ef0003970 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef0036040 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/623716643 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0003b70 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4f06414740 10 client.17232 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55efd2f60620 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/623716643 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef0036040 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/623716643 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4ef002fb70 con 0x55efd2e7c1b0 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.279+0000 7f4f06414740 10 client.17232 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4f06414740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4f06414740 20 client.17232 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4f06414740 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e3ad80 0x55efd323a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4f06414740 4 client.17232 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efd3198640 con 0x55efd2e3ad80 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4efc967640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e3ad80 0x55efd323a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4efc967640 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e3ad80 0x55efd323a830 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9854391321c7c95 server_cookie=11f108d70732e734 in_seq=0 out_seq=0 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.280+0000 7f4ee0ff9640 10 client.17232 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/623716643 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efd3198640 con 0x55efd2e3ad80 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4ee0ff9640 10 client.17232 handle_command_reply: tid=0 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 client.17232 shutdown 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 msgr2=0x55efd322e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd322e770 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0004400 tx=0x7f4ef003e040 comp rx=0 tx=0).stop 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 shutdown_connections 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e3ad80 0x55efd323a830 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e7c1b0 0x55efd322e770 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e5b4d0 0x55efd322dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 --2- 172.21.15.92:0/623716643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd322d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4ee0ff9640 0 client.17232 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.281+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 >> 172.21.15.92:0/623716643 conn(0x55efd2e70300 msgr2=0x55efd3224ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.282+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 shutdown_connections 2024-09-21T02:11:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.282+0000 7f4f06414740 1 -- 172.21.15.92:0/623716643 wait complete. 2024-09-21T02:11:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.282+0000 7f4f06414740 20 client.17232 trim_cache size 0 max 16384 2024-09-21T02:11:03.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:11:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.296+0000 7f4f06414740 1 Processor -- start 2024-09-21T02:11:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.296+0000 7f4f06414740 1 -- start start 2024-09-21T02:11:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4f06414740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 0x55efd3223df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4f06414740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3224330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4f06414740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3225f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4f06414740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81aafe0 con 0x55efd2e532f0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4f06414740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81acb20 con 0x55efd2e7c1b0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4f06414740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ab6b0 con 0x55efd2e5b4d0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 0x55efd3223df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 0x55efd3223df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41240/0 (socket says 172.21.15.92:41240) 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 -- 172.21.15.92:0/937089487 learned_addr learned my addr 172.21.15.92:0/937089487 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efc967640 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3224330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 -- 172.21.15.92:0/937089487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 msgr2=0x55efd3224330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3224330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 -- 172.21.15.92:0/937089487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 msgr2=0x55efd3225f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3225f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 -- 172.21.15.92:0/937089487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd322b710 con 0x55efd2e532f0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4efd969640 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 0x55efd3223df0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009880 tx=0x7f4ef0009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c201a878edd6d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4ee2ffd640 1 -- 172.21.15.92:0/937089487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0003d10 con 0x55efd2e532f0 2024-09-21T02:11:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4ee2ffd640 1 -- 172.21.15.92:0/937089487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef0036070 con 0x55efd2e532f0 2024-09-21T02:11:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.297+0000 7f4ee2ffd640 1 -- 172.21.15.92:0/937089487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0030a40 con 0x55efd2e532f0 2024-09-21T02:11:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.298+0000 7f4f06414740 1 -- 172.21.15.92:0/937089487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 msgr2=0x55efd3223df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.298+0000 7f4f06414740 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 0x55efd3223df0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009880 tx=0x7f4ef0009930 comp rx=0 tx=0).stop 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.298+0000 7f4f06414740 1 -- 172.21.15.92:0/937089487 shutdown_connections 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.298+0000 7f4f06414740 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3225f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.298+0000 7f4f06414740 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e5b4d0 0x55efd3224330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.298+0000 7f4f06414740 1 --2- 172.21.15.92:0/937089487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e532f0 0x55efd3223df0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- 172.21.15.92:0/937089487 >> 172.21.15.92:0/937089487 conn(0x55efd2e70300 msgr2=0x55efd322c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- 172.21.15.92:0/937089487 shutdown_connections 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- 172.21.15.92:0/937089487 wait complete. 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- start start 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 Processor -- start 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd32329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 0x55efd3233090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3233bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81ac450 con 0x55efd2e5b4d0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81abd80 con 0x55efd2e7c1b0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81a8fc0 con 0x55efd2e532f0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.299+0000 7f4f06414740 10 client.0 _ll_register_callbacks cb 0x7f4ef40ea9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 0x55efd3233090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 0x55efd3233090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41250/0 (socket says 172.21.15.92:41250) 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 -- 172.21.15.92:0/3137422397 learned_addr learned my addr 172.21.15.92:0/3137422397 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efd969640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd32329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efd969640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3233bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 -- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 msgr2=0x55efd32329d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd32329d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 -- 172.21.15.92:0/3137422397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 msgr2=0x55efd3233bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3233bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 -- 172.21.15.92:0/3137422397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd3234310 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4efc967640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 0x55efd3233090 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005ea0 tx=0x7f4ee80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c13de0c2871184 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4ee0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/3137422397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80055e0 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/3137422397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4ee80047c0 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef0036070 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/3137422397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ee8029070 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4f06414740 10 client.17238 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.300+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55efd3235700 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/3137422397 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef0036070 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/3137422397 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4ee8023ad0 con 0x55efd2e5b4d0 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4f06414740 10 client.17238 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4f06414740 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4f06414740 20 client.17238 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4f06414740 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e101c0 0x55efd32a7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4f06414740 4 client.17238 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efd31940f0 con 0x55efd2e101c0 2024-09-21T02:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4efd969640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e101c0 0x55efd32a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4efd969640 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e101c0 0x55efd32a7790 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631d285c9216ae26 server_cookie=96dc4e1d57aa1351 in_seq=0 out_seq=0 2024-09-21T02:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.301+0000 7f4ee0ff9640 10 client.17238 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4ee0ff9640 1 -- 172.21.15.92:0/3137422397 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efd31940f0 con 0x55efd2e101c0 2024-09-21T02:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4ee0ff9640 10 client.17238 handle_command_reply: tid=0 2024-09-21T02:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 client.17238 shutdown 2024-09-21T02:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 msgr2=0x55efd3233090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 0x55efd3233090 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005ea0 tx=0x7f4ee80118d0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 shutdown_connections 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55efd2e101c0 0x55efd32a7790 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55efd2e7c1b0 0x55efd3233bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55efd2e5b4d0 0x55efd3233090 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4ee0ff9640 0 client.17238 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 --2- 172.21.15.92:0/3137422397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efd2e532f0 0x55efd32329d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 >> 172.21.15.92:0/3137422397 conn(0x55efd2e70300 msgr2=0x55efd322b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 shutdown_connections 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.316+0000 7f4f06414740 1 -- 172.21.15.92:0/3137422397 wait complete. 2024-09-21T02:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4f06414740 20 client.17238 trim_cache size 0 max 16384 2024-09-21T02:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4ee40b6590 msgr2=0x7f4ee40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4ee40b6590 0x7f4ee40b8aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40aa420 tx=0x7f4ef40eab20 comp rx=0 tx=0).stop 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 msgr2=0x7f4ef806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 0x7f4ef806c210 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c970 tx=0x7f4ef400ce30 comp rx=0 tx=0).stop 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 shutdown_connections 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4ee40b6590 0x7f4ee40b8aa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8103fe0 0x7f4ef81a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ef8103640 0x7f4ef819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 --2- 172.21.15.92:0/2143971634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef81025c0 0x7f4ef806c210 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 >> 172.21.15.92:0/2143971634 conn(0x7f4ef80fdf00 msgr2=0x7f4ef8106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 shutdown_connections 2024-09-21T02:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:03.317+0000 7f4eff3cc640 1 -- 172.21.15.92:0/2143971634 wait complete. 2024-09-21T02:11:03.538 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 -- 172.21.15.92:0/2916462838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc108d70 msgr2=0x7f47cc10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 --2- 172.21.15.92:0/2916462838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc108d70 0x7f47cc10f5a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009ca0 tx=0x7f47c402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 -- 172.21.15.92:0/2916462838 shutdown_connections 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 --2- 172.21.15.92:0/2916462838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc108d70 0x7f47cc10f5a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 --2- 172.21.15.92:0/2916462838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc108420 0x7f47cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 --2- 172.21.15.92:0/2916462838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc102ae0 0x7f47cc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 -- 172.21.15.92:0/2916462838 >> 172.21.15.92:0/2916462838 conn(0x7f47cc079770 msgr2=0x7f47cc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:04.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 -- 172.21.15.92:0/2916462838 shutdown_connections 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 -- 172.21.15.92:0/2916462838 wait complete. 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.094+0000 7f47d373b640 1 Processor -- start 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 -- start start 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 0x7f47cc1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc108420 0x7f47cc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc108d70 0x7f47cc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc112a30 con 0x7f47cc102ae0 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1128b0 con 0x7f47cc108420 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d373b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc112bb0 con 0x7f47cc108d70 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 0x7f47cc1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 0x7f47cc1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41266/0 (socket says 172.21.15.92:41266) 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 -- 172.21.15.92:0/1714457505 learned_addr learned my addr 172.21.15.92:0/1714457505 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc108420 0x7f47cc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc108d70 0x7f47cc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 -- 172.21.15.92:0/1714457505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc108d70 msgr2=0x7f47cc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc108d70 0x7f47cc19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 -- 172.21.15.92:0/1714457505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc108420 msgr2=0x7f47cc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc108420 0x7f47cc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d14d7640 1 -- 172.21.15.92:0/1714457505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc19f310 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.095+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc108420 0x7f47cc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.096+0000 7f47d14d7640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 0x7f47cc1993a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009970 tx=0x7f47c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.096+0000 7f47c27fc640 1 -- 172.21.15.92:0/1714457505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c800d020 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.096+0000 7f47c27fc640 1 -- 172.21.15.92:0/1714457505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c800d1c0 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.096+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc19f600 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.096+0000 7f47c27fc640 1 -- 172.21.15.92:0/1714457505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c800cc50 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.096+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc06bb60 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.098+0000 7f47c27fc640 1 -- 172.21.15.92:0/1714457505 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f47c8007860 con 0x7f47cc102ae0 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 Processor -- start 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 -- start start 2024-09-21T02:11:04.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 0x556bb33d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556bb3347e50 con 0x556bb302a070 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556bb3337280 con 0x556bb30011b0 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d9caf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556bb333b090 con 0x556bb3009390 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47c27fc640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b80b6510 0x7f47b80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d0cd6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 0x556bb33d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d1cd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d14d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d0cd6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 0x556bb33d20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:59330/0 (socket says 172.21.15.92:59330) 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d0cd6640 1 -- 172.21.15.92:0/2723228138 learned_addr learned my addr 172.21.15.92:0/2723228138 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.099+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b80b6510 0x7f47b80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47c27fc640 1 -- 172.21.15.92:0/1714457505 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47c80e8800 con 0x7f47cc102ae0 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d0cd6640 1 -- 172.21.15.92:0/2723228138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 msgr2=0x556bb33d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d0cd6640 1 -- 172.21.15.92:0/2723228138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 msgr2=0x556bb33d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d0cd6640 1 -- 172.21.15.92:0/2723228138 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb3095650 con 0x556bb3009390 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b80b6510 0x7f47b80b8a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005e30 tx=0x7f47c4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 0x556bb33d20d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00bcd0 tx=0x7f47bc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadda0d48d609b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47b77fe640 1 -- 172.21.15.92:0/2723228138 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc015070 con 0x556bb3009390 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47b77fe640 1 -- 172.21.15.92:0/2723228138 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0118f0 con 0x556bb3009390 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.100+0000 7f47b77fe640 1 -- 172.21.15.92:0/2723228138 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc010930 con 0x556bb3009390 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 -- 172.21.15.92:0/2723228138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 msgr2=0x556bb33d20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 0x556bb33d20d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00bcd0 tx=0x7f47bc00f730 comp rx=0 tx=0).stop 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 -- 172.21.15.92:0/2723228138 shutdown_connections 2024-09-21T02:11:04.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb3009390 0x556bb33d20d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 --2- 172.21.15.92:0/2723228138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 -- 172.21.15.92:0/2723228138 >> 172.21.15.92:0/2723228138 conn(0x556bb301e460 msgr2=0x556bb33cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 -- 172.21.15.92:0/2723228138 shutdown_connections 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.101+0000 7f47d9caf740 1 -- 172.21.15.92:0/2723228138 wait complete. 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 -- start start 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 Processor -- start 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33dd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb302a070 0x556bb33e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556bb3198700 con 0x556bb3009390 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556bb32d2110 con 0x556bb30011b0 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556bb33dfe30 con 0x556bb302a070 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d9caf740 10 client.0 _ll_register_callbacks cb 0x556bb32eb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:04.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d0cd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d0cd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33dd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41282/0 (socket says 172.21.15.92:41282) 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d0cd6640 1 -- 172.21.15.92:0/622750032 learned_addr learned my addr 172.21.15.92:0/622750032 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb302a070 0x556bb33e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.102+0000 7f47d14d7640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d0cd6640 1 -- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb302a070 msgr2=0x556bb33e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb302a070 0x556bb33e9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d0cd6640 1 -- 172.21.15.92:0/622750032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 msgr2=0x556bb33dcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33dcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d0cd6640 1 -- 172.21.15.92:0/622750032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb32ffc40 con 0x556bb3009390 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33dd250 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00bca0 tx=0x7f47bc00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a339367379394d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47b4ff9640 1 -- 172.21.15.92:0/622750032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc028060 con 0x556bb3009390 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47b4ff9640 1 -- 172.21.15.92:0/622750032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47bc011aa0 con 0x556bb3009390 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc0118f0 con 0x556bb3009390 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47b4ff9640 1 -- 172.21.15.92:0/622750032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47bc015070 con 0x556bb3009390 2024-09-21T02:11:04.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d9caf740 10 client.17250 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bb33ebb40 con 0x556bb3009390 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47b4ff9640 1 -- 172.21.15.92:0/622750032 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0118f0 con 0x556bb3009390 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47b4ff9640 1 -- 172.21.15.92:0/622750032 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47bc0047e0 con 0x556bb3009390 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d9caf740 10 client.17250 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d9caf740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.103+0000 7f47d9caf740 20 client.17250 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.104+0000 7f47d9caf740 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fe89f0 0x556bb33eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.104+0000 7f47d9caf740 4 client.17250 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.104+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bb3349da0 con 0x556bb2fe89f0 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.104+0000 7f47d14d7640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fe89f0 0x556bb33eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.104+0000 7f47d14d7640 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fe89f0 0x556bb33eeea0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b39dcd4dad89f5 server_cookie=90d11b3d5d64b85e in_seq=0 out_seq=0 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.104+0000 7f47b4ff9640 10 client.17250 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47b4ff9640 1 -- 172.21.15.92:0/622750032 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bb3349da0 con 0x556bb2fe89f0 2024-09-21T02:11:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47b4ff9640 10 client.17250 handle_command_reply: tid=0 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 client.17250 shutdown 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 msgr2=0x556bb33dd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33dd250 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00bca0 tx=0x7f47bc00b7e0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 shutdown_connections 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fe89f0 0x556bb33eeea0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb302a070 0x556bb33e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47b4ff9640 0 client.17250 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33dd250 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 --2- 172.21.15.92:0/622750032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb30011b0 0x556bb33dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.105+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 >> 172.21.15.92:0/622750032 conn(0x556bb301e460 msgr2=0x556bb33d4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.106+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 shutdown_connections 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.106+0000 7f47d9caf740 1 -- 172.21.15.92:0/622750032 wait complete. 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.106+0000 7f47d9caf740 20 client.17250 trim_cache size 0 max 16384 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 Processor -- start 2024-09-21T02:11:04.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 -- start start 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb302a070 0x556bb33cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ad2a0 con 0x556bb3009390 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1aede0 con 0x556bb302a070 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d9caf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ad970 con 0x556bb30011b0 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41284/0 (socket says 172.21.15.92:41284) 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 -- 172.21.15.92:0/1477981264 learned_addr learned my addr 172.21.15.92:0/1477981264 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d14d7640 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb302a070 0x556bb33cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 -- 172.21.15.92:0/1477981264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 msgr2=0x556bb33de940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33de940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 -- 172.21.15.92:0/1477981264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb302a070 msgr2=0x556bb33cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb302a070 0x556bb33cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d0cd6640 1 -- 172.21.15.92:0/1477981264 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb33d3940 con 0x556bb3009390 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.123+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb302a070 0x556bb33cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.124+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33ceb70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f47bc010580 tx=0x7f47bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73028f778e295d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.124+0000 7f47b6ffd640 1 -- 172.21.15.92:0/1477981264 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc028070 con 0x556bb3009390 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.124+0000 7f47b6ffd640 1 -- 172.21.15.92:0/1477981264 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0105b0 con 0x556bb3009390 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.124+0000 7f47b6ffd640 1 -- 172.21.15.92:0/1477981264 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc010bc0 con 0x556bb3009390 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.124+0000 7f47d9caf740 1 -- 172.21.15.92:0/1477981264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 msgr2=0x556bb33ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33ceb70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f47bc010580 tx=0x7f47bc00f730 comp rx=0 tx=0).stop 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- 172.21.15.92:0/1477981264 shutdown_connections 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb302a070 0x556bb33cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb3009390 0x556bb33ceb70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- 172.21.15.92:0/1477981264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- 172.21.15.92:0/1477981264 >> 172.21.15.92:0/1477981264 conn(0x556bb301e460 msgr2=0x556bb33de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- 172.21.15.92:0/1477981264 shutdown_connections 2024-09-21T02:11:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- 172.21.15.92:0/1477981264 wait complete. 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- start start 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 Processor -- start 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb3009390 0x556bb33efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ae710 con 0x556bb302a070 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ae040 con 0x556bb3009390 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.125+0000 7f47d9caf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1ab280 con 0x556bb30011b0 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d9caf740 10 client.0 _ll_register_callbacks cb 0x7f47c4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41294/0 (socket says 172.21.15.92:41294) 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 -- 172.21.15.92:0/3541867399 learned_addr learned my addr 172.21.15.92:0/3541867399 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d14d7640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb3009390 0x556bb33efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 -- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 msgr2=0x556bb33f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 -- 172.21.15.92:0/3541867399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb3009390 msgr2=0x556bb33efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb3009390 0x556bb33efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 -- 172.21.15.92:0/3541867399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb33f0a80 con 0x556bb302a070 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d0cd6640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb3009390 0x556bb33efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d1cd8640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33f0310 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f47c4003c20 tx=0x7f47c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14476b3b6884c827 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47b4ff9640 1 -- 172.21.15.92:0/3541867399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4032c20 con 0x556bb302a070 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47b4ff9640 1 -- 172.21.15.92:0/3541867399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f47c40044c0 con 0x556bb302a070 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc0105b0 con 0x556bb302a070 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47b4ff9640 1 -- 172.21.15.92:0/3541867399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47c4031900 con 0x556bb302a070 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.126+0000 7f47d9caf740 10 client.17256 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:04.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bb34595e0 con 0x556bb302a070 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47b4ff9640 1 -- 172.21.15.92:0/3541867399 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0105b0 con 0x556bb302a070 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47b4ff9640 1 -- 172.21.15.92:0/3541867399 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f47c4053070 con 0x556bb302a070 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 10 client.17256 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 20 client.17256 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fbdda0 0x556bb345be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 4 client.17256 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556bb3345850 con 0x556bb2fbdda0 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.127+0000 7f47d14d7640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fbdda0 0x556bb345be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47b4ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d14d7640 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fbdda0 0x556bb345be00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e042bbb2f97b245 server_cookie=88736909801e90c1 in_seq=0 out_seq=0 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47b4ff9640 1 -- 172.21.15.92:0/3541867399 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x556bb3345850 con 0x556bb2fbdda0 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47b4ff9640 10 client.17256 handle_command_reply: tid=0 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 client.17256 shutdown 2024-09-21T02:11:04.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 msgr2=0x556bb33f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33f0310 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f47c4003c20 tx=0x7f47c40095c0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 shutdown_connections 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556bb2fbdda0 0x556bb345be00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556bb302a070 0x556bb33f0310 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556bb3009390 0x556bb33efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 --2- 172.21.15.92:0/3541867399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb30011b0 0x556bb33f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 >> 172.21.15.92:0/3541867399 conn(0x556bb301e460 msgr2=0x556bb33de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47b4ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:11:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:11:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:11:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:11:04.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 shutdown_connections 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 1 -- 172.21.15.92:0/3541867399 wait complete. 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.128+0000 7f47d9caf740 20 client.17256 trim_cache size 0 max 16384 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b80b6510 msgr2=0x7f47b80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b80b6510 0x7f47b80b8a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005e30 tx=0x7f47c4005fd0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 msgr2=0x7f47cc1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 0x7f47cc1993a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009970 tx=0x7f47c8009e30 comp rx=0 tx=0).stop 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 shutdown_connections 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f47b80b6510 0x7f47b80b8a20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47cc108d70 0x7f47cc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47cc108420 0x7f47cc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 --2- 172.21.15.92:0/1714457505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47cc102ae0 0x7f47cc1993a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 >> 172.21.15.92:0/1714457505 conn(0x7f47cc079770 msgr2=0x7f47cc1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 shutdown_connections 2024-09-21T02:11:04.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:04.129+0000 7f47d373b640 1 -- 172.21.15.92:0/1714457505 wait complete. 2024-09-21T02:11:04.751 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:04 smithi092 ceph-mon[31190]: pgmap v543: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:11:04.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:04 smithi136 ceph-mon[34754]: pgmap v543: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:11:04.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:04 smithi099 ceph-mon[34404]: pgmap v543: 97 pgs: 97 active+clean; 4.7 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:11:04.900 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:11:04.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:11:05.187 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/1793078183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0109df0 msgr2=0x7fcbf0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/1793078183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0109df0 0x7fcbf0110620 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009ca0 tx=0x7fcbe802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/1793078183 shutdown_connections 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/1793078183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0109df0 0x7fcbf0110620 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/1793078183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf01094a0 0x7fcbf01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/1793078183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbf0103b60 0x7fcbf0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/1793078183 >> 172.21.15.92:0/1793078183 conn(0x7fcbf007b850 msgr2=0x7fcbf007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/1793078183 shutdown_connections 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.688+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/1793078183 wait complete. 2024-09-21T02:11:05.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 Processor -- start 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 -- start start 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 0x7fcbf019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbf01094a0 0x7fcbf019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf0109df0 0x7fcbf019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf0113b10 con 0x7fcbf0103b60 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf0113990 con 0x7fcbf01094a0 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbf4bf3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf0113c90 con 0x7fcbf0109df0 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 0x7fcbf019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 0x7fcbf019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41316/0 (socket says 172.21.15.92:41316) 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.689+0000 7fcbee59c640 1 -- 172.21.15.92:0/4074554934 learned_addr learned my addr 172.21.15.92:0/4074554934 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbeed9d640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf0109df0 0x7fcbf019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbedd9b640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbf01094a0 0x7fcbf019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbee59c640 1 -- 172.21.15.92:0/4074554934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf0109df0 msgr2=0x7fcbf019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:05.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbee59c640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf0109df0 0x7fcbf019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbee59c640 1 -- 172.21.15.92:0/4074554934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbf01094a0 msgr2=0x7fcbf019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbee59c640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbf01094a0 0x7fcbf019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbee59c640 1 -- 172.21.15.92:0/4074554934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf01a0370 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbee59c640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 0x7fcbf019a4c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000b4a0 tx=0x7fcbe000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0014070 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe0010ac0 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf01a0660 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0017c30 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.690+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf01a6e80 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.691+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf0103f70 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.693+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fcbe0017070 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.693+0000 7fcbdf7fe640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcbd80b6540 0x7fcbd80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.694+0000 7fcbedd9b640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcbd80b6540 0x7fcbd80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.694+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcbe00e6600 con 0x7fcbf0103b60 2024-09-21T02:11:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.694+0000 7fcbedd9b640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcbd80b6540 0x7fcbd80b8a50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0109040 tx=0x7fcbe4005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:05.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.698+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcbe00ace70 con 0x7fcbf0103b60 2024-09-21T02:11:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.910+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbf019b5f0 con 0x7fcbf0103b60 2024-09-21T02:11:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.911+0000 7fcbdf7fe640 1 -- 172.21.15.92:0/4074554934 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7fcbe00b1250 con 0x7fcbf0103b60 2024-09-21T02:11:05.912 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:11:05.912 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:11:05.912 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcbd80b6540 msgr2=0x7fcbd80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcbd80b6540 0x7fcbd80b8a50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0109040 tx=0x7fcbe4005cc0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 msgr2=0x7fcbf019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 0x7fcbf019a4c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000b4a0 tx=0x7fcbe000b960 comp rx=0 tx=0).stop 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 shutdown_connections 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcbd80b6540 0x7fcbd80b8a50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbf0109df0 0x7fcbf019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbf01094a0 0x7fcbf019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 --2- 172.21.15.92:0/4074554934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbf0103b60 0x7fcbf019a4c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:05.914 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 >> 172.21.15.92:0/4074554934 conn(0x7fcbf007b850 msgr2=0x7fcbf01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 shutdown_connections 2024-09-21T02:11:05.915 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:05.913+0000 7fcbf4bf3640 1 -- 172.21.15.92:0/4074554934 wait complete. 2024-09-21T02:11:06.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:11:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:06 smithi136 ceph-mon[34754]: pgmap v544: 97 pgs: 97 active+clean; 736 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-09-21T02:11:06.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:06 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/4074554934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:11:06.804 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:06 smithi092 ceph-mon[31190]: pgmap v544: 97 pgs: 97 active+clean; 736 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-09-21T02:11:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:06 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/4074554934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:11:06.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:06 smithi099 ceph-mon[34404]: pgmap v544: 97 pgs: 97 active+clean; 736 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-09-21T02:11:06.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:06 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/4074554934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 -- 172.21.15.92:0/2852550380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 msgr2=0x7fac4c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 --2- 172.21.15.92:0/2852550380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c103f70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009900 tx=0x7fac3c02a410 comp rx=0 tx=0).stop 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 -- 172.21.15.92:0/2852550380 shutdown_connections 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 --2- 172.21.15.92:0/2852550380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 0x7fac4c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 --2- 172.21.15.92:0/2852550380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac4c1094a0 0x7fac4c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 --2- 172.21.15.92:0/2852550380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c103f70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 -- 172.21.15.92:0/2852550380 >> 172.21.15.92:0/2852550380 conn(0x7fac4c07b850 msgr2=0x7fac4c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 -- 172.21.15.92:0/2852550380 shutdown_connections 2024-09-21T02:11:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.292+0000 7fac52e05640 1 -- 172.21.15.92:0/2852550380 wait complete. 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 Processor -- start 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 -- start start 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac4c1094a0 0x7fac4c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 0x7fac4c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c113b10 con 0x7fac4c103b60 2024-09-21T02:11:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c113990 con 0x7fac4c1094a0 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac52e05640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c113c90 con 0x7fac4c109df0 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac50ba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac50ba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41344/0 (socket says 172.21.15.92:41344) 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac50ba1640 1 -- 172.21.15.92:0/4011025543 learned_addr learned my addr 172.21.15.92:0/4011025543 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac4bfff640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac4c1094a0 0x7fac4c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac513a2640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 0x7fac4c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac50ba1640 1 -- 172.21.15.92:0/4011025543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 msgr2=0x7fac4c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac50ba1640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 0x7fac4c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.293+0000 7fac50ba1640 1 -- 172.21.15.92:0/4011025543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac4c1094a0 msgr2=0x7fac4c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac50ba1640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac4c1094a0 0x7fac4c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac50ba1640 1 -- 172.21.15.92:0/4011025543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c1a03b0 con 0x7fac4c103b60 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac513a2640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 0x7fac4c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac50ba1640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c19a500 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fac3c006010 tx=0x7fac3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac49ffb640 1 -- 172.21.15.92:0/4011025543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03f030 con 0x7fac4c103b60 2024-09-21T02:11:07.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac49ffb640 1 -- 172.21.15.92:0/4011025543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c003800 con 0x7fac4c103b60 2024-09-21T02:11:07.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c1a0640 con 0x7fac4c103b60 2024-09-21T02:11:07.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac49ffb640 1 -- 172.21.15.92:0/4011025543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c035070 con 0x7fac4c103b60 2024-09-21T02:11:07.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.294+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c1a6ec0 con 0x7fac4c103b60 2024-09-21T02:11:07.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac49ffb640 1 -- 172.21.15.92:0/4011025543 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fac3c003cb0 con 0x7fac4c103b60 2024-09-21T02:11:07.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 Processor -- start 2024-09-21T02:11:07.299 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 -- start start 2024-09-21T02:11:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 0x55bb923eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb9201b1b0 0x55bb923eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92023390 0x55bb923ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bb92361ea0 con 0x55bb92044070 2024-09-21T02:11:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb923512d0 con 0x55bb92023390 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac59379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb923550e0 con 0x55bb9201b1b0 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac513a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 0x55bb923eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.297+0000 7fac513a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 0x55bb923eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41354/0 (socket says 172.21.15.92:41354) 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 -- 172.21.15.92:0/2382859086 learned_addr learned my addr 172.21.15.92:0/2382859086 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac49ffb640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac380b65c0 0x7fac380b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac50ba1640 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb9201b1b0 0x55bb923eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac380b65c0 0x7fac380b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92023390 0x55bb923ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 -- 172.21.15.92:0/2382859086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb9201b1b0 msgr2=0x55bb923eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb9201b1b0 0x55bb923eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 -- 172.21.15.92:0/2382859086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92023390 msgr2=0x55bb923ebd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92023390 0x55bb923ebd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 -- 172.21.15.92:0/2382859086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb920af650 con 0x55bb92044070 2024-09-21T02:11:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92023390 0x55bb923ebd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac49ffb640 1 -- 172.21.15.92:0/4011025543 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac3c107d40 con 0x7fac4c103b60 2024-09-21T02:11:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 0x55bb923eb2c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fac44010a30 tx=0x7fac44010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f08118c7967e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.298+0000 7fac513a2640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac380b65c0 0x7fac380b8ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac44013880 tx=0x7fac44019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac42ffd640 1 -- 172.21.15.92:0/2382859086 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44020070 con 0x55bb92044070 2024-09-21T02:11:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac42ffd640 1 -- 172.21.15.92:0/2382859086 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4400ede0 con 0x55bb92044070 2024-09-21T02:11:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac42ffd640 1 -- 172.21.15.92:0/2382859086 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4400d610 con 0x55bb92044070 2024-09-21T02:11:07.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac59379740 1 -- 172.21.15.92:0/2382859086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 msgr2=0x55bb923eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac59379740 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 0x55bb923eb2c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fac44010a30 tx=0x7fac44010ef0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac59379740 1 -- 172.21.15.92:0/2382859086 shutdown_connections 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac59379740 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92023390 0x55bb923ebd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac59379740 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb9201b1b0 0x55bb923eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.299+0000 7fac59379740 1 --2- 172.21.15.92:0/2382859086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92044070 0x55bb923eb2c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 -- 172.21.15.92:0/2382859086 >> 172.21.15.92:0/2382859086 conn(0x55bb92038460 msgr2=0x55bb923e91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 -- 172.21.15.92:0/2382859086 shutdown_connections 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 -- 172.21.15.92:0/2382859086 wait complete. 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 -- start start 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 Processor -- start 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 0x55bb923f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92023390 0x55bb923f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.300+0000 7fac59379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92044070 0x55bb92402d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac59379740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bb921b2700 con 0x55bb9201b1b0 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac59379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb922ec160 con 0x55bb92044070 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac59379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb923f9a40 con 0x55bb92023390 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac59379740 10 client.0 _ll_register_callbacks cb 0x55bb923059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 0x55bb923f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92023390 0x55bb923f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 0x55bb923f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41364/0 (socket says 172.21.15.92:41364) 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92023390 0x55bb923f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:59424/0 (socket says 172.21.15.92:59424) 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 -- 172.21.15.92:0/4104231560 learned_addr learned my addr 172.21.15.92:0/4104231560 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac513a2640 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92044070 0x55bb92402d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 -- 172.21.15.92:0/4104231560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92023390 msgr2=0x55bb923f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92023390 0x55bb923f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 -- 172.21.15.92:0/4104231560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92044070 msgr2=0x55bb92402d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92044070 0x55bb92402d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 -- 172.21.15.92:0/4104231560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb92319c90 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac50ba1640 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 0x55bb923f69b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0ca420 tx=0x7fac3c099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3810825ea77d0945 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac2bfff640 1 -- 172.21.15.92:0/4104231560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c049070 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac2bfff640 1 -- 172.21.15.92:0/4104231560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac3c0b50b0 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.301+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac4400ede0 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac2bfff640 1 -- 172.21.15.92:0/4104231560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac3c0efe60 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 10 client.17274 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb92405780 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac2bfff640 1 -- 172.21.15.92:0/4104231560 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4400ede0 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac2bfff640 1 -- 172.21.15.92:0/4104231560 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac3c0d73e0 con 0x55bb9201b1b0 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 10 client.17274 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 10 client.17274 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 20 client.17274 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb920029f0 0x55bb92408c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 4 client.17274 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb92363df0 con 0x55bb920029f0 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.302+0000 7fac4bfff640 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb920029f0 0x55bb92408c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac2bfff640 10 client.17274 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac4bfff640 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb920029f0 0x55bb92408c40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=963952fec20ce059 server_cookie=cfb7c264939b041b in_seq=0 out_seq=0 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac2bfff640 1 -- 172.21.15.92:0/4104231560 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb92363df0 con 0x55bb920029f0 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac2bfff640 10 client.17274 handle_command_reply: tid=0 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 client.17274 shutdown 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 msgr2=0x55bb923f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 0x55bb923f69b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0ca420 tx=0x7fac3c099a50 comp rx=0 tx=0).stop 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 shutdown_connections 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb920029f0 0x55bb92408c40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb92044070 0x55bb92402d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92023390 0x55bb923f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 --2- 172.21.15.92:0/4104231560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb9201b1b0 0x55bb923f69b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.303+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 >> 172.21.15.92:0/4104231560 conn(0x55bb92038460 msgr2=0x55bb923ee4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.304+0000 7fac2bfff640 0 client.17274 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.304+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 shutdown_connections 2024-09-21T02:11:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.304+0000 7fac59379740 1 -- 172.21.15.92:0/4104231560 wait complete. 2024-09-21T02:11:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.304+0000 7fac59379740 20 client.17274 trim_cache size 0 max 16384 2024-09-21T02:11:07.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 Processor -- start 2024-09-21T02:11:07.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 -- start start 2024-09-21T02:11:07.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c1af4e0 con 0x55bb92023390 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c1b1020 con 0x55bb9201b1b0 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac59379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c1afbb0 con 0x55bb92044070 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41370/0 (socket says 172.21.15.92:41370) 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 -- 172.21.15.92:0/2999523230 learned_addr learned my addr 172.21.15.92:0/2999523230 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac50ba1640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac513a2640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 -- 172.21.15.92:0/2999523230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 msgr2=0x55bb923f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 -- 172.21.15.92:0/2999523230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 msgr2=0x55bb923eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.317+0000 7fac4bfff640 1 -- 172.21.15.92:0/2999523230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb923f7c70 con 0x55bb92023390 2024-09-21T02:11:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.318+0000 7fac513a2640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.318+0000 7fac50ba1640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.318+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923f3ba0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fac4c06c9f0 tx=0x7fac34013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866c908c695c038b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.318+0000 7fac427fc640 1 -- 172.21.15.92:0/2999523230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34013110 con 0x55bb92023390 2024-09-21T02:11:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.318+0000 7fac427fc640 1 -- 172.21.15.92:0/2999523230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac340132b0 con 0x55bb92023390 2024-09-21T02:11:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.318+0000 7fac427fc640 1 -- 172.21.15.92:0/2999523230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34011440 con 0x55bb92023390 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 -- 172.21.15.92:0/2999523230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 msgr2=0x55bb923f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923f3ba0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fac4c06c9f0 tx=0x7fac34013830 comp rx=0 tx=0).stop 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 -- 172.21.15.92:0/2999523230 shutdown_connections 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923f3ba0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 --2- 172.21.15.92:0/2999523230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 -- 172.21.15.92:0/2999523230 >> 172.21.15.92:0/2999523230 conn(0x55bb92038460 msgr2=0x55bb923ee3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 -- 172.21.15.92:0/2999523230 shutdown_connections 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 -- 172.21.15.92:0/2999523230 wait complete. 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 -- start start 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.319+0000 7fac59379740 1 Processor -- start 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c1b0950 con 0x55bb92023390 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c1b0280 con 0x55bb9201b1b0 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c1ad4c0 con 0x55bb92044070 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac59379740 10 client.0 _ll_register_callbacks cb 0x7fac3c0cdd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923fb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41376/0 (socket says 172.21.15.92:41376) 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 -- 172.21.15.92:0/2318387990 learned_addr learned my addr 172.21.15.92:0/2318387990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac50ba1640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac513a2640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 -- 172.21.15.92:0/2318387990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 msgr2=0x55bb923fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 -- 172.21.15.92:0/2318387990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 msgr2=0x55bb923fad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923fad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 -- 172.21.15.92:0/2318387990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb92472660 con 0x55bb92023390 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac513a2640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac4bfff640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923fb260 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fac340047e0 tx=0x7fac34004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ba82f5d6ef8131 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.320+0000 7fac2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac2bfff640 1 -- 172.21.15.92:0/2318387990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34028070 con 0x55bb92023390 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac2bfff640 1 -- 172.21.15.92:0/2318387990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac34005230 con 0x55bb92023390 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac340132b0 con 0x55bb92023390 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac2bfff640 1 -- 172.21.15.92:0/2318387990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac34005430 con 0x55bb92023390 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 10 client.17280 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:07.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb92473850 con 0x55bb92023390 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac2bfff640 1 -- 172.21.15.92:0/2318387990 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac340132b0 con 0x55bb92023390 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac2bfff640 1 -- 172.21.15.92:0/2318387990 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac3401c440 con 0x55bb92023390 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 10 client.17280 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 10 client.17280 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 20 client.17280 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb91fd7da0 0x55bb92475f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 4 client.17280 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb9235f8a0 con 0x55bb91fd7da0 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.321+0000 7fac50ba1640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb91fd7da0 0x55bb92475f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac2bfff640 10 client.17280 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac50ba1640 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb91fd7da0 0x55bb92475f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a93d8761681f524 server_cookie=afc1106f993a9033 in_seq=0 out_seq=0 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac2bfff640 1 -- 172.21.15.92:0/2318387990 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb9235f8a0 con 0x55bb91fd7da0 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac2bfff640 10 client.17280 handle_command_reply: tid=0 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 client.17280 shutdown 2024-09-21T02:11:07.327 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 msgr2=0x55bb923fb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923fb260 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fac340047e0 tx=0x7fac34004f90 comp rx=0 tx=0).stop 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 shutdown_connections 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55bb91fd7da0 0x55bb92475f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb92044070 0x55bb923fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bb92023390 0x55bb923fb260 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 --2- 172.21.15.92:0/2318387990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb9201b1b0 0x55bb923fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac2bfff640 0 client.17280 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 >> 172.21.15.92:0/2318387990 conn(0x55bb92038460 msgr2=0x55bb923f4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 shutdown_connections 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.322+0000 7fac59379740 1 -- 172.21.15.92:0/2318387990 wait complete. 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac59379740 20 client.17280 trim_cache size 0 max 16384 2024-09-21T02:11:07.328 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac380b65c0 msgr2=0x7fac380b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac380b65c0 0x7fac380b8ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac44013880 tx=0x7fac44019000 comp rx=0 tx=0).stop 2024-09-21T02:11:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 msgr2=0x7fac4c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c19a500 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fac3c006010 tx=0x7fac3c0042f0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 shutdown_connections 2024-09-21T02:11:07.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fac380b65c0 0x7fac380b8ad0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac4c109df0 0x7fac4c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac4c1094a0 0x7fac4c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 --2- 172.21.15.92:0/4011025543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac4c103b60 0x7fac4c19a500 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 >> 172.21.15.92:0/4011025543 conn(0x7fac4c07b850 msgr2=0x7fac4c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 shutdown_connections 2024-09-21T02:11:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:07.323+0000 7fac52e05640 1 -- 172.21.15.92:0/4011025543 wait complete. 2024-09-21T02:11:08.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:11:08.253 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:08.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:08 smithi092 ceph-mon[31190]: pgmap v545: 97 pgs: 97 active+clean; 736 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 -- 172.21.15.92:0/3734156897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 msgr2=0x7f1a64108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/3734156897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a64108830 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009900 tx=0x7f1a5802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 -- 172.21.15.92:0/3734156897 shutdown_connections 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/3734156897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 0x7f1a6410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/3734156897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a64108830 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/3734156897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 0x7f1a64102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 -- 172.21.15.92:0/3734156897 >> 172.21.15.92:0/3734156897 conn(0x7f1a640796b0 msgr2=0x7f1a64079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 -- 172.21.15.92:0/3734156897 shutdown_connections 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.707+0000 7f1a69d1a640 1 -- 172.21.15.92:0/3734156897 wait complete. 2024-09-21T02:11:08.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.708+0000 7f1a69d1a640 1 Processor -- start 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.708+0000 7f1a69d1a640 1 -- start start 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 0x7f1a6419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a6419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 0x7f1a641a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64112b20 con 0x7f1a64108420 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641129a0 con 0x7f1a64108d70 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64112ca0 con 0x7f1a64102ae0 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a6419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a6419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41388/0 (socket says 172.21.15.92:41388) 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 -- 172.21.15.92:0/2137290079 learned_addr learned my addr 172.21.15.92:0/2137290079 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a637fe640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 0x7f1a6419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a63fff640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 0x7f1a641a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 -- 172.21.15.92:0/2137290079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 msgr2=0x7f1a6419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 0x7f1a6419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 -- 172.21.15.92:0/2137290079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 msgr2=0x7f1a641a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 0x7f1a641a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 -- 172.21.15.92:0/2137290079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a641a2820 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a637fe640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 0x7f1a6419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a63fff640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 0x7f1a641a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a6419ce50 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1a58031440 tx=0x7f1a58003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a60ff9640 1 -- 172.21.15.92:0/2137290079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5803f070 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a60ff9640 1 -- 172.21.15.92:0/2137290079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a58031d60 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a641a2ab0 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.709+0000 7f1a60ff9640 1 -- 172.21.15.92:0/2137290079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58035030 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.710+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641a9270 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 Processor -- start 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a60ff9640 1 -- 172.21.15.92:0/2137290079 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1a58042070 con 0x7f1a64108420 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 -- start start 2024-09-21T02:11:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7ca070 0x55992eb716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb71be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb72120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55992eae7ea0 con 0x55992e7a11b0 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55992ead72d0 con 0x55992e7ca070 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a7028e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55992eadb0e0 con 0x55992e7a9390 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb71be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.712+0000 7f1a637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb71be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41398/0 (socket says 172.21.15.92:41398) 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 -- 172.21.15.92:0/2490699742 learned_addr learned my addr 172.21.15.92:0/2490699742 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb72120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a63fff640 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7ca070 0x55992eb716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 -- 172.21.15.92:0/2490699742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 msgr2=0x55992eb72120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb72120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 -- 172.21.15.92:0/2490699742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7ca070 msgr2=0x55992eb716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7ca070 0x55992eb716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 -- 172.21.15.92:0/2490699742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992e835650 con 0x55992e7a11b0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a637fe640 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb71be0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1a54007850 tx=0x7f1a54007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4975cf780539b2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a60ff9640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1a4c0b6510 0x7f1a4c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a49ffb640 1 -- 172.21.15.92:0/2490699742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400d950 con 0x55992e7a11b0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a49ffb640 1 -- 172.21.15.92:0/2490699742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a54018070 con 0x55992e7a11b0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a63fff640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1a4c0b6510 0x7f1a4c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a49ffb640 1 -- 172.21.15.92:0/2490699742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54013de0 con 0x55992e7a11b0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.713+0000 7f1a60ff9640 1 -- 172.21.15.92:0/2137290079 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1a58107cc0 con 0x7f1a64108420 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 -- 172.21.15.92:0/2490699742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 msgr2=0x55992eb71be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a63fff640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1a4c0b6510 0x7f1a4c0b8a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00f4c0 tx=0x7f1a5c007740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb71be0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1a54007850 tx=0x7f1a54007d10 comp rx=0 tx=0).stop 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 -- 172.21.15.92:0/2490699742 shutdown_connections 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb72120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb71be0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 --2- 172.21.15.92:0/2490699742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7ca070 0x55992eb716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 -- 172.21.15.92:0/2490699742 >> 172.21.15.92:0/2490699742 conn(0x55992e7be460 msgr2=0x55992eb6f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 -- 172.21.15.92:0/2490699742 shutdown_connections 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.714+0000 7f1a7028e740 1 -- 172.21.15.92:0/2490699742 wait complete. 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 -- start start 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 Processor -- start 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a11b0 0x55992eb7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb7d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 0x55992eb7d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55992e938700 con 0x55992e7ca070 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55992ea72160 con 0x55992e7a11b0 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55992eb7fe80 con 0x55992e7a9390 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a7028e740 10 client.0 _ll_register_callbacks cb 0x55992ea8b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a63fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 0x55992eb7d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a63fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 0x55992eb7d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41412/0 (socket says 172.21.15.92:41412) 2024-09-21T02:11:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.715+0000 7f1a63fff640 1 -- 172.21.15.92:0/1671129184 learned_addr learned my addr 172.21.15.92:0/1671129184 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a637fe640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a11b0 0x55992eb7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb7d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a63fff640 1 -- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 msgr2=0x55992eb7d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a63fff640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb7d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a63fff640 1 -- 172.21.15.92:0/1671129184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a11b0 msgr2=0x55992eb7cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a63fff640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a11b0 0x55992eb7cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a63fff640 1 -- 172.21.15.92:0/1671129184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55992ea9fc90 con 0x55992e7ca070 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb7d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a63fff640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 0x55992eb7d7c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c010eb0 tx=0x7f1a5c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34261f1494ed3237 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a42ffd640 1 -- 172.21.15.92:0/1671129184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c026680 con 0x55992e7ca070 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a42ffd640 1 -- 172.21.15.92:0/1671129184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1a5c026820 con 0x55992e7ca070 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a54018070 con 0x55992e7ca070 2024-09-21T02:11:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a42ffd640 1 -- 172.21.15.92:0/1671129184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a5c02c070 con 0x55992e7ca070 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a7028e740 10 client.17292 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.716+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55992eb8bb90 con 0x55992e7ca070 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a42ffd640 1 -- 172.21.15.92:0/1671129184 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a54018070 con 0x55992e7ca070 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a42ffd640 1 -- 172.21.15.92:0/1671129184 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a5c01db70 con 0x55992e7ca070 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a7028e740 10 client.17292 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a7028e740 10 client.17292 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a7028e740 20 client.17292 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a7028e740 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7889f0 0x55992eb8f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a7028e740 4 client.17292 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55992eae9df0 con 0x55992e7889f0 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.717+0000 7f1a637fe640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7889f0 0x55992eb8f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.718+0000 7f1a42ffd640 10 client.17292 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.718+0000 7f1a637fe640 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7889f0 0x55992eb8f170 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3fa9b9fc417a8a6 server_cookie=b741d968ab4c5c85 in_seq=0 out_seq=0 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.718+0000 7f1a42ffd640 1 -- 172.21.15.92:0/1671129184 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55992eae9df0 con 0x55992e7889f0 2024-09-21T02:11:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.718+0000 7f1a42ffd640 10 client.17292 handle_command_reply: tid=0 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.718+0000 7f1a7028e740 1 client.17292 shutdown 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 msgr2=0x55992eb7d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 0x55992eb7d7c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c010eb0 tx=0x7f1a5c004b50 comp rx=0 tx=0).stop 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 shutdown_connections 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7ca070 0x55992eb7d7c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992eb7d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7889f0 0x55992eb8f170 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a42ffd640 0 client.17292 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 --2- 172.21.15.92:0/1671129184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a11b0 0x55992eb7cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 >> 172.21.15.92:0/1671129184 conn(0x55992e7be460 msgr2=0x55992eb74830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 shutdown_connections 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 1 -- 172.21.15.92:0/1671129184 wait complete. 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.719+0000 7f1a7028e740 20 client.17292 trim_cache size 0 max 16384 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 Processor -- start 2024-09-21T02:11:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 -- start start 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7889f0 0x55992eb73090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb7e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a9390 0x55992eb6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641b17c0 con 0x55992e7a11b0 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641b3300 con 0x55992e7a9390 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a7028e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641b1e90 con 0x55992e7889f0 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb7e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb7e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41428/0 (socket says 172.21.15.92:41428) 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a62ffd640 1 -- 172.21.15.92:0/513306573 learned_addr learned my addr 172.21.15.92:0/513306573 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a637fe640 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7889f0 0x55992eb73090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a62ffd640 1 -- 172.21.15.92:0/513306573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7889f0 msgr2=0x55992eb73090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7889f0 0x55992eb73090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a62ffd640 1 -- 172.21.15.92:0/513306573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a9390 msgr2=0x55992eb6ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.732+0000 7f1a63fff640 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a9390 0x55992eb6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.733+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a9390 0x55992eb6ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.733+0000 7f1a62ffd640 1 -- 172.21.15.92:0/513306573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992eb73990 con 0x55992e7a11b0 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.733+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb7e950 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1a580cddc0 tx=0x7f1a5802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3f7d79fd916985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.733+0000 7f1a497fa640 1 -- 172.21.15.92:0/513306573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58109020 con 0x55992e7a11b0 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.733+0000 7f1a497fa640 1 -- 172.21.15.92:0/513306573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a580cd940 con 0x55992e7a11b0 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.733+0000 7f1a497fa640 1 -- 172.21.15.92:0/513306573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58049070 con 0x55992e7a11b0 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 -- 172.21.15.92:0/513306573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 msgr2=0x55992eb7e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb7e950 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1a580cddc0 tx=0x7f1a5802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 -- 172.21.15.92:0/513306573 shutdown_connections 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7a9390 0x55992eb6ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb7e950 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 --2- 172.21.15.92:0/513306573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7889f0 0x55992eb73090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 -- 172.21.15.92:0/513306573 >> 172.21.15.92:0/513306573 conn(0x55992e7be460 msgr2=0x55992eb7e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 -- 172.21.15.92:0/513306573 shutdown_connections 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 -- 172.21.15.92:0/513306573 wait complete. 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 -- start start 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.734+0000 7f1a7028e740 1 Processor -- start 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7889f0 0x55992eb81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992ebf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641b2c30 con 0x55992e7a11b0 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641b2560 con 0x55992e7889f0 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641af7a0 con 0x55992e7a9390 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a7028e740 10 client.0 _ll_register_callbacks cb 0x7f1a5c007cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7889f0 0x55992eb81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb81970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41440/0 (socket says 172.21.15.92:41440) 2024-09-21T02:11:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a62ffd640 1 -- 172.21.15.92:0/3695578900 learned_addr learned my addr 172.21.15.92:0/3695578900 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a63fff640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992ebf8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 -- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 msgr2=0x55992ebf8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992ebf8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 -- 172.21.15.92:0/3695578900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 msgr2=0x55992eb81970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb81970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 -- 172.21.15.92:0/3695578900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55992eb82260 con 0x55992e7889f0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a637fe640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7889f0 0x55992eb81430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a54004810 tx=0x7f1a54006f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a16b1842784397 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.735+0000 7f1a42ffd640 1 -- 172.21.15.92:0/3695578900 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400dd60 con 0x55992e7889f0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a580cd940 con 0x55992e7889f0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a42ffd640 1 -- 172.21.15.92:0/3695578900 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1a54013480 con 0x55992e7889f0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a42ffd640 1 -- 172.21.15.92:0/3695578900 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a54013e30 con 0x55992e7889f0 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 10 client.26330 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55992ebfa1c0 con 0x55992e7889f0 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a42ffd640 1 -- 172.21.15.92:0/3695578900 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a580cd940 con 0x55992e7889f0 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a42ffd640 1 -- 172.21.15.92:0/3695578900 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a54030970 con 0x55992e7889f0 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 10 client.26330 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 10 client.26330 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 20 client.26330 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7ca070 0x55992ebfc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 4 client.26330 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55992eae58a0 con 0x55992e7ca070 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.736+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7ca070 0x55992ebfc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a42ffd640 10 client.26330 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a62ffd640 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7ca070 0x55992ebfc670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0dcaffaaddb643a server_cookie=6c49e4109d121597 in_seq=0 out_seq=0 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a42ffd640 1 -- 172.21.15.92:0/3695578900 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55992eae58a0 con 0x55992e7ca070 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a42ffd640 10 client.26330 handle_command_reply: tid=0 2024-09-21T02:11:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 client.26330 shutdown 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7889f0 msgr2=0x55992eb81430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7889f0 0x55992eb81430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a54004810 tx=0x7f1a54006f70 comp rx=0 tx=0).stop 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 shutdown_connections 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55992e7a9390 0x55992ebf8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55992e7a11b0 0x55992eb81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55992e7ca070 0x55992ebfc670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 --2- 172.21.15.92:0/3695578900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55992e7889f0 0x55992eb81430 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a42ffd640 0 client.26330 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 >> 172.21.15.92:0/3695578900 conn(0x55992e7be460 msgr2=0x55992eb72e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 shutdown_connections 2024-09-21T02:11:08.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.737+0000 7f1a7028e740 1 -- 172.21.15.92:0/3695578900 wait complete. 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a7028e740 20 client.26330 trim_cache size 0 max 16384 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1a4c0b6510 msgr2=0x7f1a4c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1a4c0b6510 0x7f1a4c0b8a20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00f4c0 tx=0x7f1a5c007740 comp rx=0 tx=0).stop 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 msgr2=0x7f1a6419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a6419ce50 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f1a58031440 tx=0x7f1a58003d50 comp rx=0 tx=0).stop 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 shutdown_connections 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1a4c0b6510 0x7f1a4c0b8a20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a64108d70 0x7f1a641a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a64108420 0x7f1a6419ce50 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 --2- 172.21.15.92:0/2137290079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64102ae0 0x7f1a6419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 >> 172.21.15.92:0/2137290079 conn(0x7f1a640796b0 msgr2=0x7f1a64106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 shutdown_connections 2024-09-21T02:11:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:08.738+0000 7f1a69d1a640 1 -- 172.21.15.92:0/2137290079 wait complete. 2024-09-21T02:11:08.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:08 smithi136 ceph-mon[34754]: pgmap v545: 97 pgs: 97 active+clean; 736 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:11:08.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:08 smithi099 ceph-mon[34404]: pgmap v545: 97 pgs: 97 active+clean; 736 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:11:09.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:11:09.745 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.174+0000 7f56b097b640 1 -- 172.21.15.92:0/2894968834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 msgr2=0x7f56ac10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.174+0000 7f56b097b640 1 --2- 172.21.15.92:0/2894968834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac10f5a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009ca0 tx=0x7f56a402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 -- 172.21.15.92:0/2894968834 shutdown_connections 2024-09-21T02:11:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 --2- 172.21.15.92:0/2894968834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac10f5a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 --2- 172.21.15.92:0/2894968834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56ac108420 0x7f56ac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 --2- 172.21.15.92:0/2894968834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac102ae0 0x7f56ac102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 -- 172.21.15.92:0/2894968834 >> 172.21.15.92:0/2894968834 conn(0x7f56ac079770 msgr2=0x7f56ac079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 -- 172.21.15.92:0/2894968834 shutdown_connections 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.175+0000 7f56b097b640 1 -- 172.21.15.92:0/2894968834 wait complete. 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 Processor -- start 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 -- start start 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56ac102ae0 0x7f56ac19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac108420 0x7f56ac19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac112a30 con 0x7f56ac108d70 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1128b0 con 0x7f56ac108420 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56b097b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac112bb0 con 0x7f56ac102ae0 2024-09-21T02:11:10.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56aad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56aad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41456/0 (socket says 172.21.15.92:41456) 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.176+0000 7f56aad9d640 1 -- 172.21.15.92:0/2727116134 learned_addr learned my addr 172.21.15.92:0/2727116134 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac108420 0x7f56ac19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56ac102ae0 0x7f56ac19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aad9d640 1 -- 172.21.15.92:0/2727116134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56ac102ae0 msgr2=0x7f56ac19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aad9d640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56ac102ae0 0x7f56ac19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aad9d640 1 -- 172.21.15.92:0/2727116134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac108420 msgr2=0x7f56ac19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aad9d640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac108420 0x7f56ac19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aad9d640 1 -- 172.21.15.92:0/2727116134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac1a6580 con 0x7f56ac108d70 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac108420 0x7f56ac19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56aad9d640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac19bd40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098d0 tx=0x7f56a40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56a3fff640 1 -- 172.21.15.92:0/2727116134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003b70 con 0x7f56ac108d70 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac1a6810 con 0x7f56ac108d70 2024-09-21T02:11:10.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56a3fff640 1 -- 172.21.15.92:0/2727116134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a4003d10 con 0x7f56ac108d70 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56a3fff640 1 -- 172.21.15.92:0/2727116134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a402fa20 con 0x7f56ac108d70 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.177+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1a6d50 con 0x7f56ac108d70 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 Processor -- start 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56a3fff640 1 -- 172.21.15.92:0/2727116134 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f56a402fc40 con 0x7f56ac108d70 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 -- start start 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557302b8fea0 con 0x557302851390 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557302b7f2d0 con 0x557302872070 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56b6eef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557302b830e0 con 0x5573028491b0 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.180+0000 7f56a1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c19d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c19d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41460/0 (socket says 172.21.15.92:41460) 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 -- 172.21.15.92:0/1160070481 learned_addr learned my addr 172.21.15.92:0/1160070481 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56aa59c640 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 -- 172.21.15.92:0/1160070481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 msgr2=0x557302c19800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c19800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 -- 172.21.15.92:0/1160070481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 msgr2=0x557302c192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 -- 172.21.15.92:0/1160070481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573028dd650 con 0x557302851390 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a3fff640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56980b6510 0x7f56980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c19d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f56940127c0 tx=0x7f56940096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a9d4d1990001fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56980b6510 0x7f56980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a3fff640 1 -- 172.21.15.92:0/2727116134 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f56a4108710 con 0x7f56ac108d70 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a0d99640 1 -- 172.21.15.92:0/1160070481 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694012d40 con 0x557302851390 2024-09-21T02:11:10.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a0d99640 1 -- 172.21.15.92:0/1160070481 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5694010c80 con 0x557302851390 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.181+0000 7f56a0d99640 1 -- 172.21.15.92:0/1160070481 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569401eb30 con 0x557302851390 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 -- 172.21.15.92:0/1160070481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 msgr2=0x557302c19d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56980b6510 0x7f56980b8a20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x557302c17f40 tx=0x7f569c011850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c19d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f56940127c0 tx=0x7f56940096c0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 -- 172.21.15.92:0/1160070481 shutdown_connections 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c19d40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c19800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 --2- 172.21.15.92:0/1160070481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 -- 172.21.15.92:0/1160070481 >> 172.21.15.92:0/1160070481 conn(0x557302866460 msgr2=0x557302c171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 -- 172.21.15.92:0/1160070481 shutdown_connections 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.182+0000 7f56b6eef740 1 -- 172.21.15.92:0/1160070481 wait complete. 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 -- start start 2024-09-21T02:11:10.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 Processor -- start 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c24ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573029e0700 con 0x557302851390 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557302b1a160 con 0x557302872070 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557302c27aa0 con 0x5573028491b0 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56b6eef740 10 client.0 _ll_register_callbacks cb 0x557302b339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56a1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c24ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56aa59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.183+0000 7f56aa59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c249b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:59532/0 (socket says 172.21.15.92:59532) 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 -- 172.21.15.92:0/2691427576 learned_addr learned my addr 172.21.15.92:0/2691427576 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aad9d640 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 -- 172.21.15.92:0/2691427576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 msgr2=0x557302c395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 -- 172.21.15.92:0/2691427576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 msgr2=0x557302c24ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c24ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 -- 172.21.15.92:0/2691427576 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557302b47c90 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c249b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f569c01d640 tx=0x7f569c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce9068699b35755 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f5691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f5691ffb640 1 -- 172.21.15.92:0/2691427576 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c00f850 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5694010c80 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.184+0000 7f5691ffb640 1 -- 172.21.15.92:0/2691427576 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f569c00f9f0 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f5691ffb640 1 -- 172.21.15.92:0/2691427576 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f569c00fc30 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 10 client.26470 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557302c3c0d0 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f5691ffb640 1 -- 172.21.15.92:0/2691427576 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5694010c80 con 0x5573028491b0 2024-09-21T02:11:10.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f5691ffb640 1 -- 172.21.15.92:0/2691427576 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f569c024020 con 0x5573028491b0 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 10 client.26470 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 10 client.26470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 20 client.26470 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5573028309f0 0x557302c3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 4 client.26470 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557302b91df0 con 0x5573028309f0 2024-09-21T02:11:10.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.185+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5573028309f0 0x557302c3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.186+0000 7f5691ffb640 10 client.26470 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.186+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5573028309f0 0x557302c3f600 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1107fd4602ea5b5b server_cookie=df46ca8bc41fa5fc in_seq=0 out_seq=0 2024-09-21T02:11:10.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f5691ffb640 1 -- 172.21.15.92:0/2691427576 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557302b91df0 con 0x5573028309f0 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f5691ffb640 10 client.26470 handle_command_reply: tid=0 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 client.26470 shutdown 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 msgr2=0x557302c249b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c249b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f569c01d640 tx=0x7f569c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 shutdown_connections 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5573028309f0 0x557302c3f600 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302872070 0x557302c395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302851390 0x557302c24ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2691427576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573028491b0 0x557302c249b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 >> 172.21.15.92:0/2691427576 conn(0x557302866460 msgr2=0x557302c1c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f5691ffb640 0 client.26470 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 shutdown_connections 2024-09-21T02:11:10.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 1 -- 172.21.15.92:0/2691427576 wait complete. 2024-09-21T02:11:10.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.187+0000 7f56b6eef740 20 client.26470 trim_cache size 0 max 16384 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 Processor -- start 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 -- start start 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302866460 0x557302c24e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 0x557302c39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302851390 0x557302c21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1ad180 con 0x5573028491b0 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1aecc0 con 0x557302851390 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.200+0000 7f56b6eef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1ad850 con 0x557302866460 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 0x557302c39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 0x557302c39550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41480/0 (socket says 172.21.15.92:41480) 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 -- 172.21.15.92:0/2565051851 learned_addr learned my addr 172.21.15.92:0/2565051851 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302866460 0x557302c24e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56aad9d640 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302851390 0x557302c21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 -- 172.21.15.92:0/2565051851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302866460 msgr2=0x557302c24e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302866460 0x557302c24e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 -- 172.21.15.92:0/2565051851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302851390 msgr2=0x557302c21ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302851390 0x557302c21ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 -- 172.21.15.92:0/2565051851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557302c25a50 con 0x5573028491b0 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56aa59c640 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302866460 0x557302c24e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 0x557302c39550 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f569401e700 tx=0x7f5694014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f1887870a87b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f5693fff640 1 -- 172.21.15.92:0/2565051851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694035070 con 0x5573028491b0 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f5693fff640 1 -- 172.21.15.92:0/2565051851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5694010c50 con 0x5573028491b0 2024-09-21T02:11:10.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.201+0000 7f5693fff640 1 -- 172.21.15.92:0/2565051851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694024040 con 0x5573028491b0 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 -- 172.21.15.92:0/2565051851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 msgr2=0x557302c39550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 0x557302c39550 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f569401e700 tx=0x7f5694014040 comp rx=0 tx=0).stop 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 -- 172.21.15.92:0/2565051851 shutdown_connections 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557302851390 0x557302c21ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573028491b0 0x557302c39550 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 --2- 172.21.15.92:0/2565051851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302866460 0x557302c24e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 -- 172.21.15.92:0/2565051851 >> 172.21.15.92:0/2565051851 conn(0x557302c171f0 msgr2=0x557302c24760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 -- 172.21.15.92:0/2565051851 shutdown_connections 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 -- 172.21.15.92:0/2565051851 wait complete. 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 -- start start 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.202+0000 7f56b6eef740 1 Processor -- start 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 0x557302c318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302851390 0x557302c31df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302866460 0x557302ca8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1ae5f0 con 0x557302866460 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1adf20 con 0x5573028491b0 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1ab160 con 0x557302851390 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56b6eef740 10 client.0 _ll_register_callbacks cb 0x7f569c0115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aa59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 0x557302c318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302866460 0x557302ca8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aa59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 0x557302c318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:43612/0 (socket says 172.21.15.92:43612) 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aa59c640 1 -- 172.21.15.92:0/199868084 learned_addr learned my addr 172.21.15.92:0/199868084 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56a1d9b640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302851390 0x557302c31df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 -- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302851390 msgr2=0x557302c31df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302851390 0x557302c31df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 -- 172.21.15.92:0/199868084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 msgr2=0x557302c318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:10.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 0x557302c318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 -- 172.21.15.92:0/199868084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557302c326c0 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aa59c640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 0x557302c318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f56aad9d640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302866460 0x557302ca8fc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f56a4005e30 tx=0x7f56a40caf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929d56a864d5a824 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.203+0000 7f5691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f5691ffb640 1 -- 172.21.15.92:0/199868084 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a40cdff0 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f5691ffb640 1 -- 172.21.15.92:0/199868084 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f56a40d8020 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5694010c50 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f5691ffb640 1 -- 172.21.15.92:0/199868084 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56a4044e60 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 10 client.17316 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557302caa5c0 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f5691ffb640 1 -- 172.21.15.92:0/199868084 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5694010c50 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f5691ffb640 1 -- 172.21.15.92:0/199868084 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f56a40cfc40 con 0x557302866460 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 10 client.17316 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:10.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 10 client.17316 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 20 client.17316 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x557302872070 0x557302cacdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 4 client.17316 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.204+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557302b8d8a0 con 0x557302872070 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56aa59c640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x557302872070 0x557302cacdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f5691ffb640 10 client.17316 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56aa59c640 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x557302872070 0x557302cacdd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c84c457aedc3588 server_cookie=fd75d96b9256f358 in_seq=0 out_seq=0 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f5691ffb640 1 -- 172.21.15.92:0/199868084 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557302b8d8a0 con 0x557302872070 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f5691ffb640 10 client.17316 handle_command_reply: tid=0 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 client.17316 shutdown 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302866460 msgr2=0x557302ca8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302866460 0x557302ca8fc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f56a4005e30 tx=0x7f56a40caf90 comp rx=0 tx=0).stop 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 shutdown_connections 2024-09-21T02:11:10.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x557302872070 0x557302cacdd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557302866460 0x557302ca8fc0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557302851390 0x557302c31df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 --2- 172.21.15.92:0/199868084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573028491b0 0x557302c318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f5691ffb640 0 client.17316 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.205+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 >> 172.21.15.92:0/199868084 conn(0x557302c171f0 msgr2=0x557302c22e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 shutdown_connections 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b6eef740 1 -- 172.21.15.92:0/199868084 wait complete. 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b6eef740 20 client.17316 trim_cache size 0 max 16384 2024-09-21T02:11:10.212 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56980b6510 msgr2=0x7f56980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56980b6510 0x7f56980b8a20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x557302c17f40 tx=0x7f569c011850 comp rx=0 tx=0).stop 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 msgr2=0x7f56ac19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac19bd40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098d0 tx=0x7f56a40317b0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 shutdown_connections 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56980b6510 0x7f56980b8a20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56ac108d70 0x7f56ac19bd40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56ac108420 0x7f56ac19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 --2- 172.21.15.92:0/2727116134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56ac102ae0 0x7f56ac19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:10.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 >> 172.21.15.92:0/2727116134 conn(0x7f56ac079770 msgr2=0x7f56ac106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:10.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 shutdown_connections 2024-09-21T02:11:10.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:10.206+0000 7f56b097b640 1 -- 172.21.15.92:0/2727116134 wait complete. 2024-09-21T02:11:10.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:10 smithi136 ceph-mon[34754]: pgmap v546: 97 pgs: 97 active+clean; 808 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:11:10.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:10 smithi092 ceph-mon[31190]: pgmap v546: 97 pgs: 97 active+clean; 808 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:11:10.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:10 smithi099 ceph-mon[34404]: pgmap v546: 97 pgs: 97 active+clean; 808 KiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:11:10.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:11:11.183 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 -- 172.21.15.92:0/1212901186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c108d10 msgr2=0x7f267c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 --2- 172.21.15.92:0/1212901186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c108d10 0x7f267c10f540 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ca0 tx=0x7f267402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 -- 172.21.15.92:0/1212901186 shutdown_connections 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 --2- 172.21.15.92:0/1212901186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c108d10 0x7f267c10f540 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 --2- 172.21.15.92:0/1212901186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c1083c0 0x7f267c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 --2- 172.21.15.92:0/1212901186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c102a80 0x7f267c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 -- 172.21.15.92:0/1212901186 >> 172.21.15.92:0/1212901186 conn(0x7f267c079740 msgr2=0x7f267c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 -- 172.21.15.92:0/1212901186 shutdown_connections 2024-09-21T02:11:11.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 -- 172.21.15.92:0/1212901186 wait complete. 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 Processor -- start 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.699+0000 7f26828d4640 1 -- start start 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f26828d4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c102a80 0x7f267c0ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f26828d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 0x7f267c0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f26828d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c108d10 0x7f267c100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f26828d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1129e0 con 0x7f267c1083c0 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f26828d4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f267c112860 con 0x7f267c108d10 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f26828d4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f267c112b60 con 0x7f267c102a80 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 0x7f267c0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 0x7f267c0ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41516/0 (socket says 172.21.15.92:41516) 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 -- 172.21.15.92:0/4138323999 learned_addr learned my addr 172.21.15.92:0/4138323999 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f2680e71640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c108d10 0x7f267c100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267bfff640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c102a80 0x7f267c0ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 -- 172.21.15.92:0/4138323999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c102a80 msgr2=0x7f267c0ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c102a80 0x7f267c0ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 -- 172.21.15.92:0/4138323999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c108d10 msgr2=0x7f267c100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c108d10 0x7f267c100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.700+0000 7f267b7fe640 1 -- 172.21.15.92:0/4138323999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1a86d0 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.701+0000 7f267b7fe640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 0x7f267c0ffbd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f267000b450 tx=0x7f267000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.701+0000 7f26797fa640 1 -- 172.21.15.92:0/4138323999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670015020 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.701+0000 7f26797fa640 1 -- 172.21.15.92:0/4138323999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2670011d40 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.701+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1a89c0 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.701+0000 7f26797fa640 1 -- 172.21.15.92:0/4138323999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267000f340 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.701+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1a8f80 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.703+0000 7f2688e48740 1 Processor -- start 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.703+0000 7f2688e48740 1 -- start start 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2688e48740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca61c070 0x555bca9c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f26797fa640 1 -- 172.21.15.92:0/4138323999 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2670018020 con 0x7f267c1083c0 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2688e48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2688e48740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5fb390 0x555bca9c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2688e48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555bca939ea0 con 0x555bca5f31b0 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2688e48740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555bca9292d0 con 0x555bca61c070 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2688e48740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555bca92d0e0 con 0x555bca5fb390 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41526/0 (socket says 172.21.15.92:41526) 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 -- 172.21.15.92:0/1685362134 learned_addr learned my addr 172.21.15.92:0/1685362134 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267b7fe640 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5fb390 0x555bca9c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f2680e71640 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca61c070 0x555bca9c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 -- 172.21.15.92:0/1685362134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5fb390 msgr2=0x555bca9c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5fb390 0x555bca9c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 -- 172.21.15.92:0/1685362134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca61c070 msgr2=0x555bca9c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca61c070 0x555bca9c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 -- 172.21.15.92:0/1685362134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca687650 con 0x555bca5f31b0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f267bfff640 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9c3be0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f266c00cc20 tx=0x7f266c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b8fc03286fabe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.704+0000 7f26797fa640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f26640b6510 0x7f26640b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.705+0000 7f26627fc640 1 -- 172.21.15.92:0/1685362134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c014070 con 0x555bca5f31b0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.705+0000 7f2680e71640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f26640b6510 0x7f26640b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.705+0000 7f26797fa640 1 -- 172.21.15.92:0/4138323999 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f26700e6dc0 con 0x7f267c1083c0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.705+0000 7f26627fc640 1 -- 172.21.15.92:0/1685362134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0047a0 con 0x555bca5f31b0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.705+0000 7f26627fc640 1 -- 172.21.15.92:0/1685362134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c010690 con 0x555bca5f31b0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 -- 172.21.15.92:0/1685362134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 msgr2=0x555bca9c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9c3be0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f266c00cc20 tx=0x7f266c00e720 comp rx=0 tx=0).stop 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2680e71640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f26640b6510 0x7f26640b8a20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2674005cc0 tx=0x7f2674005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 -- 172.21.15.92:0/1685362134 shutdown_connections 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5fb390 0x555bca9c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9c3be0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 --2- 172.21.15.92:0/1685362134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca61c070 0x555bca9c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 -- 172.21.15.92:0/1685362134 >> 172.21.15.92:0/1685362134 conn(0x555bca610460 msgr2=0x555bca9c1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 -- 172.21.15.92:0/1685362134 shutdown_connections 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 -- 172.21.15.92:0/1685362134 wait complete. 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 -- start start 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.706+0000 7f2688e48740 1 Processor -- start 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca61c070 0x555bca9cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555bca78a700 con 0x555bca5f31b0 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555bca8c4160 con 0x555bca5fb390 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555bca9d1e80 con 0x555bca61c070 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2688e48740 10 client.0 _ll_register_callbacks cb 0x555bca8dd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41530/0 (socket says 172.21.15.92:41530) 2024-09-21T02:11:11.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 -- 172.21.15.92:0/2071720910 learned_addr learned my addr 172.21.15.92:0/2071720910 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267b7fe640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f2680e71640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca61c070 0x555bca9cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 -- 172.21.15.92:0/2071720910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca61c070 msgr2=0x555bca9cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca61c070 0x555bca9cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 -- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 msgr2=0x555bca9cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 -- 172.21.15.92:0/2071720910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bca8f1c90 con 0x555bca5f31b0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f267bfff640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9ced40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f266c0058c0 tx=0x7f266c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a32207b3935910 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f265b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.707+0000 7f265b7fe640 1 -- 172.21.15.92:0/2071720910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c027060 con 0x555bca5f31b0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266c0047a0 con 0x555bca5f31b0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f265b7fe640 1 -- 172.21.15.92:0/2071720910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f266c005230 con 0x555bca5f31b0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f265b7fe640 1 -- 172.21.15.92:0/2071720910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f266c014070 con 0x555bca5f31b0 2024-09-21T02:11:11.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 10 client.17328 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555bca9ddb50 con 0x555bca5f31b0 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f265b7fe640 1 -- 172.21.15.92:0/2071720910 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0047a0 con 0x555bca5f31b0 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f265b7fe640 1 -- 172.21.15.92:0/2071720910 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f266c022410 con 0x555bca5f31b0 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 10 client.17328 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 10 client.17328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 20 client.17328 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5da9f0 0x555bca9e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 4 client.17328 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.708+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bca93bdf0 con 0x555bca5da9f0 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.709+0000 7f267b7fe640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5da9f0 0x555bca9e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.709+0000 7f265b7fe640 10 client.17328 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:11.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.709+0000 7f267b7fe640 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5da9f0 0x555bca9e0f10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b30f7012a8d2a1b5 server_cookie=946aa0d277209821 in_seq=0 out_seq=0 2024-09-21T02:11:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f265b7fe640 1 -- 172.21.15.92:0/2071720910 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bca93bdf0 con 0x555bca5da9f0 2024-09-21T02:11:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f265b7fe640 10 client.17328 handle_command_reply: tid=0 2024-09-21T02:11:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 client.17328 shutdown 2024-09-21T02:11:11.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 msgr2=0x555bca9ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9ced40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f266c0058c0 tx=0x7f266c004700 comp rx=0 tx=0).stop 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 shutdown_connections 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca61c070 0x555bca9cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5da9f0 0x555bca9e0f10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 --2- 172.21.15.92:0/2071720910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca5f31b0 0x555bca9ced40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 >> 172.21.15.92:0/2071720910 conn(0x555bca610460 msgr2=0x555bca9c6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f265b7fe640 0 client.17328 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 shutdown_connections 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 1 -- 172.21.15.92:0/2071720910 wait complete. 2024-09-21T02:11:11.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.710+0000 7f2688e48740 20 client.17328 trim_cache size 0 max 16384 2024-09-21T02:11:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 Processor -- start 2024-09-21T02:11:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 -- start start 2024-09-21T02:11:11.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1b16d0 con 0x555bca61c070 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1b3210 con 0x555bca5fb390 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2688e48740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1b1da0 con 0x555bca5f31b0 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2680e71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2680e71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41536/0 (socket says 172.21.15.92:41536) 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.723+0000 7f2680e71640 1 -- 172.21.15.92:0/968480643 learned_addr learned my addr 172.21.15.92:0/968480643 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f267b7fe640 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f267bfff640 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2680e71640 1 -- 172.21.15.92:0/968480643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 msgr2=0x555bca9d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2680e71640 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2680e71640 1 -- 172.21.15.92:0/968480643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 msgr2=0x555bca9cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2680e71640 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2680e71640 1 -- 172.21.15.92:0/968480643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca9d0000 con 0x555bca61c070 2024-09-21T02:11:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f267bfff640 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:11.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2680e71640 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9cc470 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2674033000 tx=0x7f2674044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370d4109b21da6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:11.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2661ffb640 1 -- 172.21.15.92:0/968480643 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674003af0 con 0x555bca61c070 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2661ffb640 1 -- 172.21.15.92:0/968480643 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2674036040 con 0x555bca61c070 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.724+0000 7f2661ffb640 1 -- 172.21.15.92:0/968480643 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267402f990 con 0x555bca61c070 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 -- 172.21.15.92:0/968480643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 msgr2=0x555bca9cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9cc470 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2674033000 tx=0x7f2674044000 comp rx=0 tx=0).stop 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 -- 172.21.15.92:0/968480643 shutdown_connections 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9cc470 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 --2- 172.21.15.92:0/968480643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 -- 172.21.15.92:0/968480643 >> 172.21.15.92:0/968480643 conn(0x555bca610460 msgr2=0x555bca9d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 -- 172.21.15.92:0/968480643 shutdown_connections 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 -- 172.21.15.92:0/968480643 wait complete. 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.725+0000 7f2688e48740 1 -- start start 2024-09-21T02:11:11.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 Processor -- start 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9e2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1b2b40 con 0x555bca61c070 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1b2470 con 0x555bca5fb390 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1af6b0 con 0x555bca5f31b0 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2688e48740 10 client.0 _ll_register_callbacks cb 0x7f2674009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9e29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41546/0 (socket says 172.21.15.92:41546) 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 -- 172.21.15.92:0/3206665653 learned_addr learned my addr 172.21.15.92:0/3206665653 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f267b7fe640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f267bfff640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 -- 172.21.15.92:0/3206665653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 msgr2=0x555bca9e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9e1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 -- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 msgr2=0x555bca9e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.726+0000 7f2680e71640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f2680e71640 1 -- 172.21.15.92:0/3206665653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcaa4a690 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f2680e71640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9e29c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f267402ade0 tx=0x7f2674031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20e078365e9de77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f265b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f265b7fe640 1 -- 172.21.15.92:0/3206665653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267405b070 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f265b7fe640 1 -- 172.21.15.92:0/3206665653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f267402f8d0 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2674036040 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f265b7fe640 1 -- 172.21.15.92:0/3206665653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2674067050 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f2688e48740 10 client.17334 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555bcaa4b910 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f265b7fe640 1 -- 172.21.15.92:0/3206665653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2674036040 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.727+0000 7f265b7fe640 1 -- 172.21.15.92:0/3206665653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2674030b90 con 0x555bca61c070 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 10 client.17334 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:11.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 10 client.17334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 20 client.17334 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5afda0 0x555bcaa4e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 4 client.17334 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bca9378a0 con 0x555bca5afda0 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f267bfff640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5afda0 0x555bcaa4e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f267bfff640 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5afda0 0x555bcaa4e0b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d130713e42009ba server_cookie=1b910ee3328d1631 in_seq=0 out_seq=0 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f265b7fe640 10 client.17334 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f265b7fe640 1 -- 172.21.15.92:0/3206665653 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bca9378a0 con 0x555bca5afda0 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f265b7fe640 10 client.17334 handle_command_reply: tid=0 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.728+0000 7f2688e48740 1 client.17334 shutdown 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 msgr2=0x555bca9e29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9e29c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f267402ade0 tx=0x7f2674031c70 comp rx=0 tx=0).stop 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 shutdown_connections 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x555bca5afda0 0x555bcaa4e0b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555bca61c070 0x555bca9e29c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bca5fb390 0x555bca9e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 --2- 172.21.15.92:0/3206665653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bca5f31b0 0x555bca9e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 >> 172.21.15.92:0/3206665653 conn(0x555bca610460 msgr2=0x555bca9c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f265b7fe640 0 client.17334 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 shutdown_connections 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 1 -- 172.21.15.92:0/3206665653 wait complete. 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f2688e48740 20 client.17334 trim_cache size 0 max 16384 2024-09-21T02:11:11.735 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f26640b6510 msgr2=0x7f26640b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f26640b6510 0x7f26640b8a20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2674005cc0 tx=0x7f2674005c30 comp rx=0 tx=0).stop 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 msgr2=0x7f267c0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 0x7f267c0ffbd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f267000b450 tx=0x7f267000b910 comp rx=0 tx=0).stop 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 shutdown_connections 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f26640b6510 0x7f26640b8a20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f267c108d10 0x7f267c100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c1083c0 0x7f267c0ffbd0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.729+0000 7f26828d4640 1 --2- 172.21.15.92:0/4138323999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c102a80 0x7f267c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:11.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.730+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 >> 172.21.15.92:0/4138323999 conn(0x7f267c079740 msgr2=0x7f267c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:11.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.730+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 shutdown_connections 2024-09-21T02:11:11.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:11.730+0000 7f26828d4640 1 -- 172.21.15.92:0/4138323999 wait complete. 2024-09-21T02:11:12.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:11:12.721 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:12.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:12 smithi136 ceph-mon[34754]: pgmap v547: 97 pgs: 97 active+clean; 808 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:11:12.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:12 smithi092 ceph-mon[31190]: pgmap v547: 97 pgs: 97 active+clean; 808 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:11:12.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:12 smithi099 ceph-mon[34404]: pgmap v547: 97 pgs: 97 active+clean; 808 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 -- 172.21.15.92:0/2195452411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c108420 msgr2=0x7f9d0c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/2195452411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c108420 0x7f9d0c108830 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ca0 tx=0x7f9cfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 -- 172.21.15.92:0/2195452411 shutdown_connections 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/2195452411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108d70 0x7f9d0c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/2195452411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c108420 0x7f9d0c108830 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/2195452411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 -- 172.21.15.92:0/2195452411 >> 172.21.15.92:0/2195452411 conn(0x7f9d0c079770 msgr2=0x7f9d0c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:13.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 -- 172.21.15.92:0/2195452411 shutdown_connections 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.242+0000 7f9d12f4c640 1 -- 172.21.15.92:0/2195452411 wait complete. 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 Processor -- start 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 -- start start 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108420 0x7f9d0c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c108d70 0x7f9d0c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c112a30 con 0x7f9d0c102ae0 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1128b0 con 0x7f9d0c108d70 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d12f4c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c112bb0 con 0x7f9d0c108420 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d10ce8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d10ce8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35352/0 (socket says 172.21.15.92:35352) 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.243+0000 7f9d10ce8640 1 -- 172.21.15.92:0/237729766 learned_addr learned my addr 172.21.15.92:0/237729766 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:13.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d114e9640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c108d70 0x7f9d0c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108420 0x7f9d0c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d10ce8640 1 -- 172.21.15.92:0/237729766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108420 msgr2=0x7f9d0c0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108420 0x7f9d0c0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d10ce8640 1 -- 172.21.15.92:0/237729766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c108d70 msgr2=0x7f9d0c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c108d70 0x7f9d0c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d10ce8640 1 -- 172.21.15.92:0/237729766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0c1a8810 con 0x7f9d0c102ae0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108420 0x7f9d0c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d114e9640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c108d70 0x7f9d0c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c0ff670 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c890 tx=0x7f9d0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d09ffb640 1 -- 172.21.15.92:0/237729766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004220 con 0x7f9d0c102ae0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d09ffb640 1 -- 172.21.15.92:0/237729766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d040043c0 con 0x7f9d0c102ae0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0c1a8b00 con 0x7f9d0c102ae0 2024-09-21T02:11:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d09ffb640 1 -- 172.21.15.92:0/237729766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004a00 con 0x7f9d0c102ae0 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.244+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0c1a9090 con 0x7f9d0c102ae0 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 Processor -- start 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 -- start start 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d09ffb640 1 -- 172.21.15.92:0/237729766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f9d040079b0 con 0x7f9d0c102ae0 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ae5070 0x563c37e8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563c37e02ea0 con 0x563c37abc1b0 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c37df22d0 con 0x563c37ac4390 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.247+0000 7f9d194c0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c37df60e0 con 0x563c37ae5070 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e8c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35368/0 (socket says 172.21.15.92:35368) 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 -- 172.21.15.92:0/2051471333 learned_addr learned my addr 172.21.15.92:0/2051471333 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:13.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d114e9640 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ae5070 0x563c37e8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 -- 172.21.15.92:0/2051471333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ae5070 msgr2=0x563c37e8c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ae5070 0x563c37e8c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 -- 172.21.15.92:0/2051471333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 msgr2=0x563c37e8cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e8cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 -- 172.21.15.92:0/2051471333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c37b50650 con 0x563c37abc1b0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e8c800 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ef270 tx=0x7f9d040ef730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5cffaad3b937dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d09ffb640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d114e9640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d02ffd640 1 -- 172.21.15.92:0/2051471333 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040fa070 con 0x563c37abc1b0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.248+0000 7f9d09ffb640 1 -- 172.21.15.92:0/237729766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9d040e6ff0 con 0x7f9d0c102ae0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d02ffd640 1 -- 172.21.15.92:0/2051471333 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d04005290 con 0x563c37abc1b0 2024-09-21T02:11:13.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d02ffd640 1 -- 172.21.15.92:0/2051471333 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040acfd0 con 0x563c37abc1b0 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 -- 172.21.15.92:0/2051471333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 msgr2=0x563c37e8c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d114e9640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400cc80 tx=0x7f9cf400c430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e8c800 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ef270 tx=0x7f9d040ef730 comp rx=0 tx=0).stop 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 -- 172.21.15.92:0/2051471333 shutdown_connections 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e8c800 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2051471333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ae5070 0x563c37e8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.249+0000 7f9d194c0740 1 -- 172.21.15.92:0/2051471333 >> 172.21.15.92:0/2051471333 conn(0x563c37ad9460 msgr2=0x563c37e8a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.250+0000 7f9d194c0740 1 -- 172.21.15.92:0/2051471333 shutdown_connections 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.250+0000 7f9d194c0740 1 -- 172.21.15.92:0/2051471333 wait complete. 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.250+0000 7f9d194c0740 1 -- start start 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.250+0000 7f9d194c0740 1 Processor -- start 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37abc1b0 0x563c37e979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e97f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 0x563c37ea3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563c37c53700 con 0x563c37ae5070 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c37d8d160 con 0x563c37ac4390 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c37e9aaa0 con 0x563c37abc1b0 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d194c0740 10 client.0 _ll_register_callbacks cb 0x563c37da69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 0x563c37ea3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 0x563c37ea3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35370/0 (socket says 172.21.15.92:35370) 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 -- 172.21.15.92:0/453676262 learned_addr learned my addr 172.21.15.92:0/453676262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e97f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37abc1b0 0x563c37e979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 -- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37abc1b0 msgr2=0x563c37e979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37abc1b0 0x563c37e979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.260 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 -- 172.21.15.92:0/453676262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 msgr2=0x563c37e97f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e97f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 -- 172.21.15.92:0/453676262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c37dbac90 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9d114e9640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 0x563c37ea3dc0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4005d50 tx=0x7f9cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467a73b9089e6d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.251+0000 7f9cebfff640 1 -- 172.21.15.92:0/453676262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf401cb20 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9cebfff640 1 -- 172.21.15.92:0/453676262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9cf401ccc0 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c37e04df0 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9cebfff640 1 -- 172.21.15.92:0/453676262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9cf4028480 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 10 client.17346 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c37ea6840 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9cebfff640 1 -- 172.21.15.92:0/453676262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563c37e04df0 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9cebfff640 1 -- 172.21.15.92:0/453676262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9cf401a390 con 0x563c37ae5070 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 10 client.17346 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 20 client.17346 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37aa39f0 0x563c37ea9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 4 client.17346 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:13.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.252+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c37e008a0 con 0x563c37aa39f0 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.253+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37aa39f0 0x563c37ea9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.253+0000 7f9cebfff640 10 client.17346 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.253+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37aa39f0 0x563c37ea9d90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf12bed7d316839f server_cookie=53ce240f360bb78b in_seq=0 out_seq=0 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9cebfff640 1 -- 172.21.15.92:0/453676262 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c37e008a0 con 0x563c37aa39f0 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9cebfff640 10 client.17346 handle_command_reply: tid=0 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 client.17346 shutdown 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 msgr2=0x563c37ea3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 0x563c37ea3dc0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4005d50 tx=0x7f9cf40046c0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 shutdown_connections 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37aa39f0 0x563c37ea9d90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37ae5070 0x563c37ea3dc0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9cebfff640 0 client.17346 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e97f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 --2- 172.21.15.92:0/453676262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37abc1b0 0x563c37e979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 >> 172.21.15.92:0/453676262 conn(0x563c37ad9460 msgr2=0x563c37e8f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 shutdown_connections 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 1 -- 172.21.15.92:0/453676262 wait complete. 2024-09-21T02:11:13.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.254+0000 7f9d194c0740 20 client.17346 trim_cache size 0 max 16384 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 Processor -- start 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 -- start start 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 0x563c37e8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37abc1b0 0x563c37ea3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ac4390 0x563c37e89ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b17f0 con 0x563c37aa39f0 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b3330 con 0x563c37abc1b0 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d194c0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b1ec0 con 0x563c37ac4390 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 0x563c37e8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 0x563c37e8fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35386/0 (socket says 172.21.15.92:35386) 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 -- 172.21.15.92:0/3360157117 learned_addr learned my addr 172.21.15.92:0/3360157117 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 -- 172.21.15.92:0/3360157117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ac4390 msgr2=0x563c37e89ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d114e9640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ac4390 0x563c37e89ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37abc1b0 0x563c37ea3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ac4390 0x563c37e89ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 -- 172.21.15.92:0/3360157117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37abc1b0 msgr2=0x563c37ea3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37abc1b0 0x563c37ea3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 -- 172.21.15.92:0/3360157117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c37e8aaa0 con 0x563c37aa39f0 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d114e9640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ac4390 0x563c37e89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37abc1b0 0x563c37ea3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 0x563c37e8fad0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9d040aa470 tx=0x7f9d04005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302ac916514558b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d027fc640 1 -- 172.21.15.92:0/3360157117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0401d040 con 0x563c37aa39f0 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d027fc640 1 -- 172.21.15.92:0/3360157117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d040ea690 con 0x563c37aa39f0 2024-09-21T02:11:13.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.268+0000 7f9d027fc640 1 -- 172.21.15.92:0/3360157117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040eac50 con 0x563c37aa39f0 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 -- 172.21.15.92:0/3360157117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 msgr2=0x563c37e8fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 0x563c37e8fad0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9d040aa470 tx=0x7f9d04005760 comp rx=0 tx=0).stop 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 -- 172.21.15.92:0/3360157117 shutdown_connections 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37ac4390 0x563c37e89ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37abc1b0 0x563c37ea3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 --2- 172.21.15.92:0/3360157117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37aa39f0 0x563c37e8fad0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 -- 172.21.15.92:0/3360157117 >> 172.21.15.92:0/3360157117 conn(0x563c37ad9460 msgr2=0x563c37e8f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 -- 172.21.15.92:0/3360157117 shutdown_connections 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 -- 172.21.15.92:0/3360157117 wait complete. 2024-09-21T02:11:13.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 -- start start 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.270+0000 7f9d194c0740 1 Processor -- start 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37aa39f0 0x563c37ea0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e9be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b2c60 con 0x563c37abc1b0 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b2590 con 0x563c37ac4390 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1af7d0 con 0x563c37aa39f0 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d194c0740 10 client.0 _ll_register_callbacks cb 0x7f9cf400c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e9b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35388/0 (socket says 172.21.15.92:35388) 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 -- 172.21.15.92:0/2928867313 learned_addr learned my addr 172.21.15.92:0/2928867313 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37aa39f0 0x563c37ea0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 -- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37aa39f0 msgr2=0x563c37ea0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37aa39f0 0x563c37ea0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 -- 172.21.15.92:0/2928867313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 msgr2=0x563c37e9be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e9be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d0bfff640 1 -- 172.21.15.92:0/2928867313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c37e9c510 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.271+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37aa39f0 0x563c37ea0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d0bfff640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e9b8e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c06b9c0 tx=0x7f9cfc0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27543994f7d08fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 1 -- 172.21.15.92:0/2928867313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc003970 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 1 -- 172.21.15.92:0/2928867313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9cfc003b10 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c37dfdcd0 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 1 -- 172.21.15.92:0/2928867313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc003d10 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 10 client.17352 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c37f144c0 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 1 -- 172.21.15.92:0/2928867313 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563c37dfdcd0 con 0x563c37abc1b0 2024-09-21T02:11:13.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 1 -- 172.21.15.92:0/2928867313 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9cfc02f660 con 0x563c37abc1b0 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 10 client.17352 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 20 client.17352 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37ae5070 0x563c37f16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 4 client.17352 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c37d73aa0 con 0x563c37ae5070 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37ae5070 0x563c37f16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d10ce8640 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37ae5070 0x563c37f16cb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8791f3efe0066855 server_cookie=a1cdb1c0876d964e in_seq=0 out_seq=0 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 10 client.17352 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 1 -- 172.21.15.92:0/2928867313 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c37d73aa0 con 0x563c37ae5070 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9cebfff640 10 client.17352 handle_command_reply: tid=0 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 client.17352 shutdown 2024-09-21T02:11:13.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 msgr2=0x563c37e9b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e9b8e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c06b9c0 tx=0x7f9cfc0316f0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 shutdown_connections 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x563c37ae5070 0x563c37f16cb0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c37ac4390 0x563c37e9be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c37abc1b0 0x563c37e9b8e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 --2- 172.21.15.92:0/2928867313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c37aa39f0 0x563c37ea0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.272+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 >> 172.21.15.92:0/2928867313 conn(0x563c37ad9460 msgr2=0x563c37e89970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9cebfff640 0 client.17352 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 shutdown_connections 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d194c0740 1 -- 172.21.15.92:0/2928867313 wait complete. 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d194c0740 20 client.17352 trim_cache size 0 max 16384 2024-09-21T02:11:13.279 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9cf80b6590 msgr2=0x7f9cf80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400cc80 tx=0x7f9cf400c430 comp rx=0 tx=0).stop 2024-09-21T02:11:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 msgr2=0x7f9d0c0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c0ff670 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c890 tx=0x7f9d0400cd50 comp rx=0 tx=0).stop 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 shutdown_connections 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9cf80b6590 0x7f9cf80b8aa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0c108d70 0x7f9d0c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d0c108420 0x7f9d0c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 --2- 172.21.15.92:0/237729766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c102ae0 0x7f9d0c0ff670 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.273+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 >> 172.21.15.92:0/237729766 conn(0x7f9d0c079770 msgr2=0x7f9d0c101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.274+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 shutdown_connections 2024-09-21T02:11:13.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:13.274+0000 7f9d12f4c640 1 -- 172.21.15.92:0/237729766 wait complete. 2024-09-21T02:11:14.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:14 smithi136 ceph-mon[34754]: pgmap v548: 97 pgs: 97 active+clean; 865 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:11:14.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:14 smithi092 ceph-mon[31190]: pgmap v548: 97 pgs: 97 active+clean; 865 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:11:14.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:14 smithi099 ceph-mon[34404]: pgmap v548: 97 pgs: 97 active+clean; 865 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:11:15.010 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:11:15.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:11:15.275 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/1118332862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c41094a0 msgr2=0x7ff4c41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/1118332862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c41094a0 0x7ff4c41098b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009900 tx=0x7ff4b802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/1118332862 shutdown_connections 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/1118332862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c4109df0 0x7ff4c4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/1118332862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c41094a0 0x7ff4c41098b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/1118332862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c4103b60 0x7ff4c4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/1118332862 >> 172.21.15.92:0/1118332862 conn(0x7ff4c407b850 msgr2=0x7ff4c407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/1118332862 shutdown_connections 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/1118332862 wait complete. 2024-09-21T02:11:15.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.746+0000 7ff4c9d6c640 1 Processor -- start 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 -- start start 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c4103b60 0x7ff4c419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c41094a0 0x7ff4c419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 0x7ff4c419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4113af0 con 0x7ff4c4109df0 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4113970 con 0x7ff4c4103b60 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c9d6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4113c70 con 0x7ff4c41094a0 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 0x7ff4c419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 0x7ff4c419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35408/0 (socket says 172.21.15.92:35408) 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 -- 172.21.15.92:0/2006456510 learned_addr learned my addr 172.21.15.92:0/2006456510 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c41094a0 0x7ff4c419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c4103b60 0x7ff4c419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 -- 172.21.15.92:0/2006456510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c41094a0 msgr2=0x7ff4c419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c41094a0 0x7ff4c419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 -- 172.21.15.92:0/2006456510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c4103b60 msgr2=0x7ff4c419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c4103b60 0x7ff4c419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.747+0000 7ff4c3fff640 1 -- 172.21.15.92:0/2006456510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c41a0490 con 0x7ff4c4109df0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.748+0000 7ff4c3fff640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 0x7ff4c419fd60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00eaa0 tx=0x7ff4bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.748+0000 7ff4c0ff9640 1 -- 172.21.15.92:0/2006456510 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00cdb0 con 0x7ff4c4109df0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.748+0000 7ff4c0ff9640 1 -- 172.21.15.92:0/2006456510 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc005020 con 0x7ff4c4109df0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.748+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c41a0780 con 0x7ff4c4109df0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.748+0000 7ff4c0ff9640 1 -- 172.21.15.92:0/2006456510 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0116d0 con 0x7ff4c4109df0 2024-09-21T02:11:15.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.748+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a6ee0 con 0x7ff4c4109df0 2024-09-21T02:11:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.750+0000 7ff4c0ff9640 1 -- 172.21.15.92:0/2006456510 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff4bc0051d0 con 0x7ff4c4109df0 2024-09-21T02:11:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 Processor -- start 2024-09-21T02:11:15.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 -- start start 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c9583430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91b3490 0x5602c9583970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91bb670 0x5602c9583eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5602c94f9e60 con 0x5602c91dc350 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5602c94e9290 con 0x5602c91bb670 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5602c94ed0a0 con 0x5602c91b3490 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c9583430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c9583430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35418/0 (socket says 172.21.15.92:35418) 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/2699871491 learned_addr learned my addr 172.21.15.92:0/2699871491 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:15.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c0ff9640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff4ac0b6510 0x7ff4ac0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91b3490 0x5602c9583970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91bb670 0x5602c9583eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c3fff640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff4ac0b6510 0x7ff4ac0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/2699871491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91b3490 msgr2=0x5602c9583970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91b3490 0x5602c9583970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/2699871491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91bb670 msgr2=0x5602c9583eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.751+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91bb670 0x5602c9583eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/2699871491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c9247930 con 0x5602c91dc350 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4c0ff9640 1 -- 172.21.15.92:0/2006456510 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff4bc0e7460 con 0x7ff4c4109df0 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c9583430 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff4c406c9f0 tx=0x7ff4b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4f42b02a0c0259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4a9ffb640 1 -- 172.21.15.92:0/2699871491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8044020 con 0x5602c91dc350 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4a9ffb640 1 -- 172.21.15.92:0/2699871491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b8031820 con 0x5602c91dc350 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4a9ffb640 1 -- 172.21.15.92:0/2699871491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8035040 con 0x5602c91dc350 2024-09-21T02:11:15.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.752+0000 7ff4c3fff640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff4ac0b6510 0x7ff4ac0b8a20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0aa840 tx=0x7ff4bc07a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 -- 172.21.15.92:0/2699871491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 msgr2=0x5602c9583430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c9583430 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff4c406c9f0 tx=0x7ff4b8000bc0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 -- 172.21.15.92:0/2699871491 shutdown_connections 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91bb670 0x5602c9583eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91b3490 0x5602c9583970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/2699871491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c9583430 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 -- 172.21.15.92:0/2699871491 >> 172.21.15.92:0/2699871491 conn(0x5602c91d0740 msgr2=0x5602c9581320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 -- 172.21.15.92:0/2699871491 shutdown_connections 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 -- 172.21.15.92:0/2699871491 wait complete. 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.753+0000 7ff4d02e0740 1 -- start start 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 Processor -- start 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c958f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c958f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c959aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5602c934a890 con 0x5602c91bb670 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5602c9483d40 con 0x5602c91b3490 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5602c9591bb0 con 0x5602c91dc350 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4d02e0740 10 client.0 _ll_register_callbacks cb 0x5602c94e7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c958f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c958f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35424/0 (socket says 172.21.15.92:35424) 2024-09-21T02:11:15.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 -- 172.21.15.92:0/3286898125 learned_addr learned my addr 172.21.15.92:0/3286898125 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c958f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 -- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 msgr2=0x5602c959aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c959aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c959aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 -- 172.21.15.92:0/3286898125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 msgr2=0x5602c958f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c958f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c37fe640 1 -- 172.21.15.92:0/3286898125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c959b490 con 0x5602c91bb670 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.754+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c958f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c958f570 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4010500 tx=0x7ff4b40109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bab121929bff1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/3286898125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401a020 con 0x5602c91bb670 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/3286898125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff4b4016690 con 0x5602c91bb670 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b8031820 con 0x5602c91bb670 2024-09-21T02:11:15.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/3286898125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4b4025050 con 0x5602c91bb670 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 10 client.17364 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602c959de20 con 0x5602c91bb670 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/3286898125 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b8031820 con 0x5602c91bb670 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/3286898125 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff4b40149f0 con 0x5602c91bb670 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 10 client.17364 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 20 client.17364 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c919acd0 0x5602c95a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 4 client.17364 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.755+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c94fbdb0 con 0x5602c919acd0 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.756+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c919acd0 0x5602c95a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.756+0000 7ff4a2ffd640 10 client.17364 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:15.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.756+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c919acd0 0x5602c95a13f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71455a49bb41b3d3 server_cookie=9885a972fc5b3c3c in_seq=0 out_seq=0 2024-09-21T02:11:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/3286898125 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602c94fbdb0 con 0x5602c919acd0 2024-09-21T02:11:15.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4a2ffd640 10 client.17364 handle_command_reply: tid=0 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 client.17364 shutdown 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 msgr2=0x5602c958f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c958f570 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4010500 tx=0x7ff4b40109c0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 shutdown_connections 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c919acd0 0x5602c95a13f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c959aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4a2ffd640 0 client.17364 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c958f570 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/3286898125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c958f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 >> 172.21.15.92:0/3286898125 conn(0x5602c91d0740 msgr2=0x5602c9586610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 shutdown_connections 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 1 -- 172.21.15.92:0/3286898125 wait complete. 2024-09-21T02:11:15.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.757+0000 7ff4d02e0740 20 client.17364 trim_cache size 0 max 16384 2024-09-21T02:11:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 Processor -- start 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 -- start start 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9588510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c9588a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c958c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41af530 con 0x5602c91bb670 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b1070 con 0x5602c91b3490 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41afc00 con 0x5602c91dc350 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c9588a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c9588a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35426/0 (socket says 172.21.15.92:35426) 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 -- 172.21.15.92:0/968981252 learned_addr learned my addr 172.21.15.92:0/968981252 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9588510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c958c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 -- 172.21.15.92:0/968981252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 msgr2=0x5602c958c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c958c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 -- 172.21.15.92:0/968981252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 msgr2=0x5602c9588510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9588510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 -- 172.21.15.92:0/968981252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c958dd90 con 0x5602c91bb670 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.775+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c9588a50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400bad0 tx=0x7ff4b4006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e60a703fd90dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4a97fa640 1 -- 172.21.15.92:0/968981252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401a040 con 0x5602c91bb670 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4a97fa640 1 -- 172.21.15.92:0/968981252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b4012440 con 0x5602c91bb670 2024-09-21T02:11:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4a97fa640 1 -- 172.21.15.92:0/968981252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40147a0 con 0x5602c91bb670 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 -- 172.21.15.92:0/968981252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 msgr2=0x5602c9588a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c9588a50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400bad0 tx=0x7ff4b4006ce0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 -- 172.21.15.92:0/968981252 shutdown_connections 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91dc350 0x5602c958c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91bb670 0x5602c9588a50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/968981252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9588510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 -- 172.21.15.92:0/968981252 >> 172.21.15.92:0/968981252 conn(0x5602c91d0740 msgr2=0x5602c9589fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.776+0000 7ff4d02e0740 1 -- 172.21.15.92:0/968981252 shutdown_connections 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 -- 172.21.15.92:0/968981252 wait complete. 2024-09-21T02:11:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 -- start start 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 Processor -- start 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9594110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91bb670 0x5602c9594650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c960be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b09a0 con 0x5602c91dc350 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41b02d0 con 0x5602c91b3490 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41ad510 con 0x5602c91bb670 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.777+0000 7ff4d02e0740 10 client.0 _ll_register_callbacks cb 0x7ff4b400b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c960be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c960be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35432/0 (socket says 172.21.15.92:35432) 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/1255794229 learned_addr learned my addr 172.21.15.92:0/1255794229 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9594110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91bb670 0x5602c9594650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91bb670 msgr2=0x5602c9594650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91bb670 0x5602c9594650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/1255794229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 msgr2=0x5602c9594110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9594110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 -- 172.21.15.92:0/1255794229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c9594ed0 con 0x5602c91dc350 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4c2ffd640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c960be00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8033000 tx=0x7ff4b804e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5749f9ab16260c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/1255794229 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8044070 con 0x5602c91dc350 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/1255794229 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff4b8005570 con 0x5602c91dc350 2024-09-21T02:11:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b4012440 con 0x5602c91dc350 2024-09-21T02:11:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/1255794229 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4b8035040 con 0x5602c91dc350 2024-09-21T02:11:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4d02e0740 10 client.17370 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.778+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602c960d3c0 con 0x5602c91dc350 2024-09-21T02:11:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/1255794229 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b4012440 con 0x5602c91dc350 2024-09-21T02:11:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/1255794229 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff4b8057030 con 0x5602c91dc350 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4d02e0740 10 client.17370 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4d02e0740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4d02e0740 20 client.17370 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c9170080 0x5602c960f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4d02e0740 4 client.17370 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5602c94f7860 con 0x5602c9170080 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.779+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c9170080 0x5602c960f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.780+0000 7ff4a2ffd640 10 client.17370 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.780+0000 7ff4c37fe640 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c9170080 0x5602c960f750 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ce41c7b748b6e9 server_cookie=1e1cdf99fac4d10e in_seq=0 out_seq=0 2024-09-21T02:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.781+0000 7ff4a2ffd640 1 -- 172.21.15.92:0/1255794229 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5602c94f7860 con 0x5602c9170080 2024-09-21T02:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.781+0000 7ff4a2ffd640 10 client.17370 handle_command_reply: tid=0 2024-09-21T02:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.781+0000 7ff4d02e0740 1 client.17370 shutdown 2024-09-21T02:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.781+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 msgr2=0x5602c960be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.781+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c960be00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8033000 tx=0x7ff4b804e040 comp rx=0 tx=0).stop 2024-09-21T02:11:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 shutdown_connections 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5602c9170080 0x5602c960f750 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5602c91dc350 0x5602c960be00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5602c91bb670 0x5602c9594650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 --2- 172.21.15.92:0/1255794229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602c91b3490 0x5602c9594110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4a2ffd640 0 client.17370 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 >> 172.21.15.92:0/1255794229 conn(0x5602c91d0740 msgr2=0x5602c9589fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 shutdown_connections 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 1 -- 172.21.15.92:0/1255794229 wait complete. 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4d02e0740 20 client.17370 trim_cache size 0 max 16384 2024-09-21T02:11:15.788 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "c273971d-f2a7-4d64-b784-1321103a1652", 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff4ac0b6510 msgr2=0x7ff4ac0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff4ac0b6510 0x7ff4ac0b8a20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0aa840 tx=0x7ff4bc07a2d0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 msgr2=0x7ff4c419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 0x7ff4c419fd60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00eaa0 tx=0x7ff4bc00ef60 comp rx=0 tx=0).stop 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 shutdown_connections 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff4ac0b6510 0x7ff4ac0b8a20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4c4109df0 0x7ff4c419fd60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4c41094a0 0x7ff4c419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 --2- 172.21.15.92:0/2006456510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c4103b60 0x7ff4c419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.782+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 >> 172.21.15.92:0/2006456510 conn(0x7ff4c407b850 msgr2=0x7ff4c4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.783+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 shutdown_connections 2024-09-21T02:11:15.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:15.783+0000 7ff4c9d6c640 1 -- 172.21.15.92:0/2006456510 wait complete. 2024-09-21T02:11:16.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 -- 172.21.15.92:0/2269815989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110108300 msgr2=0x7f811010a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 --2- 172.21.15.92:0/2269815989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110108300 0x7f811010a810 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8108009900 tx=0x7f810802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 -- 172.21.15.92:0/2269815989 shutdown_connections 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 --2- 172.21.15.92:0/2269815989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110108300 0x7f811010a810 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 --2- 172.21.15.92:0/2269815989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 0x7f8110107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 --2- 172.21.15.92:0/2269815989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8110102c20 0x7f81101050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 -- 172.21.15.92:0/2269815989 >> 172.21.15.92:0/2269815989 conn(0x7f81100fd6d0 msgr2=0x7f81100ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 -- 172.21.15.92:0/2269815989 shutdown_connections 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.416+0000 7f811832b640 1 -- 172.21.15.92:0/2269815989 wait complete. 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 Processor -- start 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 -- start start 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 0x7f81101021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 0x7f811019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8110108300 0x7f81101a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f811010e220 con 0x7f8110102c20 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f811010e0a0 con 0x7f81101056e0 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f811832b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f811010e3a0 con 0x7f8110108300 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 0x7f81101021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 0x7f81101021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35440/0 (socket says 172.21.15.92:35440) 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 -- 172.21.15.92:0/291477471 learned_addr learned my addr 172.21.15.92:0/291477471 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81158c6640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 0x7f811019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81168c8640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8110108300 0x7f81101a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 -- 172.21.15.92:0/291477471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8110108300 msgr2=0x7f81101a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8110108300 0x7f81101a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 -- 172.21.15.92:0/291477471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 msgr2=0x7f811019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 0x7f811019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81160c7640 1 -- 172.21.15.92:0/291477471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81101a19a0 con 0x7f8110102c20 2024-09-21T02:11:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.417+0000 7f81158c6640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 0x7f811019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.418+0000 7f81160c7640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 0x7f81101021a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f810000ea80 tx=0x7f810000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.418+0000 7f810f7fe640 1 -- 172.21.15.92:0/291477471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810000ce40 con 0x7f8110102c20 2024-09-21T02:11:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.418+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81101a1c90 con 0x7f8110102c20 2024-09-21T02:11:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.418+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81101a2190 con 0x7f8110102c20 2024-09-21T02:11:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.418+0000 7f810f7fe640 1 -- 172.21.15.92:0/291477471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8100005020 con 0x7f8110102c20 2024-09-21T02:11:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.418+0000 7f810f7fe640 1 -- 172.21.15.92:0/291477471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810001f060 con 0x7f8110102c20 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 Processor -- start 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 -- start start 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f810f7fe640 1 -- 172.21.15.92:0/291477471 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8100004a40 con 0x7f8110102c20 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 0x55e275a86420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e2759f36f0 con 0x55e2756b84d0 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2759e2b20 con 0x55e2756b02f0 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f811f298740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e2759e6930 con 0x55e2756d91b0 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.421+0000 7f81158c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 0x55e275a86420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 0x55e275a86420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35452/0 (socket says 172.21.15.92:35452) 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 -- 172.21.15.92:0/2536963097 learned_addr learned my addr 172.21.15.92:0/2536963097 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81168c8640 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a85260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81160c7640 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 -- 172.21.15.92:0/2536963097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 msgr2=0x55e275a85260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a85260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 -- 172.21.15.92:0/2536963097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 msgr2=0x55e275a858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 -- 172.21.15.92:0/2536963097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2759a8530 con 0x55e2756b84d0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81160c7640 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f810f7fe640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80f80b6510 0x7f80f80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81158c6640 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 0x55e275a86420 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8104008c60 tx=0x7f810400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433fee3b276459a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f81168c8640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80f80b6510 0x7f80f80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f810f7fe640 1 -- 172.21.15.92:0/291477471 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81000e7520 con 0x7f8110102c20 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f810cbfe640 1 -- 172.21.15.92:0/2536963097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104014070 con 0x55e2756b84d0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f810cbfe640 1 -- 172.21.15.92:0/2536963097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81040047a0 con 0x55e2756b84d0 2024-09-21T02:11:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.422+0000 7f810cbfe640 1 -- 172.21.15.92:0/2536963097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104010690 con 0x55e2756b84d0 2024-09-21T02:11:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 -- 172.21.15.92:0/2536963097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 msgr2=0x55e275a86420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f81168c8640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80f80b6510 0x7f80f80b8a20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8108005ab0 tx=0x7f81080314c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 0x55e275a86420 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8104008c60 tx=0x7f810400e720 comp rx=0 tx=0).stop 2024-09-21T02:11:16.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 -- 172.21.15.92:0/2536963097 shutdown_connections 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b84d0 0x55e275a86420 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 --2- 172.21.15.92:0/2536963097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 -- 172.21.15.92:0/2536963097 >> 172.21.15.92:0/2536963097 conn(0x55e2756cd300 msgr2=0x55e275a7c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 -- 172.21.15.92:0/2536963097 shutdown_connections 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.423+0000 7f811f298740 1 -- 172.21.15.92:0/2536963097 wait complete. 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 -- start start 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 Processor -- start 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a8ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a8f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756d91b0 0x55e275a8fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e27591aab0 con 0x55e2756b02f0 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e275988730 con 0x55e2756d91b0 2024-09-21T02:11:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e275a86f40 con 0x55e2756b84d0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f811f298740 10 client.0 _ll_register_callbacks cb 0x55e2759dc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a8ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a8ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35454/0 (socket says 172.21.15.92:35454) 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 -- 172.21.15.92:0/35126572 learned_addr learned my addr 172.21.15.92:0/35126572 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81158c6640 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a8f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 -- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 msgr2=0x55e275a8f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a8f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 -- 172.21.15.92:0/35126572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756d91b0 msgr2=0x55e275a8fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756d91b0 0x55e275a8fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.424+0000 7f81160c7640 1 -- 172.21.15.92:0/35126572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e275744790 con 0x55e2756b02f0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f81160c7640 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a8ea40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f810000efd0 tx=0x7f8100004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95018a76df32b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f80f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f80f5ffb640 1 -- 172.21.15.92:0/35126572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000acfe0 con 0x55e2756b02f0 2024-09-21T02:11:16.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f80f5ffb640 1 -- 172.21.15.92:0/35126572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81000a8fe0 con 0x55e2756b02f0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81040047a0 con 0x55e2756b02f0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f80f5ffb640 1 -- 172.21.15.92:0/35126572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f81000ace00 con 0x55e2756b02f0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 10 client.17382 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2757bd620 con 0x55e2756b02f0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f80f5ffb640 1 -- 172.21.15.92:0/35126572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81040047a0 con 0x55e2756b02f0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f80f5ffb640 1 -- 172.21.15.92:0/35126572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f810000f040 con 0x55e2756b02f0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 10 client.17382 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 20 client.17382 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e275697d80 0x55e275a9c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 4 client.17382 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.425+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2759f5640 con 0x55e275697d80 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.426+0000 7f81158c6640 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e275697d80 0x55e275a9c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.426+0000 7f80f5ffb640 10 client.17382 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.426+0000 7f81158c6640 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e275697d80 0x55e275a9c260 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ad6098fc8c8f1d server_cookie=d51e86f1b9e4430d in_seq=0 out_seq=0 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f80f5ffb640 1 -- 172.21.15.92:0/35126572 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e2759f5640 con 0x55e275697d80 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f80f5ffb640 10 client.17382 handle_command_reply: tid=0 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 client.17382 shutdown 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 msgr2=0x55e275a8ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a8ea40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f810000efd0 tx=0x7f8100004fb0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 shutdown_connections 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e275697d80 0x55e275a9c260 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756d91b0 0x55e275a8fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a8f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 --2- 172.21.15.92:0/35126572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a8ea40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f80f5ffb640 0 client.17382 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 >> 172.21.15.92:0/35126572 conn(0x55e2756cd300 msgr2=0x55e275a7aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 shutdown_connections 2024-09-21T02:11:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 1 -- 172.21.15.92:0/35126572 wait complete. 2024-09-21T02:11:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.427+0000 7f811f298740 20 client.17382 trim_cache size 0 max 16384 2024-09-21T02:11:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.441+0000 7f811f298740 1 Processor -- start 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.441+0000 7f811f298740 1 -- start start 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.441+0000 7f811f298740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a7e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.441+0000 7f811f298740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a7c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f811f298740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 0x55e275a8dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f811f298740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc00e4e0 con 0x55e2756d91b0 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f811f298740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc010020 con 0x55e2756b02f0 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f811f298740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc00ebb0 con 0x55e2756b84d0 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 0x55e275a8dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 0x55e275a8dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35470/0 (socket says 172.21.15.92:35470) 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 -- 172.21.15.92:0/1714176368 learned_addr learned my addr 172.21.15.92:0/1714176368 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81158c6640 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a7c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81160c7640 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a7e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 -- 172.21.15.92:0/1714176368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 msgr2=0x55e275a7c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a7c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 -- 172.21.15.92:0/1714176368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 msgr2=0x55e275a7e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a7e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 -- 172.21.15.92:0/1714176368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e275a96f10 con 0x55e2756d91b0 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81160c7640 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a7e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f81168c8640 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 0x55e275a8dc60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8108033000 tx=0x7f810803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04fa16856a1bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f80f7fff640 1 -- 172.21.15.92:0/1714176368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810804c070 con 0x55e2756d91b0 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f80f7fff640 1 -- 172.21.15.92:0/1714176368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81080054f0 con 0x55e2756d91b0 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.442+0000 7f80f7fff640 1 -- 172.21.15.92:0/1714176368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108035040 con 0x55e2756d91b0 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 -- 172.21.15.92:0/1714176368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 msgr2=0x55e275a8dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 0x55e275a8dc60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8108033000 tx=0x7f810803e070 comp rx=0 tx=0).stop 2024-09-21T02:11:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 -- 172.21.15.92:0/1714176368 shutdown_connections 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756d91b0 0x55e275a8dc60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756b84d0 0x55e275a7c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 --2- 172.21.15.92:0/1714176368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b02f0 0x55e275a7e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 -- 172.21.15.92:0/1714176368 >> 172.21.15.92:0/1714176368 conn(0x55e2756cd300 msgr2=0x55e275a860f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 -- 172.21.15.92:0/1714176368 shutdown_connections 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 -- 172.21.15.92:0/1714176368 wait complete. 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.443+0000 7f811f298740 1 -- start start 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 Processor -- start 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a93af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b84d0 0x55e275a941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a94d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc00f950 con 0x55e2756b02f0 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc00f280 con 0x55e2756b84d0 2024-09-21T02:11:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc00c4c0 con 0x55e2756d91b0 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 10 client.0 _ll_register_callbacks cb 0x7f81000a9ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a93af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a93af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35476/0 (socket says 172.21.15.92:35476) 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 -- 172.21.15.92:0/335321722 learned_addr learned my addr 172.21.15.92:0/335321722 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81158c6640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b84d0 0x55e275a941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81168c8640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a94d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 -- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 msgr2=0x55e275a94d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a94d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 -- 172.21.15.92:0/335321722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b84d0 msgr2=0x55e275a941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b84d0 0x55e275a941b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 -- 172.21.15.92:0/335321722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e275a95480 con 0x55e2756b02f0 2024-09-21T02:11:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f81160c7640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a93af0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f81000a9460 tx=0x7f81000ad050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6849bd16e11520de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f80f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f80f5ffb640 1 -- 172.21.15.92:0/335321722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000b6240 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f80f5ffb640 1 -- 172.21.15.92:0/335321722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81000b3300 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.444+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81080054f0 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f80f5ffb640 1 -- 172.21.15.92:0/335321722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f81000b63e0 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 10 client.17388 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e275a96810 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f80f5ffb640 1 -- 172.21.15.92:0/335321722 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81080054f0 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f80f5ffb640 1 -- 172.21.15.92:0/335321722 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f81000ae9c0 con 0x55e2756b02f0 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 10 client.17388 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 20 client.17388 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e27566d1c0 0x55e275b08f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 4 client.17388 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:11:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2759f10f0 con 0x55e27566d1c0 2024-09-21T02:11:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.445+0000 7f81158c6640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e27566d1c0 0x55e275b08f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.446+0000 7f80f5ffb640 10 client.17388 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.446+0000 7f81158c6640 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e27566d1c0 0x55e275b08f60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928aa2b54d584ad3 server_cookie=e5281201becfb620 in_seq=0 out_seq=0 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.460+0000 7f80f5ffb640 1 -- 172.21.15.92:0/335321722 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2759f10f0 con 0x55e27566d1c0 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f80f5ffb640 10 client.17388 handle_command_reply: tid=0 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 client.17388 shutdown 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 msgr2=0x55e275a93af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a93af0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f81000a9460 tx=0x7f81000ad050 comp rx=0 tx=0).stop 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 shutdown_connections 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55e27566d1c0 0x55e275b08f60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2756d91b0 0x55e275a94d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2756b84d0 0x55e275a941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 --2- 172.21.15.92:0/335321722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2756b02f0 0x55e275a93af0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f80f5ffb640 0 client.17388 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 >> 172.21.15.92:0/335321722 conn(0x55e2756cd300 msgr2=0x55e275a860f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 shutdown_connections 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 1 -- 172.21.15.92:0/335321722 wait complete. 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811f298740 20 client.17388 trim_cache size 0 max 16384 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:11:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:11:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80f80b6510 msgr2=0x7f80f80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811832b640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80f80b6510 0x7f80f80b8a20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8108005ab0 tx=0x7f81080314c0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 msgr2=0x7f81101021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.461+0000 7f811832b640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 0x7f81101021a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f810000ea80 tx=0x7f810000ef40 comp rx=0 tx=0).stop 2024-09-21T02:11:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 shutdown_connections 2024-09-21T02:11:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f80f80b6510 0x7f80f80b8a20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8110108300 0x7f81101a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81101056e0 0x7f811019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 --2- 172.21.15.92:0/291477471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8110102c20 0x7f81101021a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 >> 172.21.15.92:0/291477471 conn(0x7f81100fd6d0 msgr2=0x7f8110103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 shutdown_connections 2024-09-21T02:11:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:16.462+0000 7f811832b640 1 -- 172.21.15.92:0/291477471 wait complete. 2024-09-21T02:11:16.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:11:16.737 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:16 smithi136 ceph-mon[34754]: pgmap v549: 97 pgs: 97 active+clean; 865 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:16 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:11:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:11:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:16 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:11:16.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:11:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:16 smithi092 ceph-mon[31190]: pgmap v549: 97 pgs: 97 active+clean; 865 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:16.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:16 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:11:16.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:11:16.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:16 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:11:16.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:11:16.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:16 smithi099 ceph-mon[34404]: pgmap v549: 97 pgs: 97 active+clean; 865 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:16.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:16 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:11:16.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:11:16.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:16 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:11:16.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2665273856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f68108cf0 msgr2=0x7f2f6810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2665273856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f68108cf0 0x7f2f6810f520 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009ca0 tx=0x7f2f6002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2665273856 shutdown_connections 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2665273856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f68108cf0 0x7f2f6810f520 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2665273856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f681083a0 0x7f2f681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2665273856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f68102a60 0x7f2f68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.199+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2665273856 >> 172.21.15.92:0/2665273856 conn(0x7f2f68079730 msgr2=0x7f2f68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.200+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2665273856 shutdown_connections 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.200+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2665273856 wait complete. 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.200+0000 7f2f6fb6c640 1 Processor -- start 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.200+0000 7f2f6fb6c640 1 -- start start 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.200+0000 7f2f6fb6c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f68102a60 0x7f2f6819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.200+0000 7f2f6fb6c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 0x7f2f6819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6fb6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68108cf0 0x7f2f6819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6fb6c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68112960 con 0x7f2f681083a0 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6fb6c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681127e0 con 0x7f2f68108cf0 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6fb6c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68112ae0 con 0x7f2f68102a60 2024-09-21T02:11:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 0x7f2f6819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 0x7f2f6819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35496/0 (socket says 172.21.15.92:35496) 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 -- 172.21.15.92:0/2410344419 learned_addr learned my addr 172.21.15.92:0/2410344419 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d908640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f68102a60 0x7f2f6819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6e109640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68108cf0 0x7f2f6819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 -- 172.21.15.92:0/2410344419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f68102a60 msgr2=0x7f2f6819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f68102a60 0x7f2f6819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 -- 172.21.15.92:0/2410344419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68108cf0 msgr2=0x7f2f6819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68108cf0 0x7f2f6819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 -- 172.21.15.92:0/2410344419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f681a0350 con 0x7f2f681083a0 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6e109640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68108cf0 0x7f2f6819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:17.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6d107640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 0x7f2f6819a980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400e9c0 tx=0x7f2f6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f5affd640 1 -- 172.21.15.92:0/2410344419 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400cd80 con 0x7f2f681083a0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f5affd640 1 -- 172.21.15.92:0/2410344419 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f640054d0 con 0x7f2f681083a0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f681a0640 con 0x7f2f681083a0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f5affd640 1 -- 172.21.15.92:0/2410344419 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64010430 con 0x7f2f681083a0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.201+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f681a6da0 con 0x7f2f681083a0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.204+0000 7f2f760e0740 1 Processor -- start 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.204+0000 7f2f760e0740 1 -- start start 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.204+0000 7f2f5affd640 1 -- 172.21.15.92:0/2410344419 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2f64004fd0 con 0x7f2f681083a0 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.204+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3dc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.204+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d00c1b0 0x56186d3dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.204+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 0x56186d3dd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f760e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56186d352e50 con 0x56186d014390 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f760e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56186d342280 con 0x56186d035070 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f760e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56186d346090 con 0x56186d00c1b0 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d107640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 0x56186d3dd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d00c1b0 0x56186d3dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d107640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 0x56186d3dd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35502/0 (socket says 172.21.15.92:35502) 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d107640 1 -- 172.21.15.92:0/468618779 learned_addr learned my addr 172.21.15.92:0/468618779 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6e109640 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 -- 172.21.15.92:0/468618779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 msgr2=0x56186d3dc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3dc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 -- 172.21.15.92:0/468618779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 msgr2=0x56186d3dd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 0x56186d3dd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 -- 172.21.15.92:0/468618779 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186d0a0650 con 0x56186d00c1b0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d107640 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 0x56186d3dd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f5affd640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2f540b6590 0x7f2f540b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6d908640 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d00c1b0 0x56186d3dcb90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c004630 tx=0x7f2f5c00bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e3d83973833f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.205+0000 7f2f6e109640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2f540b6590 0x7f2f540b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.206+0000 7f2f5affd640 1 -- 172.21.15.92:0/2410344419 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2f640e7750 con 0x7f2f681083a0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.206+0000 7f2f53fff640 1 -- 172.21.15.92:0/468618779 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c003bd0 con 0x56186d00c1b0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.206+0000 7f2f53fff640 1 -- 172.21.15.92:0/468618779 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c015070 con 0x56186d00c1b0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.206+0000 7f2f53fff640 1 -- 172.21.15.92:0/468618779 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c004c80 con 0x56186d00c1b0 2024-09-21T02:11:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 -- 172.21.15.92:0/468618779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d00c1b0 msgr2=0x56186d3dcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d00c1b0 0x56186d3dcb90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c004630 tx=0x7f2f5c00bb20 comp rx=0 tx=0).stop 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f6e109640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2f540b6590 0x7f2f540b8aa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f60004af0 tx=0x7f2f60005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 -- 172.21.15.92:0/468618779 shutdown_connections 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d014390 0x56186d3dd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d00c1b0 0x56186d3dcb90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 --2- 172.21.15.92:0/468618779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 -- 172.21.15.92:0/468618779 >> 172.21.15.92:0/468618779 conn(0x56186d029460 msgr2=0x56186d3da540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 -- 172.21.15.92:0/468618779 shutdown_connections 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 -- 172.21.15.92:0/468618779 wait complete. 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 -- start start 2024-09-21T02:11:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.207+0000 7f2f760e0740 1 Processor -- start 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56186d1a3700 con 0x56186d00c1b0 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56186d2dd110 con 0x56186d035070 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56186d3eae30 con 0x56186d014390 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f760e0740 10 client.0 _ll_register_callbacks cb 0x56186d2f6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3e7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35512/0 (socket says 172.21.15.92:35512) 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 -- 172.21.15.92:0/4159428545 learned_addr learned my addr 172.21.15.92:0/4159428545 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d107640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6e109640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 -- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 msgr2=0x56186d3e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 -- 172.21.15.92:0/4159428545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 msgr2=0x56186d3f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 -- 172.21.15.92:0/4159428545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186d30ac40 con 0x56186d00c1b0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f6d908640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3e7cc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bfa0 tx=0x7f2f5c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0d6f3179cb398f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f517fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.208+0000 7f2f517fa640 1 -- 172.21.15.92:0/4159428545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c028060 con 0x56186d00c1b0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f517fa640 1 -- 172.21.15.92:0/4159428545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2f5c011640 con 0x56186d00c1b0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5c015070 con 0x56186d00c1b0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f517fa640 1 -- 172.21.15.92:0/4159428545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c011840 con 0x56186d00c1b0 2024-09-21T02:11:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 10 client.17400 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56186d3f6b30 con 0x56186d00c1b0 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f517fa640 1 -- 172.21.15.92:0/4159428545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c015070 con 0x56186d00c1b0 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f517fa640 1 -- 172.21.15.92:0/4159428545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2f5c022070 con 0x56186d00c1b0 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 10 client.17400 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 20 client.17400 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cff39f0 0x56186d3f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 4 client.17400 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.209+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56186d354da0 con 0x56186cff39f0 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.210+0000 7f2f6d107640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cff39f0 0x56186d3f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.210+0000 7f2f6d107640 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cff39f0 0x56186d3f9f40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb705b8d2d7c1d9 server_cookie=db3bde2b0201aacc in_seq=0 out_seq=0 2024-09-21T02:11:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.210+0000 7f2f517fa640 10 client.17400 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f517fa640 1 -- 172.21.15.92:0/4159428545 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56186d354da0 con 0x56186cff39f0 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f517fa640 10 client.17400 handle_command_reply: tid=0 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 client.17400 shutdown 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 msgr2=0x56186d3e7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3e7cc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bfa0 tx=0x7f2f5c0043b0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 shutdown_connections 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cff39f0 0x56186d3f9f40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 --2- 172.21.15.92:0/4159428545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3e7cc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f517fa640 0 client.17400 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 >> 172.21.15.92:0/4159428545 conn(0x56186d029460 msgr2=0x56186d3df7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 shutdown_connections 2024-09-21T02:11:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.211+0000 7f2f760e0740 1 -- 172.21.15.92:0/4159428545 wait complete. 2024-09-21T02:11:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.212+0000 7f2f760e0740 20 client.17400 trim_cache size 0 max 16384 2024-09-21T02:11:17.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.228+0000 7f2f760e0740 1 Processor -- start 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 -- start start 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d00c1b0 0x56186d3ddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 0x56186d3da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681af440 con 0x56186d035070 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b0f80 con 0x56186d00c1b0 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f760e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681afb10 con 0x56186d014390 2024-09-21T02:11:17.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 0x56186d3da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 0x56186d3da0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35526/0 (socket says 172.21.15.92:35526) 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 -- 172.21.15.92:0/769467721 learned_addr learned my addr 172.21.15.92:0/769467721 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6d107640 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6d908640 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d00c1b0 0x56186d3ddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 -- 172.21.15.92:0/769467721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 msgr2=0x56186d3d9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3d9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 -- 172.21.15.92:0/769467721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d00c1b0 msgr2=0x56186d3ddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d00c1b0 0x56186d3ddd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 -- 172.21.15.92:0/769467721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186d3ea9a0 con 0x56186d035070 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6d107640 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:17.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.229+0000 7f2f6e109640 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 0x56186d3da0b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2f60032e80 tx=0x7f2f60040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1f9f2a239a9897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f537fe640 1 -- 172.21.15.92:0/769467721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60032eb0 con 0x56186d035070 2024-09-21T02:11:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f537fe640 1 -- 172.21.15.92:0/769467721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f60036050 con 0x56186d035070 2024-09-21T02:11:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f537fe640 1 -- 172.21.15.92:0/769467721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60030700 con 0x56186d035070 2024-09-21T02:11:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f760e0740 1 -- 172.21.15.92:0/769467721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 msgr2=0x56186d3da0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f760e0740 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 0x56186d3da0b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2f60032e80 tx=0x7f2f60040000 comp rx=0 tx=0).stop 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f760e0740 1 -- 172.21.15.92:0/769467721 shutdown_connections 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.230+0000 7f2f760e0740 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d035070 0x56186d3da0b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 --2- 172.21.15.92:0/769467721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d00c1b0 0x56186d3ddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 -- 172.21.15.92:0/769467721 >> 172.21.15.92:0/769467721 conn(0x56186d029460 msgr2=0x56186d3dd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 -- 172.21.15.92:0/769467721 shutdown_connections 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 -- 172.21.15.92:0/769467721 wait complete. 2024-09-21T02:11:17.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 -- start start 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 Processor -- start 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3faa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b08b0 con 0x56186d00c1b0 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.231+0000 7f2f760e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b01e0 con 0x56186d035070 2024-09-21T02:11:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f760e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681ad420 con 0x56186d014390 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f760e0740 10 client.0 _ll_register_callbacks cb 0x7f2f640ed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35536/0 (socket says 172.21.15.92:35536) 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 -- 172.21.15.92:0/3198199331 learned_addr learned my addr 172.21.15.92:0/3198199331 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d107640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6e109640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 -- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 msgr2=0x56186d3faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3faa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 -- 172.21.15.92:0/3198199331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 msgr2=0x56186d3fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 -- 172.21.15.92:0/3198199331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186d3fb5c0 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f6d908640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3fd5c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c010 tx=0x7f2f5c02e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eefafc438624c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f517fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f517fa640 1 -- 172.21.15.92:0/3198199331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c028050 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f517fa640 1 -- 172.21.15.92:0/3198199331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2f5c011be0 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.232+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f60036050 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f517fa640 1 -- 172.21.15.92:0/3198199331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c03d070 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 10 client.17406 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56186d464470 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f517fa640 1 -- 172.21.15.92:0/3198199331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f60036050 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f517fa640 1 -- 172.21.15.92:0/3198199331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2f5c019ce0 con 0x56186d00c1b0 2024-09-21T02:11:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 10 client.17406 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 20 client.17406 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cfc8da0 0x56186d466b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 4 client.17406 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56186d350850 con 0x56186cfc8da0 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.233+0000 7f2f6d107640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cfc8da0 0x56186d466b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f517fa640 10 client.17406 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f6d107640 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cfc8da0 0x56186d466b40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70366d4c49ab78f6 server_cookie=8ff14f8a1778436 in_seq=0 out_seq=0 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f517fa640 1 -- 172.21.15.92:0/3198199331 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x56186d350850 con 0x56186cfc8da0 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f517fa640 10 client.17406 handle_command_reply: tid=0 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 client.17406 shutdown 2024-09-21T02:11:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 msgr2=0x56186d3fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3fd5c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c010 tx=0x7f2f5c02e480 comp rx=0 tx=0).stop 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 shutdown_connections 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56186cfc8da0 0x56186d466b40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56186d035070 0x56186d3fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56186d014390 0x56186d3faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 --2- 172.21.15.92:0/3198199331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56186d00c1b0 0x56186d3fd5c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f517fa640 0 client.17406 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 >> 172.21.15.92:0/3198199331 conn(0x56186d029460 msgr2=0x56186d3e8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 shutdown_connections 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 1 -- 172.21.15.92:0/3198199331 wait complete. 2024-09-21T02:11:17.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.234+0000 7f2f760e0740 20 client.17406 trim_cache size 0 max 16384 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "c273971d-f2a7-4d64-b784-1321103a1652", 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.244 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.245 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.246 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.247 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:11:17.248 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:17.250 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:17.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:17.251 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2f540b6590 msgr2=0x7f2f540b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2f540b6590 0x7f2f540b8aa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f60004af0 tx=0x7f2f60005d80 comp rx=0 tx=0).stop 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 msgr2=0x7f2f6819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 0x7f2f6819a980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400e9c0 tx=0x7f2f6400ee80 comp rx=0 tx=0).stop 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 shutdown_connections 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2f540b6590 0x7f2f540b8aa0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68108cf0 0x7f2f6819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f681083a0 0x7f2f6819a980 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 --2- 172.21.15.92:0/2410344419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f68102a60 0x7f2f6819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 >> 172.21.15.92:0/2410344419 conn(0x7f2f68079730 msgr2=0x7f2f6806b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 shutdown_connections 2024-09-21T02:11:17.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:17.235+0000 7f2f6fb6c640 1 -- 172.21.15.92:0/2410344419 wait complete. 2024-09-21T02:11:18.036 INFO:tasks.cephfs.filesystem:scrub status for tag:c273971d-f2a7-4d64-b784-1321103a1652 - {'path': '/', 'tag': 'c273971d-f2a7-4d64-b784-1321103a1652', 'options': 'recursive,force'} 2024-09-21T02:11:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:18 smithi136 ceph-mon[34754]: pgmap v550: 97 pgs: 97 active+clean; 865 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:18.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:18 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:11:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:18 smithi092 ceph-mon[31190]: pgmap v550: 97 pgs: 97 active+clean; 865 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:18.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:18 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:11:18.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:18 smithi099 ceph-mon[34404]: pgmap v550: 97 pgs: 97 active+clean; 865 KiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:18.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:18 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:11:20.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:20 smithi136 ceph-mon[34754]: pgmap v551: 97 pgs: 97 active+clean; 943 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:11:20.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:20 smithi092 ceph-mon[31190]: pgmap v551: 97 pgs: 97 active+clean; 943 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:11:20.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:20 smithi099 ceph-mon[34404]: pgmap v551: 97 pgs: 97 active+clean; 943 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:11:22.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:22 smithi136 ceph-mon[34754]: pgmap v552: 97 pgs: 97 active+clean; 943 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:11:22.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:22 smithi092 ceph-mon[31190]: pgmap v552: 97 pgs: 97 active+clean; 943 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:11:22.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:22 smithi099 ceph-mon[34404]: pgmap v552: 97 pgs: 97 active+clean; 943 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:11:24.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:24 smithi136 ceph-mon[34754]: pgmap v553: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:11:24.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:24 smithi092 ceph-mon[31190]: pgmap v553: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:11:24.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:24 smithi099 ceph-mon[34404]: pgmap v553: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 20 op/s 2024-09-21T02:11:26.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:26 smithi136 ceph-mon[34754]: pgmap v554: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-21T02:11:26.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:26 smithi092 ceph-mon[31190]: pgmap v554: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-21T02:11:26.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:26 smithi099 ceph-mon[34404]: pgmap v554: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-21T02:11:28.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:28 smithi136 ceph-mon[34754]: pgmap v555: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:11:28.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:28 smithi092 ceph-mon[31190]: pgmap v555: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:11:28.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:28 smithi099 ceph-mon[34404]: pgmap v555: 97 pgs: 97 active+clean; 1000 KiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-21T02:11:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 -- 172.21.15.92:0/3659700554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d01093f0 msgr2=0x7fe1d010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 --2- 172.21.15.92:0/3659700554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d01093f0 0x7fe1d010b900 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009ca0 tx=0x7fe1c802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 -- 172.21.15.92:0/3659700554 shutdown_connections 2024-09-21T02:11:29.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 --2- 172.21.15.92:0/3659700554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d01093f0 0x7fe1d010b900 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 --2- 172.21.15.92:0/3659700554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1d01068e0 0x7fe1d0108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 --2- 172.21.15.92:0/3659700554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d006c450 0x7fe1d01063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 -- 172.21.15.92:0/3659700554 >> 172.21.15.92:0/3659700554 conn(0x7fe1d00fdf00 msgr2=0x7fe1d01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 -- 172.21.15.92:0/3659700554 shutdown_connections 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.570+0000 7fe1d658a640 1 -- 172.21.15.92:0/3659700554 wait complete. 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 Processor -- start 2024-09-21T02:11:29.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 -- start start 2024-09-21T02:11:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 0x7fe1d0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d01068e0 0x7fe1d019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1d01093f0 0x7fe1d019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d010f300 con 0x7fe1d006c450 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d010f180 con 0x7fe1d01093f0 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d658a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d010f480 con 0x7fe1d01068e0 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 0x7fe1d0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 0x7fe1d0199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55702/0 (socket says 172.21.15.92:55702) 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1cffff640 1 -- 172.21.15.92:0/2047141076 learned_addr learned my addr 172.21.15.92:0/2047141076 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1d01093f0 0x7fe1d019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.571+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d01068e0 0x7fe1d019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cffff640 1 -- 172.21.15.92:0/2047141076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d01068e0 msgr2=0x7fe1d019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cffff640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d01068e0 0x7fe1d019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cffff640 1 -- 172.21.15.92:0/2047141076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1d01093f0 msgr2=0x7fe1d019b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cffff640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1d01093f0 0x7fe1d019b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cffff640 1 -- 172.21.15.92:0/2047141076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d019b930 con 0x7fe1d006c450 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d01068e0 0x7fe1d019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cffff640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 0x7fe1d0199ef0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d770 tx=0x7fe1c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cd7fa640 1 -- 172.21.15.92:0/2047141076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0017070 con 0x7fe1d006c450 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cd7fa640 1 -- 172.21.15.92:0/2047141076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c00054d0 con 0x7fe1d006c450 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d019bc20 con 0x7fe1d006c450 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1cd7fa640 1 -- 172.21.15.92:0/2047141076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004c30 con 0x7fe1d006c450 2024-09-21T02:11:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.572+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d01a2310 con 0x7fe1d006c450 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 Processor -- start 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 -- start start 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1cd7fa640 1 -- 172.21.15.92:0/2047141076 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe1c0004e10 con 0x7fe1d006c450 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x565380769e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538076a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x56538076b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5653806dc6f0 con 0x5653803c21b0 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5653806cbb20 con 0x5653803a14d0 2024-09-21T02:11:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653806cf930 con 0x5653803992f0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x565380769e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x565380769e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55714/0 (socket says 172.21.15.92:55714) 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 -- 172.21.15.92:0/1565167101 learned_addr learned my addr 172.21.15.92:0/1565167101 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x56538076b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1cffff640 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538076a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 -- 172.21.15.92:0/1565167101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 msgr2=0x56538076a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538076a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 -- 172.21.15.92:0/1565167101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 msgr2=0x56538076b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x56538076b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.575+0000 7fe1d4b27640 1 -- 172.21.15.92:0/1565167101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565380691530 con 0x5653803c21b0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x565380769e50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8035000 tx=0x7fe1c802f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b11b416224651e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1cd7fa640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1b80b6510 0x7fe1b80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1b67fc640 1 -- 172.21.15.92:0/1565167101 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8031650 con 0x5653803c21b0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1cffff640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1b80b6510 0x7fe1b80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1b67fc640 1 -- 172.21.15.92:0/1565167101 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c8036030 con 0x5653803c21b0 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1b67fc640 1 -- 172.21.15.92:0/1565167101 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80047a0 con 0x5653803c21b0 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.576+0000 7fe1cd7fa640 1 -- 172.21.15.92:0/2047141076 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe1c00e6e10 con 0x7fe1d006c450 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/1565167101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 msgr2=0x565380769e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1cffff640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1b80b6510 0x7fe1b80b8a20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00a9550 tx=0x7fe1c0009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x565380769e50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8035000 tx=0x7fe1c802f4c0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/1565167101 shutdown_connections 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x56538076b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538076a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/1565167101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x565380769e50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/1565167101 >> 172.21.15.92:0/1565167101 conn(0x5653803b6300 msgr2=0x5653807658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/1565167101 shutdown_connections 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/1565167101 wait complete. 2024-09-21T02:11:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.577+0000 7fe1dd5cf740 1 -- start start 2024-09-21T02:11:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 Processor -- start 2024-09-21T02:11:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803992f0 0x5653807733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803a14d0 0x565380773a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x5653807745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565380603ab0 con 0x5653803c21b0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565380671730 con 0x5653803992f0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56538076b950 con 0x5653803a14d0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1dd5cf740 10 client.0 _ll_register_callbacks cb 0x5653806c56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x5653807745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x5653807745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55718/0 (socket says 172.21.15.92:55718) 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 -- 172.21.15.92:0/2376274807 learned_addr learned my addr 172.21.15.92:0/2376274807 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803a14d0 0x565380773a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803992f0 0x5653807733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 -- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803a14d0 msgr2=0x565380773a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803a14d0 0x565380773a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 -- 172.21.15.92:0/2376274807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803992f0 msgr2=0x5653807733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803992f0 0x5653807733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 -- 172.21.15.92:0/2376274807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56538042d790 con 0x5653803c21b0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x5653807745e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8031b50 tx=0x7fe1c80099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac26c3a29953723f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.578+0000 7fe1af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1af7fe640 1 -- 172.21.15.92:0/2376274807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8058060 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1af7fe640 1 -- 172.21.15.92:0/2376274807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe1c8003d80 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c8036030 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1af7fe640 1 -- 172.21.15.92:0/2376274807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1c802f660 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 10 client.17418 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653804a6620 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1af7fe640 1 -- 172.21.15.92:0/2376274807 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c8036030 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1af7fe640 1 -- 172.21.15.92:0/2376274807 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe1c8050070 con 0x5653803c21b0 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 10 client.17418 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 20 client.17418 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565380380d80 0x565380780730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 4 client.17418 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.579+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653806de640 con 0x565380380d80 2024-09-21T02:11:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.580+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565380380d80 0x565380780730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.580+0000 7fe1af7fe640 10 client.17418 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.580+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565380380d80 0x565380780730 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494e9ac169eb123a server_cookie=a71b4b9cc597a956 in_seq=0 out_seq=0 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1af7fe640 1 -- 172.21.15.92:0/2376274807 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653806de640 con 0x565380380d80 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1af7fe640 10 client.17418 handle_command_reply: tid=0 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 client.17418 shutdown 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 msgr2=0x5653807745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x5653807745e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8031b50 tx=0x7fe1c80099f0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 shutdown_connections 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565380380d80 0x565380780730 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803c21b0 0x5653807745e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803a14d0 0x565380773a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1af7fe640 0 client.17418 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/2376274807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803992f0 0x5653807733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 >> 172.21.15.92:0/2376274807 conn(0x5653803b6300 msgr2=0x565380763ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 shutdown_connections 2024-09-21T02:11:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/2376274807 wait complete. 2024-09-21T02:11:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.581+0000 7fe1dd5cf740 20 client.17418 trim_cache size 0 max 16384 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 Processor -- start 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 -- start start 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 0x565380765220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x565380769df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803c21b0 0x56538076a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01aaf40 con 0x5653803992f0 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01aca80 con 0x5653803a14d0 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ab610 con 0x5653803c21b0 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 0x565380765220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 0x565380765220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55732/0 (socket says 172.21.15.92:55732) 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 -- 172.21.15.92:0/3411709186 learned_addr learned my addr 172.21.15.92:0/3411709186 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x565380769df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 -- 172.21.15.92:0/3411709186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803c21b0 msgr2=0x56538076a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803c21b0 0x56538076a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 -- 172.21.15.92:0/3411709186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 msgr2=0x565380769df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x565380769df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 -- 172.21.15.92:0/3411709186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538076bd50 con 0x5653803992f0 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.596+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 0x565380765220 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009800 tx=0x7fe1c80309d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27d55e93b351cde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1b5ffb640 1 -- 172.21.15.92:0/3411709186 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8062070 con 0x5653803992f0 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1b5ffb640 1 -- 172.21.15.92:0/3411709186 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c8036040 con 0x5653803992f0 2024-09-21T02:11:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1b5ffb640 1 -- 172.21.15.92:0/3411709186 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8058040 con 0x5653803992f0 2024-09-21T02:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/3411709186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 msgr2=0x565380765220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 0x565380765220 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009800 tx=0x7fe1c80309d0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/3411709186 shutdown_connections 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803c21b0 0x56538076a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803a14d0 0x565380769df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.597+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/3411709186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803992f0 0x565380765220 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/3411709186 >> 172.21.15.92:0/3411709186 conn(0x5653803b6300 msgr2=0x5653807605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/3411709186 shutdown_connections 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/3411709186 wait complete. 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- start start 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 Processor -- start 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538077b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 0x565380779dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803c21b0 0x56538077a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01ac3b0 con 0x5653803a14d0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01abce0 con 0x5653803c21b0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01a8f20 con 0x5653803992f0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1dd5cf740 10 client.0 _ll_register_callbacks cb 0x7fe1c40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 0x565380779dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.598+0000 7fe1cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 0x565380779dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55740/0 (socket says 172.21.15.92:55740) 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 -- 172.21.15.92:0/359126533 learned_addr learned my addr 172.21.15.92:0/359126533 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538077b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803c21b0 0x56538077a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 -- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 msgr2=0x56538077b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538077b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 -- 172.21.15.92:0/359126533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803c21b0 msgr2=0x56538077a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803c21b0 0x56538077a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 -- 172.21.15.92:0/359126533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56538077aa70 con 0x5653803a14d0 2024-09-21T02:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803c21b0 0x56538077a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1cf7fe640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 0x565380779dc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x565380780ea0 tx=0x7fe1c4012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d5828e0588bdec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1af7fe640 1 -- 172.21.15.92:0/359126533 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4004320 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1af7fe640 1 -- 172.21.15.92:0/359126533 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe1c40035f0 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c8036040 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1af7fe640 1 -- 172.21.15.92:0/359126533 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1c40044c0 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1dd5cf740 10 client.17424 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653807eb190 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.599+0000 7fe1af7fe640 1 -- 172.21.15.92:0/359126533 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c8036040 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1af7fe640 1 -- 172.21.15.92:0/359126533 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe1c400f040 con 0x5653803a14d0 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1dd5cf740 10 client.17424 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1dd5cf740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1dd5cf740 20 client.17424 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5653803561c0 0x5653807edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1dd5cf740 4 client.17424 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:11:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653806da0f0 con 0x5653803561c0 2024-09-21T02:11:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5653803561c0 0x5653807edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1af7fe640 10 client.17424 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.600+0000 7fe1d4b27640 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5653803561c0 0x5653807edbb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9cb0fd6bdd0b6fb server_cookie=7af2b4703c7f8061 in_seq=0 out_seq=0 2024-09-21T02:11:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1af7fe640 1 -- 172.21.15.92:0/359126533 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653806da0f0 con 0x5653803561c0 2024-09-21T02:11:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1af7fe640 10 client.17424 handle_command_reply: tid=0 2024-09-21T02:11:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 client.17424 shutdown 2024-09-21T02:11:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 msgr2=0x565380779dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 0x565380779dc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x565380780ea0 tx=0x7fe1c4012950 comp rx=0 tx=0).stop 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 shutdown_connections 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5653803561c0 0x5653807edbb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653803c21b0 0x56538077a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5653803a14d0 0x565380779dc0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1af7fe640 0 client.17424 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 --2- 172.21.15.92:0/359126533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653803992f0 0x56538077b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 >> 172.21.15.92:0/359126533 conn(0x5653803b6300 msgr2=0x56538076df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 shutdown_connections 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 1 -- 172.21.15.92:0/359126533 wait complete. 2024-09-21T02:11:29.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.613+0000 7fe1dd5cf740 20 client.17424 trim_cache size 0 max 16384 2024-09-21T02:11:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:11:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:11:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:11:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:11:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1b80b6510 msgr2=0x7fe1b80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1b80b6510 0x7fe1b80b8a20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00a9550 tx=0x7fe1c0009f90 comp rx=0 tx=0).stop 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 msgr2=0x7fe1d0199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 0x7fe1d0199ef0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d770 tx=0x7fe1c000dc30 comp rx=0 tx=0).stop 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 shutdown_connections 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe1b80b6510 0x7fe1b80b8a20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1d01093f0 0x7fe1d019b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d01068e0 0x7fe1d019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 --2- 172.21.15.92:0/2047141076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d006c450 0x7fe1d0199ef0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 >> 172.21.15.92:0/2047141076 conn(0x7fe1d00fdf00 msgr2=0x7fe1d0104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 shutdown_connections 2024-09-21T02:11:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:29.614+0000 7fe1d658a640 1 -- 172.21.15.92:0/2047141076 wait complete. 2024-09-21T02:11:30.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:30 smithi136 ceph-mon[34754]: pgmap v556: 97 pgs: 97 active+clean; 1.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:11:30.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:30 smithi092 ceph-mon[31190]: pgmap v556: 97 pgs: 97 active+clean; 1.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:11:30.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:30 smithi099 ceph-mon[34404]: pgmap v556: 97 pgs: 97 active+clean; 1.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:11:32.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:32 smithi136 ceph-mon[34754]: pgmap v557: 97 pgs: 97 active+clean; 1.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:11:32.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:32 smithi092 ceph-mon[31190]: pgmap v557: 97 pgs: 97 active+clean; 1.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:11:32.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:32 smithi099 ceph-mon[34404]: pgmap v557: 97 pgs: 97 active+clean; 1.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:11:34.783 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:34 smithi136 ceph-mon[34754]: pgmap v558: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:11:34.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:34 smithi092 ceph-mon[31190]: pgmap v558: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:11:34.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:34 smithi099 ceph-mon[34404]: pgmap v558: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:11:36.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:36 smithi092 ceph-mon[31190]: pgmap v559: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:36.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:36 smithi099 ceph-mon[34404]: pgmap v559: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:36 smithi136 ceph-mon[34754]: pgmap v559: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:38.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:38 smithi092 ceph-mon[31190]: pgmap v560: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:11:38.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:38 smithi099 ceph-mon[34404]: pgmap v560: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:11:39.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:38 smithi136 ceph-mon[34754]: pgmap v560: 97 pgs: 97 active+clean; 1.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:11:40.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:40 smithi092 ceph-mon[31190]: pgmap v561: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:11:40.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:40 smithi099 ceph-mon[34404]: pgmap v561: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:11:41.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:40 smithi136 ceph-mon[34754]: pgmap v561: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:11:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 -- 172.21.15.92:0/1308384202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103640 msgr2=0x7f8ecc103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 --2- 172.21.15.92:0/1308384202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103640 0x7f8ecc103a50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb402a410 comp rx=0 tx=0).stop 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 -- 172.21.15.92:0/1308384202 shutdown_connections 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 --2- 172.21.15.92:0/1308384202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 --2- 172.21.15.92:0/1308384202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103640 0x7f8ecc103a50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 --2- 172.21.15.92:0/1308384202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ecc1025c0 0x7f8ecc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 -- 172.21.15.92:0/1308384202 >> 172.21.15.92:0/1308384202 conn(0x7f8ecc0fdf00 msgr2=0x7f8ecc1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.713+0000 7f8ed1524640 1 -- 172.21.15.92:0/1308384202 shutdown_connections 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 -- 172.21.15.92:0/1308384202 wait complete. 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 Processor -- start 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 -- start start 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ecc1025c0 0x7f8ecc06c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ecc103640 0x7f8ecc19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc1a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc10e210 con 0x7f8ecc103fe0 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc10e090 con 0x7f8ecc103640 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ed1524640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc10e390 con 0x7f8ecc1025c0 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.714+0000 7f8ecb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc1a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc1a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39552/0 (socket says 172.21.15.92:39552) 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 -- 172.21.15.92:0/3713159929 learned_addr learned my addr 172.21.15.92:0/3713159929 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ecc103640 0x7f8ecc19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ecc1025c0 0x7f8ecc06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 -- 172.21.15.92:0/3713159929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ecc1025c0 msgr2=0x7f8ecc06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ecc1025c0 0x7f8ecc06c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 -- 172.21.15.92:0/3713159929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ecc103640 msgr2=0x7f8ecc19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ecc103640 0x7f8ecc19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 -- 172.21.15.92:0/3713159929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1a1a00 con 0x7f8ecc103fe0 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ecb7fe640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc1a1200 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400eaa0 tx=0x7f8ec400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ec8ff9640 1 -- 172.21.15.92:0/3713159929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400cdb0 con 0x7f8ecc103fe0 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ec8ff9640 1 -- 172.21.15.92:0/3713159929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec4005020 con 0x7f8ecc103fe0 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1a1cf0 con 0x7f8ecc103fe0 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ec8ff9640 1 -- 172.21.15.92:0/3713159929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40116d0 con 0x7f8ecc103fe0 2024-09-21T02:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.715+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1a2190 con 0x7f8ecc103fe0 2024-09-21T02:11:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.718+0000 7f8ed848e740 1 Processor -- start 2024-09-21T02:11:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.718+0000 7f8ec8ff9640 1 -- 172.21.15.92:0/3713159929 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8ec40054a0 con 0x7f8ecc103fe0 2024-09-21T02:11:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.718+0000 7f8ed848e740 1 -- start start 2024-09-21T02:11:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.718+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e76890260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768908d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e76891420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ed848e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557e767fe6f0 con 0x557e764e41b0 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ed848e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e767edb20 con 0x557e764bb2f0 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ed848e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557e767f1930 con 0x557e764c34d0 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e76890260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e76890260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39554/0 (socket says 172.21.15.92:39554) 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec3fff640 1 -- 172.21.15.92:0/2439880086 learned_addr learned my addr 172.21.15.92:0/2439880086 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768908d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e76891420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec8ff9640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8eb80b6510 0x7f8eb80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2439880086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 msgr2=0x557e76891420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e76891420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2439880086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 msgr2=0x557e76890260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e76890260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2439880086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e767b3530 con 0x557e764bb2f0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e76890260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecb7fe640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8eb80b6510 0x7f8eb80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768908d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005fd0 tx=0x7f8ebc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4928bf5affe1d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec8ff9640 1 -- 172.21.15.92:0/3713159929 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8ec40e73a0 con 0x7f8ecc103fe0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.719+0000 7f8ec17fa640 1 -- 172.21.15.92:0/2439880086 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc01c680 con 0x557e764bb2f0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ec17fa640 1 -- 172.21.15.92:0/2439880086 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc00be20 con 0x557e764bb2f0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ec17fa640 1 -- 172.21.15.92:0/2439880086 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011b90 con 0x557e764bb2f0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ed848e740 1 -- 172.21.15.92:0/2439880086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 msgr2=0x557e768908d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ecb7fe640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8eb80b6510 0x7f8eb80b8a20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40aa830 tx=0x7f8ec4089040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768908d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005fd0 tx=0x7f8ebc0107a0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ed848e740 1 -- 172.21.15.92:0/2439880086 shutdown_connections 2024-09-21T02:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.720+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e76891420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768908d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2439880086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e76890260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- 172.21.15.92:0/2439880086 >> 172.21.15.92:0/2439880086 conn(0x557e764d8300 msgr2=0x557e768878e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- 172.21.15.92:0/2439880086 shutdown_connections 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- 172.21.15.92:0/2439880086 wait complete. 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- start start 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 Processor -- start 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e7689a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e7689ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557e76725ab0 con 0x557e764e41b0 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e76793730 con 0x557e764bb2f0 2024-09-21T02:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557e76891f40 con 0x557e764c34d0 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ed848e740 10 client.0 _ll_register_callbacks cb 0x557e767e76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:38386/0 (socket says 172.21.15.92:38386) 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ec3fff640 1 -- 172.21.15.92:0/402904099 learned_addr learned my addr 172.21.15.92:0/402904099 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.721+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e7689a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e7689ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ecaffd640 1 -- 172.21.15.92:0/402904099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 msgr2=0x557e768999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ecaffd640 1 -- 172.21.15.92:0/402904099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 msgr2=0x557e7689ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e7689ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ecaffd640 1 -- 172.21.15.92:0/402904099 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e7654f790 con 0x557e764c34d0 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e7689a0b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005d00 tx=0x7f8ebc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2515f96ff6ff13b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8eb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8eb27fc640 1 -- 172.21.15.92:0/402904099 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011cd0 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc00be20 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8eb27fc640 1 -- 172.21.15.92:0/402904099 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8ebc004020 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8eb27fc640 1 -- 172.21.15.92:0/402904099 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ebc004220 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ed848e740 10 client.26569 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.722+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e765c8620 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8eb27fc640 1 -- 172.21.15.92:0/402904099 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc00be20 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8eb27fc640 1 -- 172.21.15.92:0/402904099 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ebc004880 con 0x557e764c34d0 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ed848e740 10 client.26569 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ed848e740 10 client.26569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ed848e740 20 client.26569 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ed848e740 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764a2d80 0x557e768a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ed848e740 4 client.26569 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e76800640 con 0x557e764a2d80 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764a2d80 0x557e768a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8eb27fc640 10 client.26569 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.723+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764a2d80 0x557e768a6e10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cf6869d26637d7 server_cookie=4413add43fa507b5 in_seq=0 out_seq=0 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8eb27fc640 1 -- 172.21.15.92:0/402904099 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e76800640 con 0x557e764a2d80 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8eb27fc640 10 client.26569 handle_command_reply: tid=0 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 client.26569 shutdown 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 msgr2=0x557e7689a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e7689a0b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc005d00 tx=0x7f8ebc0118d0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 shutdown_connections 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764a2d80 0x557e768a6e10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e7689ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e7689a0b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8eb27fc640 0 client.26569 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 --2- 172.21.15.92:0/402904099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 >> 172.21.15.92:0/402904099 conn(0x557e764d8300 msgr2=0x557e76885ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 shutdown_connections 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.724+0000 7f8ed848e740 1 -- 172.21.15.92:0/402904099 wait complete. 2024-09-21T02:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.725+0000 7f8ed848e740 20 client.26569 trim_cache size 0 max 16384 2024-09-21T02:11:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 Processor -- start 2024-09-21T02:11:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 -- start start 2024-09-21T02:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e76887d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 0x557e768a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764e41b0 0x557e76888260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1a8eb0 con 0x557e764c34d0 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1aa9f0 con 0x557e764bb2f0 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ed848e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1a9580 con 0x557e764e41b0 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.739+0000 7f8ecaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 0x557e768a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 0x557e768a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39572/0 (socket says 172.21.15.92:39572) 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2061842122 learned_addr learned my addr 172.21.15.92:0/2061842122 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e76887d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764e41b0 0x557e76888260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2061842122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764e41b0 msgr2=0x557e76888260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764e41b0 0x557e76888260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2061842122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 msgr2=0x557e76887d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e76887d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 -- 172.21.15.92:0/2061842122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e768830a0 con 0x557e764c34d0 2024-09-21T02:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e76887d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 0x557e768a1b00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0118a0 tx=0x7f8ebc011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d496eb1aae257c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ec0ff9640 1 -- 172.21.15.92:0/2061842122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc037070 con 0x557e764c34d0 2024-09-21T02:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ec0ff9640 1 -- 172.21.15.92:0/2061842122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc00b810 con 0x557e764c34d0 2024-09-21T02:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.740+0000 7f8ec0ff9640 1 -- 172.21.15.92:0/2061842122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc004e40 con 0x557e764c34d0 2024-09-21T02:11:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 -- 172.21.15.92:0/2061842122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 msgr2=0x557e768a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 0x557e768a1b00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0118a0 tx=0x7f8ebc011650 comp rx=0 tx=0).stop 2024-09-21T02:11:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 -- 172.21.15.92:0/2061842122 shutdown_connections 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764e41b0 0x557e76888260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764c34d0 0x557e768a1b00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 --2- 172.21.15.92:0/2061842122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e76887d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 -- 172.21.15.92:0/2061842122 >> 172.21.15.92:0/2061842122 conn(0x557e764d8300 msgr2=0x557e76898c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 -- 172.21.15.92:0/2061842122 shutdown_connections 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 -- 172.21.15.92:0/2061842122 wait complete. 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.741+0000 7f8ed848e740 1 -- start start 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 Processor -- start 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e768a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e768a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1aa320 con 0x557e764e41b0 2024-09-21T02:11:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1a9c50 con 0x557e764bb2f0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1a6e90 con 0x557e764c34d0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ed848e740 10 client.0 _ll_register_callbacks cb 0x7f8eb4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e768a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e768a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39576/0 (socket says 172.21.15.92:39576) 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 -- 172.21.15.92:0/3478630932 learned_addr learned my addr 172.21.15.92:0/3478630932 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e768a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 -- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 msgr2=0x557e768a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e768a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 -- 172.21.15.92:0/3478630932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 msgr2=0x557e768a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 -- 172.21.15.92:0/3478630932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e768a0ce0 con 0x557e764e41b0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8ec3fff640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e768a0770 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4031440 tx=0x7f8eb402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5b79461f82f35b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.742+0000 7f8eb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8eb27fc640 1 -- 172.21.15.92:0/3478630932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4059060 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8eb27fc640 1 -- 172.21.15.92:0/3478630932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8eb4032190 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e767fc0f0 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8eb27fc640 1 -- 172.21.15.92:0/3478630932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8eb4035040 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 10 client.17442 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e7689fc10 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8eb27fc640 1 -- 172.21.15.92:0/3478630932 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e767fc0f0 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8eb27fc640 1 -- 172.21.15.92:0/3478630932 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8eb404c040 con 0x557e764e41b0 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 10 client.17442 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 10 client.17442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 20 client.17442 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764781c0 0x557e76913f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 4 client.17442 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:11:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e767f9520 con 0x557e764781c0 2024-09-21T02:11:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.743+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764781c0 0x557e76913f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.744+0000 7f8eb27fc640 10 client.17442 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.744+0000 7f8ecaffd640 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764781c0 0x557e76913f30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96505799900f4499 server_cookie=24e494d6223d3100 in_seq=0 out_seq=0 2024-09-21T02:11:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.754+0000 7f8eb27fc640 1 -- 172.21.15.92:0/3478630932 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e767f9520 con 0x557e764781c0 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.754+0000 7f8eb27fc640 10 client.17442 handle_command_reply: tid=0 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.754+0000 7f8ed848e740 1 client.17442 shutdown 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 msgr2=0x557e768a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e768a0770 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4031440 tx=0x7f8eb402a410 comp rx=0 tx=0).stop 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 shutdown_connections 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x557e764781c0 0x557e76913f30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e764e41b0 0x557e768a0770 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e764c34d0 0x557e768a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8eb27fc640 0 client.17442 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 --2- 172.21.15.92:0/3478630932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e764bb2f0 0x557e768a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 >> 172.21.15.92:0/3478630932 conn(0x557e764d8300 msgr2=0x557e76897ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 shutdown_connections 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 1 -- 172.21.15.92:0/3478630932 wait complete. 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed848e740 20 client.17442 trim_cache size 0 max 16384 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8eb80b6510 msgr2=0x7f8eb80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8eb80b6510 0x7f8eb80b8a20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40aa830 tx=0x7f8ec4089040 comp rx=0 tx=0).stop 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 msgr2=0x7f8ecc1a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc1a1200 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400eaa0 tx=0x7f8ec400ef60 comp rx=0 tx=0).stop 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 shutdown_connections 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8eb80b6510 0x7f8eb80b8a20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ecc103fe0 0x7f8ecc1a1200 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ecc103640 0x7f8ecc19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.755+0000 7f8ed1524640 1 --2- 172.21.15.92:0/3713159929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ecc1025c0 0x7f8ecc06c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.756+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 >> 172.21.15.92:0/3713159929 conn(0x7f8ecc0fdf00 msgr2=0x7f8ecc106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.756+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 shutdown_connections 2024-09-21T02:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:42.756+0000 7f8ed1524640 1 -- 172.21.15.92:0/3713159929 wait complete. 2024-09-21T02:11:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:42 smithi092 ceph-mon[31190]: pgmap v562: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:42.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:11:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:42 smithi099 ceph-mon[34404]: pgmap v562: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:42.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:11:43.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:42 smithi136 ceph-mon[34754]: pgmap v562: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:11:43.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.805 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: Unable to set osd_memory_target on smithi092 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: Unable to set osd_memory_target on smithi099 to 184565964: error parsing value: Value '184565964' is below minimum 939524096 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:11:43.806 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:11:43.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-21T02:11:43.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-21T02:11:43.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T02:11:43.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: Unable to set osd_memory_target on smithi092 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-21T02:11:43.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: Unable to set osd_memory_target on smithi099 to 184565964: error parsing value: Value '184565964' is below minimum 939524096 2024-09-21T02:11:43.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:11:43.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:11:43.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:11:44.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi136 to 176.0M 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-21T02:11:44.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: Unable to set osd_memory_target on smithi136 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T02:11:44.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: Unable to set osd_memory_target on smithi092 to 184576716: error parsing value: Value '184576716' is below minimum 939524096 2024-09-21T02:11:44.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: Unable to set osd_memory_target on smithi099 to 184565964: error parsing value: Value '184565964' is below minimum 939524096 2024-09-21T02:11:44.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:11:44.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:11:44.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:11:45.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:44 smithi136 ceph-mon[34754]: pgmap v563: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:11:45.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:44 smithi092 ceph-mon[31190]: pgmap v563: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:11:45.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:44 smithi099 ceph-mon[34404]: pgmap v563: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-21T02:11:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:46 smithi136 ceph-mon[34754]: pgmap v564: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:46 smithi092 ceph-mon[31190]: pgmap v564: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:46 smithi099 ceph-mon[34404]: pgmap v564: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:11:48.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:11:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:313/600: open fd = 3 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:314/600: open fd = 3 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.715 INFO:tasks.workunit.client.0.smithi092.stdout:315/600: open fd = 3 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:316/600: open fd = 3 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:317/600: open fd = 3 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:318/600: open fd = 3 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:319/600: open fd = 3 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:320/600: open fd = 3 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:321/600: open fd = 3 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:322/600: open fd = 3 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.717 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:323/600: open fd = 3 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:324/600: open fd = 3 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:325/600: open fd = 3 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:326/600: open fd = 3 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.718 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:327/600: open fd = 3 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:328/600: open fd = 3 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:329/600: open fd = 3 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:330/600: open fd = 3 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.719 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:331/600: open fd = 3 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:332/600: open fd = 3 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:333/600: open fd = 3 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:334/600: open fd = 3 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.720 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:335/600: open fd = 3 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:336/600: open fd = 3 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:337/600: open fd = 3 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:338/600: open fd = 3 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.721 INFO:tasks.workunit.client.0.smithi092.stdout:339/600: open fd = 3 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:340/600: open fd = 3 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:341/600: open fd = 3 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:342/600: open fd = 3 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.722 INFO:tasks.workunit.client.0.smithi092.stdout:343/600: open fd = 3 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:344/600: open fd = 3 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:345/600: open fd = 3 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:346/600: open fd = 3 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:347/600: open fd = 3 2024-09-21T02:11:48.723 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:348/600: open fd = 3 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:349/600: open fd = 3 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:350/600: open fd = 3 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:351/600: open fd = 3 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.724 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:352/600: open fd = 3 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:353/600: open fd = 3 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:354/600: open fd = 3 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:355/600: open fd = 3 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.725 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:356/600: open fd = 3 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:357/600: open fd = 3 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:358/600: open fd = 3 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:359/600: open fd = 3 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.726 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:360/600: open fd = 3 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:361/600: open fd = 3 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:362/600: open fd = 3 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:363/600: open fd = 3 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.727 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:364/600: open fd = 3 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:365/600: open fd = 3 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:366/600: open fd = 3 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:367/600: open fd = 3 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.728 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:368/600: open fd = 3 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:369/600: open fd = 3 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:370/600: open fd = 3 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.729 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:371/600: open fd = 3 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:372/600: open fd = 3 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:373/600: open fd = 3 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:374/600: open fd = 3 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.730 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:11:48.731 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 -- 172.21.15.92:0/1597384291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103950 msgr2=0x7fa55c10f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 --2- 172.21.15.92:0/1597384291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103950 0x7fa55c10f500 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa554009ca0 tx=0x7fa55402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 -- 172.21.15.92:0/1597384291 shutdown_connections 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 --2- 172.21.15.92:0/1597384291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103950 0x7fa55c10f500 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 --2- 172.21.15.92:0/1597384291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c103000 0x7fa55c103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 --2- 172.21.15.92:0/1597384291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c107690 0x7fa55c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 -- 172.21.15.92:0/1597384291 >> 172.21.15.92:0/1597384291 conn(0x7fa55c079ea0 msgr2=0x7fa55c07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 -- 172.21.15.92:0/1597384291 shutdown_connections 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.784+0000 7fa562baf640 1 -- 172.21.15.92:0/1597384291 wait complete. 2024-09-21T02:11:48.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 Processor -- start 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 -- start start 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 0x7fa55c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c103950 0x7fa55c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c107690 0x7fa55c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c112a40 con 0x7fa55c103000 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1128c0 con 0x7fa55c107690 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa562baf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c112bc0 con 0x7fa55c103950 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 0x7fa55c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 0x7fa55c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38024/0 (socket says 172.21.15.92:38024) 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 -- 172.21.15.92:0/1296057390 learned_addr learned my addr 172.21.15.92:0/1296057390 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa55bfff640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c103950 0x7fa55c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56114c640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c107690 0x7fa55c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 -- 172.21.15.92:0/1296057390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c103950 msgr2=0x7fa55c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c103950 0x7fa55c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 -- 172.21.15.92:0/1296057390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c107690 msgr2=0x7fa55c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c107690 0x7fa55c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56094b640 1 -- 172.21.15.92:0/1296057390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c19f260 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa56114c640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c107690 0x7fa55c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.785+0000 7fa55bfff640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c103950 0x7fa55c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.786+0000 7fa56094b640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 0x7fa55c1993b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c890 tx=0x7fa54c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.786+0000 7fa559ffb640 1 -- 172.21.15.92:0/1296057390 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c004220 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.786+0000 7fa559ffb640 1 -- 172.21.15.92:0/1296057390 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa54c0043c0 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.786+0000 7fa559ffb640 1 -- 172.21.15.92:0/1296057390 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c004a00 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.786+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c19f550 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.786+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c1a5d70 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.788+0000 7fa559ffb640 1 -- 172.21.15.92:0/1296057390 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa54c0079b0 con 0x7fa55c103000 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 Processor -- start 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 -- start start 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6868270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6868cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6b67dee50 con 0x55d6b64a0390 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d6b67ce280 con 0x55d6b64c1070 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa569123740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6b67d2090 con 0x55d6b64981b0 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6868cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6868cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38036/0 (socket says 172.21.15.92:38036) 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 -- 172.21.15.92:0/4268644773 learned_addr learned my addr 172.21.15.92:0/4268644773 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa56114c640 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6868270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa559ffb640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa5440b6540 0x7fa5440b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa56094b640 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 -- 172.21.15.92:0/4268644773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 msgr2=0x55d6b68687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa56114c640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa5440b6540 0x7fa5440b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 -- 172.21.15.92:0/4268644773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 msgr2=0x55d6b6868270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa55bfff640 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6868270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.789+0000 7fa559ffb640 1 -- 172.21.15.92:0/1296057390 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa54c0e7130 con 0x7fa55c103000 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa55bfff640 1 -- 172.21.15.92:0/4268644773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b652c650 con 0x55d6b64a0390 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa56094b640 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa55bfff640 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6868cf0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa550008bd0 tx=0x7fa55000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3444224af93f0397 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa56114c640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa5440b6540 0x7fa5440b8a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa554009a60 tx=0x7fa55402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa542ffd640 1 -- 172.21.15.92:0/4268644773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004030 con 0x55d6b64a0390 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa542ffd640 1 -- 172.21.15.92:0/4268644773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550005020 con 0x55d6b64a0390 2024-09-21T02:11:48.795 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.790+0000 7fa542ffd640 1 -- 172.21.15.92:0/4268644773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004b10 con 0x55d6b64a0390 2024-09-21T02:11:48.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 -- 172.21.15.92:0/4268644773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 msgr2=0x55d6b6868cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6868cf0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa550008bd0 tx=0x7fa55000e720 comp rx=0 tx=0).stop 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 -- 172.21.15.92:0/4268644773 shutdown_connections 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6868cf0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 --2- 172.21.15.92:0/4268644773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6868270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 -- 172.21.15.92:0/4268644773 >> 172.21.15.92:0/4268644773 conn(0x55d6b64b5460 msgr2=0x55d6b6866160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 -- 172.21.15.92:0/4268644773 shutdown_connections 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 -- 172.21.15.92:0/4268644773 wait complete. 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.791+0000 7fa569123740 1 -- start start 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 Processor -- start 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6873e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b687fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6b662f700 con 0x55d6b64a0390 2024-09-21T02:11:48.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d6b6769110 con 0x55d6b64c1070 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6b6876a50 con 0x55d6b64981b0 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa569123740 10 client.0 _ll_register_callbacks cb 0x55d6b6782990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6873e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6873e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38050/0 (socket says 172.21.15.92:38050) 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa55bfff640 1 -- 172.21.15.92:0/1561544741 learned_addr learned my addr 172.21.15.92:0/1561544741 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa56094b640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.792+0000 7fa56114c640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b687fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa55bfff640 1 -- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 msgr2=0x55d6b68738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa55bfff640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa55bfff640 1 -- 172.21.15.92:0/1561544741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 msgr2=0x55d6b687fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa55bfff640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b687fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa55bfff640 1 -- 172.21.15.92:0/1561544741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b6796c40 con 0x55d6b64a0390 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa56094b640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:48.800 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa55bfff640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6873e10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa550005890 tx=0x7fa5500088d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a93763a6443fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa53bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa53bfff640 1 -- 172.21.15.92:0/1561544741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004c20 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa53bfff640 1 -- 172.21.15.92:0/1561544741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa550004dc0 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa550005020 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa53bfff640 1 -- 172.21.15.92:0/1561544741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5500115d0 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa569123740 10 client.17454 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d6b68826a0 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.793+0000 7fa53bfff640 1 -- 172.21.15.92:0/1561544741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550005020 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa53bfff640 1 -- 172.21.15.92:0/1561544741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa5500248c0 con 0x55d6b64a0390 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa569123740 10 client.17454 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa569123740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa569123740 20 client.17454 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa569123740 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b647f9f0 0x55d6b6885a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa569123740 4 client.17454 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6b67e0da0 con 0x55d6b647f9f0 2024-09-21T02:11:48.801 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa56094b640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b647f9f0 0x55d6b6885a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa53bfff640 10 client.17454 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.794+0000 7fa56094b640 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b647f9f0 0x55d6b6885a00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fb357972fd5704 server_cookie=bb53fe79218288f7 in_seq=0 out_seq=0 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa53bfff640 1 -- 172.21.15.92:0/1561544741 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6b67e0da0 con 0x55d6b647f9f0 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa53bfff640 10 client.17454 handle_command_reply: tid=0 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 client.17454 shutdown 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 msgr2=0x55d6b6873e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6873e10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa550005890 tx=0x7fa5500088d0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 shutdown_connections 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b647f9f0 0x55d6b6885a00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b687fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6873e10 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa53bfff640 0 client.17454 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 --2- 172.21.15.92:0/1561544741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b68738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.795+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 >> 172.21.15.92:0/1561544741 conn(0x55d6b64b5460 msgr2=0x55d6b686b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.796+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 shutdown_connections 2024-09-21T02:11:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.796+0000 7fa569123740 1 -- 172.21.15.92:0/1561544741 wait complete. 2024-09-21T02:11:48.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.796+0000 7fa569123740 20 client.17454 trim_cache size 0 max 16384 2024-09-21T02:11:48.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 Processor -- start 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 -- start start 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b6875500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6865710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6865c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1ad240 con 0x55d6b64a0390 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1aed80 con 0x55d6b64c1070 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa569123740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1ad910 con 0x55d6b64981b0 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6865710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6865710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38058/0 (socket says 172.21.15.92:38058) 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa55bfff640 1 -- 172.21.15.92:0/3793664591 learned_addr learned my addr 172.21.15.92:0/3793664591 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa56094b640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b6875500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa56114c640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6865c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa55bfff640 1 -- 172.21.15.92:0/3793664591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 msgr2=0x55d6b6875500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa55bfff640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b6875500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.813+0000 7fa55bfff640 1 -- 172.21.15.92:0/3793664591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 msgr2=0x55d6b6865c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa55bfff640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6865c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa55bfff640 1 -- 172.21.15.92:0/3793664591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b6876400 con 0x55d6b64a0390 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa56114c640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6865c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa56094b640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b6875500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa55bfff640 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6865710 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa550006010 tx=0x7fa550005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629f0f8c66f0cfdf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa5427fc640 1 -- 172.21.15.92:0/3793664591 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550033030 con 0x55d6b64a0390 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa5427fc640 1 -- 172.21.15.92:0/3793664591 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550019420 con 0x55d6b64a0390 2024-09-21T02:11:48.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.814+0000 7fa5427fc640 1 -- 172.21.15.92:0/3793664591 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004500 con 0x55d6b64a0390 2024-09-21T02:11:48.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 -- 172.21.15.92:0/3793664591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 msgr2=0x55d6b6865710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6865710 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa550006010 tx=0x7fa550005520 comp rx=0 tx=0).stop 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 -- 172.21.15.92:0/3793664591 shutdown_connections 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6865c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64a0390 0x55d6b6865710 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 --2- 172.21.15.92:0/3793664591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64981b0 0x55d6b6875500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 -- 172.21.15.92:0/3793664591 >> 172.21.15.92:0/3793664591 conn(0x55d6b64b5460 msgr2=0x55d6b6874df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 -- 172.21.15.92:0/3793664591 shutdown_connections 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 -- 172.21.15.92:0/3793664591 wait complete. 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 -- start start 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.815+0000 7fa569123740 1 Processor -- start 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 0x55d6b68892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64a0390 0x55d6b6886770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6886d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1ae6b0 con 0x55d6b64981b0 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1adfe0 con 0x55d6b64c1070 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c1ab220 con 0x55d6b64a0390 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa569123740 10 client.0 _ll_register_callbacks cb 0x7fa550008a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 0x55d6b68892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 0x55d6b68892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38072/0 (socket says 172.21.15.92:38072) 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 -- 172.21.15.92:0/2332309379 learned_addr learned my addr 172.21.15.92:0/2332309379 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa55bfff640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64a0390 0x55d6b6886770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56114c640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6886d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 -- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64a0390 msgr2=0x55d6b6886770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64a0390 0x55d6b6886770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 -- 172.21.15.92:0/2332309379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 msgr2=0x55d6b6886d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6886d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 -- 172.21.15.92:0/2332309379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b68874c0 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa56094b640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 0x55d6b68892a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0e4c60 tx=0x7fa54c008620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a764cd54f8b534 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.816+0000 7fa53bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa53bfff640 1 -- 172.21.15.92:0/2332309379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c01d040 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa53bfff640 1 -- 172.21.15.92:0/2332309379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa54c0a9510 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa550019420 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa53bfff640 1 -- 172.21.15.92:0/2332309379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa54c0acdb0 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 10 client.17460 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d6b68f0090 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa53bfff640 1 -- 172.21.15.92:0/2332309379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550019420 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa53bfff640 1 -- 172.21.15.92:0/2332309379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa54c0a8bf0 con 0x55d6b64981b0 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 10 client.17460 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 20 client.17460 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b6454da0 0x55d6b68f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 4 client.17460 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6b67dc850 con 0x55d6b6454da0 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.817+0000 7fa55bfff640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b6454da0 0x55d6b68f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa53bfff640 10 client.17460 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa55bfff640 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b6454da0 0x55d6b68f27b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487293be92ba817a server_cookie=4271584ab4f2efc9 in_seq=0 out_seq=0 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa53bfff640 1 -- 172.21.15.92:0/2332309379 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55d6b67dc850 con 0x55d6b6454da0 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa53bfff640 10 client.17460 handle_command_reply: tid=0 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 client.17460 shutdown 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 msgr2=0x55d6b68892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 0x55d6b68892a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0e4c60 tx=0x7fa54c008620 comp rx=0 tx=0).stop 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 shutdown_connections 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d6b6454da0 0x55d6b68f27b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6b64c1070 0x55d6b6886d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6b64a0390 0x55d6b6886770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 --2- 172.21.15.92:0/2332309379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6b64981b0 0x55d6b68892a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 >> 172.21.15.92:0/2332309379 conn(0x55d6b64b5460 msgr2=0x55d6b6873e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa53bfff640 0 client.17460 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.818+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 shutdown_connections 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa569123740 1 -- 172.21.15.92:0/2332309379 wait complete. 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa569123740 20 client.17460 trim_cache size 0 max 16384 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.824 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:11:48.825 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:11:48.826 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.827 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:11:48.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:48.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:11:48.830 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:11:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa5440b6540 msgr2=0x7fa5440b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa5440b6540 0x7fa5440b8a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa554009a60 tx=0x7fa55402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 msgr2=0x7fa55c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 0x7fa55c1993b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c890 tx=0x7fa54c00cd50 comp rx=0 tx=0).stop 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 shutdown_connections 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa5440b6540 0x7fa5440b8a50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa55c107690 0x7fa55c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa55c103950 0x7fa55c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 --2- 172.21.15.92:0/1296057390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa55c103000 0x7fa55c1993b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 >> 172.21.15.92:0/1296057390 conn(0x7fa55c079ea0 msgr2=0x7fa55c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 shutdown_connections 2024-09-21T02:11:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:48.819+0000 7fa562baf640 1 -- 172.21.15.92:0/1296057390 wait complete. 2024-09-21T02:11:49.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:48 smithi136 ceph-mon[34754]: pgmap v565: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:11:49.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:48 smithi092 ceph-mon[31190]: pgmap v565: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:11:49.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:48 smithi099 ceph-mon[34404]: pgmap v565: 97 pgs: 97 active+clean; 1.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:11:49.675 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:11:49.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:11:49.926 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.472+0000 7f2a3993b640 1 -- 172.21.15.92:0/3054593131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a34105300 msgr2=0x7f2a3410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 --2- 172.21.15.92:0/3054593131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a34105300 0x7f2a3410d850 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009900 tx=0x7f2a2802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 -- 172.21.15.92:0/3054593131 shutdown_connections 2024-09-21T02:11:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 --2- 172.21.15.92:0/3054593131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a3410de10 0x7f2a34110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 --2- 172.21.15.92:0/3054593131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a34105300 0x7f2a3410d850 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 --2- 172.21.15.92:0/3054593131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a341049b0 0x7f2a34104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 -- 172.21.15.92:0/3054593131 >> 172.21.15.92:0/3054593131 conn(0x7f2a34079730 msgr2=0x7f2a34079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 -- 172.21.15.92:0/3054593131 shutdown_connections 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 -- 172.21.15.92:0/3054593131 wait complete. 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 Processor -- start 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.473+0000 7f2a3993b640 1 -- start start 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a3993b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a341049b0 0x7f2a3419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a3993b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a34105300 0x7f2a3419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a3993b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 0x7f2a3419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a3993b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34113a70 con 0x7f2a3410de10 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a3993b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a341138f0 con 0x7f2a341049b0 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a3993b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34113bf0 con 0x7f2a34105300 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 0x7f2a3419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:50.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 0x7f2a3419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38102/0 (socket says 172.21.15.92:38102) 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 -- 172.21.15.92:0/1363121875 learned_addr learned my addr 172.21.15.92:0/1363121875 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a327fc640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a34105300 0x7f2a3419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a32ffd640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a341049b0 0x7f2a3419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 -- 172.21.15.92:0/1363121875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a34105300 msgr2=0x7f2a3419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a34105300 0x7f2a3419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 -- 172.21.15.92:0/1363121875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a341049b0 msgr2=0x7f2a3419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a341049b0 0x7f2a3419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 -- 172.21.15.92:0/1363121875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a341a0370 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.474+0000 7f2a337fe640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 0x7f2a3419fd00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00d750 tx=0x7f2a2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.475+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c017070 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.475+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a341a0660 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.475+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a2c0054d0 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.475+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c004c60 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.475+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341a6e80 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.476+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a34104dc0 con 0x7f2a3410de10 2024-09-21T02:11:50.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.477+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2a2c009bd0 con 0x7f2a3410de10 2024-09-21T02:11:50.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.478+0000 7f2a38939640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a1c0b6540 0x7f2a1c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:50.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.478+0000 7f2a32ffd640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a1c0b6540 0x7f2a1c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:50.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.478+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2a2c0e6f40 con 0x7f2a3410de10 2024-09-21T02:11:50.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.479+0000 7f2a32ffd640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a1c0b6540 0x7f2a1c0b8a50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a3406bc60 tx=0x7f2a24007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:50.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.482+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a2c0b1b90 con 0x7f2a3410de10 2024-09-21T02:11:50.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.697+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a34108f10 con 0x7f2a3410de10 2024-09-21T02:11:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.698+0000 7f2a38939640 1 -- 172.21.15.92:0/1363121875 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f2a2c0b6ad0 con 0x7f2a3410de10 2024-09-21T02:11:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:11:50.699 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:11:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:11:50.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a1c0b6540 msgr2=0x7f2a1c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a1c0b6540 0x7f2a1c0b8a50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a3406bc60 tx=0x7f2a24007600 comp rx=0 tx=0).stop 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 msgr2=0x7f2a3419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 0x7f2a3419fd00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00d750 tx=0x7f2a2c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 shutdown_connections 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a1c0b6540 0x7f2a1c0b8a50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a3410de10 0x7f2a3419fd00 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a34105300 0x7f2a3419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 --2- 172.21.15.92:0/1363121875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a341049b0 0x7f2a3419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 >> 172.21.15.92:0/1363121875 conn(0x7f2a34079730 msgr2=0x7f2a34100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 shutdown_connections 2024-09-21T02:11:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:50.701+0000 7f2a3993b640 1 -- 172.21.15.92:0/1363121875 wait complete. 2024-09-21T02:11:51.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:50 smithi136 ceph-mon[34754]: pgmap v566: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:11:51.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:50 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1363121875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:11:51.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:50 smithi092 ceph-mon[31190]: pgmap v566: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:11:51.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:50 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1363121875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:11:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:50 smithi099 ceph-mon[34404]: pgmap v566: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:11:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:50 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1363121875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:11:51.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:11:51.668 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:52.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.189+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/1910323312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 msgr2=0x7fbbf0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.189+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/1910323312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf0110670 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009900 tx=0x7fbbe802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/1910323312 shutdown_connections 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/1910323312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf0110670 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/1910323312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbf01094f0 0x7fbbf0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/1910323312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf0103bb0 0x7fbbf0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/1910323312 >> 172.21.15.92:0/1910323312 conn(0x7fbbf00ff600 msgr2=0x7fbbf0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/1910323312 shutdown_connections 2024-09-21T02:11:52.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/1910323312 wait complete. 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.190+0000 7fbbf4ac9640 1 Processor -- start 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 -- start start 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf0103bb0 0x7fbbf019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbf01094f0 0x7fbbf019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0113b00 con 0x7fbbf0109e40 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0113980 con 0x7fbbf0103bb0 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbf4ac9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0113c80 con 0x7fbbf01094f0 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38122/0 (socket says 172.21.15.92:38122) 2024-09-21T02:11:52.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 -- 172.21.15.92:0/224175321 learned_addr learned my addr 172.21.15.92:0/224175321 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbf01094f0 0x7fbbf019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbee59c640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf0103bb0 0x7fbbf019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 -- 172.21.15.92:0/224175321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbf01094f0 msgr2=0x7fbbf019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbf01094f0 0x7fbbf019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 -- 172.21.15.92:0/224175321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf0103bb0 msgr2=0x7fbbf019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf0103bb0 0x7fbbf019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 -- 172.21.15.92:0/224175321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01a0320 con 0x7fbbf0109e40 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.191+0000 7fbbeed9d640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf019fcb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8004770 tx=0x7fbbe8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.192+0000 7fbbdf7fe640 1 -- 172.21.15.92:0/224175321 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe803f030 con 0x7fbbf0109e40 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.192+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf01a05b0 con 0x7fbbf0109e40 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.192+0000 7fbbdf7fe640 1 -- 172.21.15.92:0/224175321 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8003aa0 con 0x7fbbf0109e40 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.192+0000 7fbbdf7fe640 1 -- 172.21.15.92:0/224175321 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8035070 con 0x7fbbf0109e40 2024-09-21T02:11:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.192+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01a0a90 con 0x7fbbf0109e40 2024-09-21T02:11:52.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.194+0000 7fbbfb03d740 1 Processor -- start 2024-09-21T02:11:52.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.194+0000 7fbbdf7fe640 1 -- 172.21.15.92:0/224175321 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fbbe8042030 con 0x7fbbf0109e40 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 -- start start 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d54a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5577d541aea0 con 0x5577d50dc390 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5577d540a2d0 con 0x5577d50d41b0 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577d540e0e0 con 0x5577d50fd070 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38136/0 (socket says 172.21.15.92:38136) 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbedd9b640 1 -- 172.21.15.92:0/837698589 learned_addr learned my addr 172.21.15.92:0/837698589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbee59c640 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d54a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.195+0000 7fbbdf7fe640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbbd80b6540 0x7fbbd80b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbedd9b640 1 -- 172.21.15.92:0/837698589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 msgr2=0x5577d54a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d54a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbeed9d640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbbd80b6540 0x7fbbd80b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbedd9b640 1 -- 172.21.15.92:0/837698589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 msgr2=0x5577d54a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbedd9b640 1 -- 172.21.15.92:0/837698589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d5168650 con 0x5577d50dc390 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbee59c640 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbdf7fe640 1 -- 172.21.15.92:0/224175321 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbbe8107560 con 0x7fbbf0109e40 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54a4d40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400afa0 tx=0x7fbbe400a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558b80d81d4324c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbdcff9640 1 -- 172.21.15.92:0/837698589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4005220 con 0x5577d50dc390 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbdcff9640 1 -- 172.21.15.92:0/837698589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe4004c50 con 0x5577d50dc390 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbdcff9640 1 -- 172.21.15.92:0/837698589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400fab0 con 0x5577d50dc390 2024-09-21T02:11:52.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.196+0000 7fbbeed9d640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbbd80b6540 0x7fbbd80b8a50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80c9ca0 tx=0x7fbbe809aa00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 -- 172.21.15.92:0/837698589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 msgr2=0x5577d54a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54a4d40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400afa0 tx=0x7fbbe400a7b0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 -- 172.21.15.92:0/837698589 shutdown_connections 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54a4d40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/837698589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d54a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 -- 172.21.15.92:0/837698589 >> 172.21.15.92:0/837698589 conn(0x5577d50f1460 msgr2=0x5577d54a21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 -- 172.21.15.92:0/837698589 shutdown_connections 2024-09-21T02:11:52.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.197+0000 7fbbfb03d740 1 -- 172.21.15.92:0/837698589 wait complete. 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 -- start start 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 Processor -- start 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50d41b0 0x5577d54af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50dc390 0x5577d54afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5577d526b700 con 0x5577d50fd070 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5577d53a5160 con 0x5577d50dc390 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577d54b2aa0 con 0x5577d50d41b0 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbfb03d740 10 client.0 _ll_register_callbacks cb 0x5577d53be9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38142/0 (socket says 172.21.15.92:38142) 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 -- 172.21.15.92:0/1953329408 learned_addr learned my addr 172.21.15.92:0/1953329408 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50d41b0 0x5577d54af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50dc390 0x5577d54afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 -- 172.21.15.92:0/1953329408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50d41b0 msgr2=0x5577d54af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50d41b0 0x5577d54af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 -- 172.21.15.92:0/1953329408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50dc390 msgr2=0x5577d54afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50dc390 0x5577d54afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbedd9b640 1 -- 172.21.15.92:0/1953329408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d53d2c90 con 0x5577d50fd070 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.198+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50dc390 0x5577d54afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbdf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4005790 tx=0x7fbbe400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08d673aa4b62cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1953329408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4004e00 con 0x5577d50fd070 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe4004c50 con 0x5577d50fd070 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1953329408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbbe400f830 con 0x5577d50fd070 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1953329408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbe400fa30 con 0x5577d50fd070 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbfb03d740 10 client.17478 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577d54be840 con 0x5577d50fd070 2024-09-21T02:11:52.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1953329408 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe4004c50 con 0x5577d50fd070 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1953329408 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbe4027070 con 0x5577d50fd070 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbfb03d740 10 client.17478 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbfb03d740 10 client.17478 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.199+0000 7fbbfb03d740 20 client.17478 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.200+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d50bb9f0 0x5577d54c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.200+0000 7fbbfb03d740 4 client.17478 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.200+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577d541cdf0 con 0x5577d50bb9f0 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.200+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d50bb9f0 0x5577d54c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.200+0000 7fbbd5ffb640 10 client.17478 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.200+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d50bb9f0 0x5577d54c1c00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98ca9f4d5a191d2c server_cookie=9120b99db2731d3 in_seq=0 out_seq=0 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1953329408 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577d541cdf0 con 0x5577d50bb9f0 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbd5ffb640 10 client.17478 handle_command_reply: tid=0 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 client.17478 shutdown 2024-09-21T02:11:52.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 msgr2=0x5577d54bbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbdf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4005790 tx=0x7fbbe400f730 comp rx=0 tx=0).stop 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 shutdown_connections 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d50bb9f0 0x5577d54c1c00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbdf0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50dc390 0x5577d54afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1953329408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50d41b0 0x5577d54af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 >> 172.21.15.92:0/1953329408 conn(0x5577d50f1460 msgr2=0x5577d54a7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbd5ffb640 0 client.17478 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 shutdown_connections 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1953329408 wait complete. 2024-09-21T02:11:52.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.201+0000 7fbbfb03d740 20 client.17478 trim_cache size 0 max 16384 2024-09-21T02:11:52.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 Processor -- start 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 -- start start 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50dc390 0x5577d54b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ad200 con 0x5577d50fd070 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01aed40 con 0x5577d50d41b0 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.214+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ad8d0 con 0x5577d50dc390 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50dc390 0x5577d54b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38156/0 (socket says 172.21.15.92:38156) 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbedd9b640 1 -- 172.21.15.92:0/2472272483 learned_addr learned my addr 172.21.15.92:0/2472272483 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 -- 172.21.15.92:0/2472272483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 msgr2=0x5577d54a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 -- 172.21.15.92:0/2472272483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 msgr2=0x5577d54bbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 -- 172.21.15.92:0/2472272483 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d549d5d0 con 0x5577d50dc390 2024-09-21T02:11:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:52.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbee59c640 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50dc390 0x5577d54b15e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5577d53a5270 tx=0x7fbbe0015e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2351f2c824ed2d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbd7fff640 1 -- 172.21.15.92:0/2472272483 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0010b40 con 0x5577d50dc390 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbd7fff640 1 -- 172.21.15.92:0/2472272483 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0010ce0 con 0x5577d50dc390 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.215+0000 7fbbd7fff640 1 -- 172.21.15.92:0/2472272483 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0027730 con 0x5577d50dc390 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 -- 172.21.15.92:0/2472272483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50dc390 msgr2=0x5577d54b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50dc390 0x5577d54b15e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5577d53a5270 tx=0x7fbbe0015e90 comp rx=0 tx=0).stop 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 -- 172.21.15.92:0/2472272483 shutdown_connections 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50fd070 0x5577d54bbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50dc390 0x5577d54b15e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/2472272483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 -- 172.21.15.92:0/2472272483 >> 172.21.15.92:0/2472272483 conn(0x5577d50f1460 msgr2=0x5577d54b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 -- 172.21.15.92:0/2472272483 shutdown_connections 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.216+0000 7fbbfb03d740 1 -- 172.21.15.92:0/2472272483 wait complete. 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 -- start start 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 Processor -- start 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d552b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ae670 con 0x5577d50dc390 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01adfa0 con 0x5577d50d41b0 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ab1e0 con 0x5577d50fd070 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbfb03d740 10 client.0 _ll_register_callbacks cb 0x7fbbe0009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38160/0 (socket says 172.21.15.92:38160) 2024-09-21T02:11:52.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.217+0000 7fbbee59c640 1 -- 172.21.15.92:0/1057969206 learned_addr learned my addr 172.21.15.92:0/1057969206 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d552b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbee59c640 1 -- 172.21.15.92:0/1057969206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 msgr2=0x5577d552b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d552b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbee59c640 1 -- 172.21.15.92:0/1057969206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 msgr2=0x5577d54b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbee59c640 1 -- 172.21.15.92:0/1057969206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d552b9a0 con 0x5577d50dc390 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d552b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbee59c640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54b49a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0003a80 tx=0x7fbbe0010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80389c56885fc392 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1057969206 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0021420 con 0x5577d50dc390 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1057969206 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbbe00215c0 con 0x5577d50dc390 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe0010ce0 con 0x5577d50dc390 2024-09-21T02:11:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1057969206 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbe00217c0 con 0x5577d50dc390 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbfb03d740 10 client.17484 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577d552cc20 con 0x5577d50dc390 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.218+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1057969206 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe0010ce0 con 0x5577d50dc390 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1057969206 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbe000f040 con 0x5577d50dc390 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 10 client.17484 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 10 client.17484 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 20 client.17484 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d5090da0 0x5577d552f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 4 client.17484 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577d54188a0 con 0x5577d5090da0 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d5090da0 0x5577d552f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbd5ffb640 10 client.17484 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbedd9b640 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d5090da0 0x5577d552f300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=638bd646d513b23b server_cookie=e6c416400c14ed58 in_seq=0 out_seq=0 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbd5ffb640 1 -- 172.21.15.92:0/1057969206 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577d54188a0 con 0x5577d5090da0 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbd5ffb640 10 client.17484 handle_command_reply: tid=0 2024-09-21T02:11:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.219+0000 7fbbfb03d740 1 client.17484 shutdown 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 msgr2=0x5577d54b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54b49a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0003a80 tx=0x7fbbe0010eb0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 shutdown_connections 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5577d5090da0 0x5577d552f300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577d50fd070 0x5577d552b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577d50dc390 0x5577d54b49a0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 --2- 172.21.15.92:0/1057969206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5577d50d41b0 0x5577d54b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 >> 172.21.15.92:0/1057969206 conn(0x5577d50f1460 msgr2=0x5577d54a7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbd5ffb640 0 client.17484 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 shutdown_connections 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 1 -- 172.21.15.92:0/1057969206 wait complete. 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbfb03d740 20 client.17484 trim_cache size 0 max 16384 2024-09-21T02:11:52.225 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:52.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbbd80b6540 msgr2=0x7fbbd80b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbbd80b6540 0x7fbbd80b8a50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80c9ca0 tx=0x7fbbe809aa00 comp rx=0 tx=0).stop 2024-09-21T02:11:52.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 msgr2=0x7fbbf019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf019fcb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8004770 tx=0x7fbbe8004270 comp rx=0 tx=0).stop 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 shutdown_connections 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbbd80b6540 0x7fbbd80b8a50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbf0109e40 0x7fbbf019fcb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbf01094f0 0x7fbbf019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 --2- 172.21.15.92:0/224175321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf0103bb0 0x7fbbf019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.220+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 >> 172.21.15.92:0/224175321 conn(0x7fbbf00ff600 msgr2=0x7fbbf0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.221+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 shutdown_connections 2024-09-21T02:11:52.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:52.221+0000 7fbbf4ac9640 1 -- 172.21.15.92:0/224175321 wait complete. 2024-09-21T02:11:52.954 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:52 smithi092 ceph-mon[31190]: pgmap v567: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:11:52.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:11:53.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:52 smithi136 ceph-mon[34754]: pgmap v567: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:11:53.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:52 smithi099 ceph-mon[34404]: pgmap v567: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:11:53.225 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 -- 172.21.15.92:0/461207155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c109e40 msgr2=0x7f5a4c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 --2- 172.21.15.92:0/461207155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c109e40 0x7f5a4c110670 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009900 tx=0x7f5a4402a410 comp rx=0 tx=0).stop 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 -- 172.21.15.92:0/461207155 shutdown_connections 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 --2- 172.21.15.92:0/461207155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c109e40 0x7f5a4c110670 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 --2- 172.21.15.92:0/461207155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 --2- 172.21.15.92:0/461207155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a4c103bb0 0x7f5a4c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.730+0000 7f5a51225640 1 -- 172.21.15.92:0/461207155 >> 172.21.15.92:0/461207155 conn(0x7f5a4c0ff600 msgr2=0x7f5a4c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.731+0000 7f5a51225640 1 -- 172.21.15.92:0/461207155 shutdown_connections 2024-09-21T02:11:53.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.731+0000 7f5a51225640 1 -- 172.21.15.92:0/461207155 wait complete. 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.731+0000 7f5a51225640 1 Processor -- start 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 -- start start 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a4c103bb0 0x7f5a4c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a4c109e40 0x7f5a4c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c113b00 con 0x7f5a4c1094f0 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c113980 con 0x7f5a4c109e40 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a51225640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c113c80 con 0x7f5a4c103bb0 2024-09-21T02:11:53.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53198/0 (socket says 172.21.15.92:53198) 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 -- 172.21.15.92:0/2522011289 learned_addr learned my addr 172.21.15.92:0/2522011289 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a4c109e40 0x7f5a4c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a4c103bb0 0x7f5a4c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 -- 172.21.15.92:0/2522011289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a4c103bb0 msgr2=0x7f5a4c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a4c103bb0 0x7f5a4c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 -- 172.21.15.92:0/2522011289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a4c109e40 msgr2=0x7f5a4c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a4c109e40 0x7f5a4c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.732+0000 7f5a4a59c640 1 -- 172.21.15.92:0/2522011289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c1a0320 con 0x7f5a4c1094f0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.733+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c19aa10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c8c0 tx=0x7f5a4000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.733+0000 7f5a33fff640 1 -- 172.21.15.92:0/2522011289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40007bb0 con 0x7f5a4c1094f0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.733+0000 7f5a33fff640 1 -- 172.21.15.92:0/2522011289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a40007d50 con 0x7f5a4c1094f0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.733+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c1a0610 con 0x7f5a4c1094f0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.733+0000 7f5a33fff640 1 -- 172.21.15.92:0/2522011289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400187c0 con 0x7f5a4c1094f0 2024-09-21T02:11:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.733+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1a6e30 con 0x7f5a4c1094f0 2024-09-21T02:11:53.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.735+0000 7f5a57799740 1 Processor -- start 2024-09-21T02:11:53.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 -- start start 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 0x5621628216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621624511b0 0x562162821be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x562162822120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562162797ea0 con 0x56216247a070 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621627872d0 con 0x5621624511b0 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a57799740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56216278b0e0 con 0x562162459390 2024-09-21T02:11:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a33fff640 1 -- 172.21.15.92:0/2522011289 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5a4000f450 con 0x7f5a4c1094f0 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 0x5621628216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 0x5621628216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53200/0 (socket says 172.21.15.92:53200) 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 -- 172.21.15.92:0/2593402729 learned_addr learned my addr 172.21.15.92:0/2593402729 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621624511b0 0x562162821be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x562162822120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 -- 172.21.15.92:0/2593402729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 msgr2=0x562162822120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x562162822120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 -- 172.21.15.92:0/2593402729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621624511b0 msgr2=0x562162821be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621624511b0 0x562162821be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4b59e640 1 -- 172.21.15.92:0/2593402729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621624e5650 con 0x56216247a070 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.736+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x562162822120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 0x5621628216a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009900 tx=0x7f5a44032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e0636373bb67f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a317fa640 1 -- 172.21.15.92:0/2593402729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44040070 con 0x56216247a070 2024-09-21T02:11:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a317fa640 1 -- 172.21.15.92:0/2593402729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a440316a0 con 0x56216247a070 2024-09-21T02:11:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a317fa640 1 -- 172.21.15.92:0/2593402729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44035040 con 0x56216247a070 2024-09-21T02:11:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a33fff640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a340b6540 0x7f5a340b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a340b6540 0x7f5a340b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.737+0000 7f5a33fff640 1 -- 172.21.15.92:0/2522011289 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5a400e7340 con 0x7f5a4c1094f0 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 -- 172.21.15.92:0/2593402729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 msgr2=0x5621628216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a340b6540 0x7f5a340b8a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x562162820320 tx=0x7f5a3c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 0x5621628216a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009900 tx=0x7f5a44032e70 comp rx=0 tx=0).stop 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 -- 172.21.15.92:0/2593402729 shutdown_connections 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x562162822120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621624511b0 0x562162821be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 --2- 172.21.15.92:0/2593402729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56216247a070 0x5621628216a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 -- 172.21.15.92:0/2593402729 >> 172.21.15.92:0/2593402729 conn(0x56216246e460 msgr2=0x56216281f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 -- 172.21.15.92:0/2593402729 shutdown_connections 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 -- 172.21.15.92:0/2593402729 wait complete. 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 -- start start 2024-09-21T02:11:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.738+0000 7f5a57799740 1 Processor -- start 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x56216282d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56216247a070 0x56216282d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5621625e8700 con 0x5621624511b0 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562162722160 con 0x56216247a070 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56216282fe20 con 0x562162459390 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a57799740 10 client.0 _ll_register_callbacks cb 0x56216273b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53214/0 (socket says 172.21.15.92:53214) 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/2500471251 learned_addr learned my addr 172.21.15.92:0/2500471251 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x56216282d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56216247a070 0x56216282d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 msgr2=0x56216282d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x56216282d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/2500471251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56216247a070 msgr2=0x56216282d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56216247a070 0x56216282d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/2500471251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216274fc90 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282ccf0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c013c50 tx=0x7f5a3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b742f6ce3dce782 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.739+0000 7f5a2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a2eff5640 1 -- 172.21.15.92:0/2500471251 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c02b060 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a2eff5640 1 -- 172.21.15.92:0/2500471251 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5a3c013c80 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a440316a0 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a2eff5640 1 -- 172.21.15.92:0/2500471251 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c01c460 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 10 client.17496 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56216283feb0 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a2eff5640 1 -- 172.21.15.92:0/2500471251 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a440316a0 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a2eff5640 1 -- 172.21.15.92:0/2500471251 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a3c01a040 con 0x5621624511b0 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 10 client.17496 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:53.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 10 client.17496 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 20 client.17496 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5621624389f0 0x562162843410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 4 client.17496 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.740+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562162799df0 con 0x5621624389f0 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.741+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5621624389f0 0x562162843410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.741+0000 7f5a2eff5640 10 client.17496 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.741+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5621624389f0 0x562162843410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d633beff65b56090 server_cookie=6b46e705cab84e5f in_seq=0 out_seq=0 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a2eff5640 1 -- 172.21.15.92:0/2500471251 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562162799df0 con 0x5621624389f0 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a2eff5640 10 client.17496 handle_command_reply: tid=0 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 client.17496 shutdown 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 msgr2=0x56216282ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282ccf0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c013c50 tx=0x7f5a3c004a40 comp rx=0 tx=0).stop 2024-09-21T02:11:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 shutdown_connections 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5621624389f0 0x562162843410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56216247a070 0x56216282d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a2eff5640 0 client.17496 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562162459390 0x56216282d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 --2- 172.21.15.92:0/2500471251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282ccf0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 >> 172.21.15.92:0/2500471251 conn(0x56216246e460 msgr2=0x562162824880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 shutdown_connections 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 1 -- 172.21.15.92:0/2500471251 wait complete. 2024-09-21T02:11:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.742+0000 7f5a57799740 20 client.17496 trim_cache size 0 max 16384 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.755+0000 7f5a57799740 1 Processor -- start 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.755+0000 7f5a57799740 1 -- start start 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a57799740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624389f0 0x562162823090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a57799740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a57799740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x562162829ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a57799740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ad210 con 0x5621624511b0 2024-09-21T02:11:53.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a57799740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1aed50 con 0x562162459390 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a57799740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ad8e0 con 0x5621624389f0 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53226/0 (socket says 172.21.15.92:53226) 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 -- 172.21.15.92:0/4117140559 learned_addr learned my addr 172.21.15.92:0/4117140559 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x562162829ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624389f0 0x562162823090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 -- 172.21.15.92:0/4117140559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624389f0 msgr2=0x562162823090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624389f0 0x562162823090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 -- 172.21.15.92:0/4117140559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 msgr2=0x562162829ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x562162829ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 -- 172.21.15.92:0/4117140559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562162823f30 con 0x5621624511b0 2024-09-21T02:11:53.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x562162829ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:11:53.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.756+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282e920 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000cfd0 tx=0x7f5a4000fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7f3c9938cb244a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:53.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a30ff9640 1 -- 172.21.15.92:0/4117140559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400b1f90 con 0x5621624511b0 2024-09-21T02:11:53.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a30ff9640 1 -- 172.21.15.92:0/4117140559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a400b6ed0 con 0x5621624511b0 2024-09-21T02:11:53.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a30ff9640 1 -- 172.21.15.92:0/4117140559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400a7e10 con 0x5621624511b0 2024-09-21T02:11:53.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 -- 172.21.15.92:0/4117140559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 msgr2=0x56216282e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282e920 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000cfd0 tx=0x7f5a4000fdc0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 -- 172.21.15.92:0/4117140559 shutdown_connections 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x562162829ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624511b0 0x56216282e920 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 --2- 172.21.15.92:0/4117140559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624389f0 0x562162823090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 -- 172.21.15.92:0/4117140559 >> 172.21.15.92:0/4117140559 conn(0x56216246e460 msgr2=0x56216282e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 -- 172.21.15.92:0/4117140559 shutdown_connections 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.757+0000 7f5a57799740 1 -- 172.21.15.92:0/4117140559 wait complete. 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 -- start start 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 Processor -- start 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 0x5621628333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624511b0 0x562162833930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x5621628ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ae680 con 0x5621624389f0 2024-09-21T02:11:53.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1adfb0 con 0x562162459390 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ab1f0 con 0x5621624511b0 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a57799740 10 client.0 _ll_register_callbacks cb 0x7f5a3c0138d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a4ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 0x5621628333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a4ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 0x5621628333f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53234/0 (socket says 172.21.15.92:53234) 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/3574519908 learned_addr learned my addr 172.21.15.92:0/3574519908 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624511b0 0x562162833930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.758+0000 7f5a4b59e640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x5621628ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624511b0 msgr2=0x562162833930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624511b0 0x562162833930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/3574519908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 msgr2=0x5621628ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x5621628ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a4ad9d640 1 -- 172.21.15.92:0/3574519908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562162834200 con 0x5621624389f0 2024-09-21T02:11:53.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a4ad9d640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 0x5621628333f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f80 tx=0x7f5a3c01b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60efe23d461e484f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a2eff5640 1 -- 172.21.15.92:0/3574519908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c035030 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a2eff5640 1 -- 172.21.15.92:0/3574519908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5a3c01baa0 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a400b6ed0 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a2eff5640 1 -- 172.21.15.92:0/3574519908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c040070 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a57799740 10 client.17502 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621628ae3f0 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.759+0000 7f5a2eff5640 1 -- 172.21.15.92:0/3574519908 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a400b6ed0 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a2eff5640 1 -- 172.21.15.92:0/3574519908 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a3c01a040 con 0x5621624389f0 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 10 client.17502 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 10 client.17502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 20 client.17502 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56216247a070 0x5621628b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:53.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 4 client.17502 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621627958a0 con 0x56216247a070 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56216247a070 0x5621628b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a4a59c640 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56216247a070 0x5621628b0be0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e703f6884e509a69 server_cookie=3ad75fbbb83fa000 in_seq=0 out_seq=0 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a2eff5640 10 client.17502 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a2eff5640 1 -- 172.21.15.92:0/3574519908 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621627958a0 con 0x56216247a070 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a2eff5640 10 client.17502 handle_command_reply: tid=0 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.760+0000 7f5a57799740 1 client.17502 shutdown 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 msgr2=0x5621628333f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 0x5621628333f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000f80 tx=0x7f5a3c01b760 comp rx=0 tx=0).stop 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 shutdown_connections 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56216247a070 0x5621628b0be0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562162459390 0x5621628ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621624511b0 0x562162833930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 --2- 172.21.15.92:0/3574519908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621624389f0 0x5621628333f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a2eff5640 0 client.17502 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 >> 172.21.15.92:0/3574519908 conn(0x56216246e460 msgr2=0x562162822e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 shutdown_connections 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 1 -- 172.21.15.92:0/3574519908 wait complete. 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a57799740 20 client.17502 trim_cache size 0 max 16384 2024-09-21T02:11:53.767 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a340b6540 msgr2=0x7f5a340b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a340b6540 0x7f5a340b8a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x562162820320 tx=0x7f5a3c009680 comp rx=0 tx=0).stop 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 msgr2=0x7f5a4c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c19aa10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c8c0 tx=0x7f5a4000cd80 comp rx=0 tx=0).stop 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 shutdown_connections 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a340b6540 0x7f5a340b8a50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a4c109e40 0x7f5a4c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a4c1094f0 0x7f5a4c19aa10 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 --2- 172.21.15.92:0/2522011289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a4c103bb0 0x7f5a4c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.761+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 >> 172.21.15.92:0/2522011289 conn(0x7f5a4c0ff600 msgr2=0x7f5a4c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:53.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.762+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 shutdown_connections 2024-09-21T02:11:53.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:53.762+0000 7f5a51225640 1 -- 172.21.15.92:0/2522011289 wait complete. 2024-09-21T02:11:54.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:11:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:55.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:54 smithi136 ceph-mon[34754]: pgmap v568: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:11:55.036 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:54 smithi092 ceph-mon[31190]: pgmap v568: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:11:55.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:54 smithi099 ceph-mon[34404]: pgmap v568: 97 pgs: 97 active+clean; 1.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 -- 172.21.15.92:0/1729302368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 msgr2=0x7f78a410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/1729302368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a410d930 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7898009900 tx=0x7f789802a410 comp rx=0 tx=0).stop 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 -- 172.21.15.92:0/1729302368 shutdown_connections 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/1729302368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 0x7f78a4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/1729302368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a410d930 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/1729302368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a41048c0 0x7f78a4104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 -- 172.21.15.92:0/1729302368 >> 172.21.15.92:0/1729302368 conn(0x7f78a4079790 msgr2=0x7f78a4079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 -- 172.21.15.92:0/1729302368 shutdown_connections 2024-09-21T02:11:55.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.232+0000 7f78aa6e9640 1 -- 172.21.15.92:0/1729302368 wait complete. 2024-09-21T02:11:55.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 Processor -- start 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 -- start start 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a41048c0 0x7f78a419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 0x7f78a419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113ab0 con 0x7f78a41052a0 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113930 con 0x7f78a410de70 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78aa6e9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113c30 con 0x7f78a41048c0 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53260/0 (socket says 172.21.15.92:53260) 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.233+0000 7f78a37fe640 1 -- 172.21.15.92:0/3079627436 learned_addr learned my addr 172.21.15.92:0/3079627436 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a3fff640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a41048c0 0x7f78a419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a8c86640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 0x7f78a419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a37fe640 1 -- 172.21.15.92:0/3079627436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a41048c0 msgr2=0x7f78a419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a37fe640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a41048c0 0x7f78a419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a37fe640 1 -- 172.21.15.92:0/3079627436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 msgr2=0x7f78a419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a37fe640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 0x7f78a419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a37fe640 1 -- 172.21.15.92:0/3079627436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a41a0350 con 0x7f78a41052a0 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a8c86640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 0x7f78a419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a37fe640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a419aa40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f789802a930 tx=0x7f7898004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a17fa640 1 -- 172.21.15.92:0/3079627436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789803f030 con 0x7f78a41052a0 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a17fa640 1 -- 172.21.15.92:0/3079627436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898003aa0 con 0x7f78a41052a0 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a41a05e0 con 0x7f78a41052a0 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78a17fa640 1 -- 172.21.15.92:0/3079627436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898035070 con 0x7f78a41052a0 2024-09-21T02:11:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.234+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a0ac0 con 0x7f78a41052a0 2024-09-21T02:11:55.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.237+0000 7f78b0c5d740 1 Processor -- start 2024-09-21T02:11:55.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.237+0000 7f78b0c5d740 1 -- start start 2024-09-21T02:11:55.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 0x55f8767806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876780be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763b8390 0x55f876781120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8766f6ea0 con 0x55f8763d9070 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8766e62d0 con 0x55f8763b01b0 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8766ea0e0 con 0x55f8763b8390 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 0x55f8767806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a17fa640 1 -- 172.21.15.92:0/3079627436 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f7898042030 con 0x7f78a41052a0 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 0x55f8767806a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53274/0 (socket says 172.21.15.92:53274) 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 -- 172.21.15.92:0/640532378 learned_addr learned my addr 172.21.15.92:0/640532378 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a3fff640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876780be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a37fe640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763b8390 0x55f876781120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 -- 172.21.15.92:0/640532378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763b8390 msgr2=0x55f876781120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763b8390 0x55f876781120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 -- 172.21.15.92:0/640532378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 msgr2=0x55f876780be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876780be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 -- 172.21.15.92:0/640532378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f876444650 con 0x55f8763d9070 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a37fe640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763b8390 0x55f876781120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a3fff640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876780be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.238+0000 7f78a8c86640 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 0x55f8767806a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f789c007f40 tx=0x7f789c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb61c5a1127e969 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.239+0000 7f788a7fc640 1 -- 172.21.15.92:0/640532378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c014070 con 0x55f8763d9070 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.239+0000 7f788a7fc640 1 -- 172.21.15.92:0/640532378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c004d20 con 0x55f8763d9070 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.239+0000 7f788a7fc640 1 -- 172.21.15.92:0/640532378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c005520 con 0x55f8763d9070 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.239+0000 7f78a17fa640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f788c0b65c0 0x7f788c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.239+0000 7f78a3fff640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f788c0b65c0 0x7f788c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.239+0000 7f78a17fa640 1 -- 172.21.15.92:0/3079627436 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7898107aa0 con 0x7f78a41052a0 2024-09-21T02:11:55.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 -- 172.21.15.92:0/640532378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 msgr2=0x55f8767806a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78a3fff640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f788c0b65c0 0x7f788c0b8ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55f87677f320 tx=0x7f789400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 0x55f8767806a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f789c007f40 tx=0x7f789c00e7e0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 -- 172.21.15.92:0/640532378 shutdown_connections 2024-09-21T02:11:55.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763b8390 0x55f876781120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876780be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/640532378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763d9070 0x55f8767806a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 -- 172.21.15.92:0/640532378 >> 172.21.15.92:0/640532378 conn(0x55f8763cd460 msgr2=0x55f87677e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 -- 172.21.15.92:0/640532378 shutdown_connections 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 -- 172.21.15.92:0/640532378 wait complete. 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.240+0000 7f78b0c5d740 1 -- start start 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 Processor -- start 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f87678bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f8767981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f876547700 con 0x55f8763b8390 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f876681160 con 0x55f8763b01b0 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f87678ee80 con 0x55f8763d9070 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78b0c5d740 10 client.0 _ll_register_callbacks cb 0x55f87669a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:55.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53290/0 (socket says 172.21.15.92:53290) 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 -- 172.21.15.92:0/2837703976 learned_addr learned my addr 172.21.15.92:0/2837703976 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a8c86640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f8767981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f87678bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 -- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 msgr2=0x55f8767981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f8767981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 -- 172.21.15.92:0/2837703976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 msgr2=0x55f87678bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f87678bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a37fe640 1 -- 172.21.15.92:0/2837703976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8766aec90 con 0x55f8763b8390 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.241+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f87678bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678c320 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f78980ca180 tx=0x7f78980c9b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb840e5527856749 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78837fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78837fe640 1 -- 172.21.15.92:0/2837703976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898049070 con 0x55f8763b8390 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789c004d20 con 0x55f8763b8390 2024-09-21T02:11:55.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78837fe640 1 -- 172.21.15.92:0/2837703976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f78980c9e20 con 0x55f8763b8390 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78837fe640 1 -- 172.21.15.92:0/2837703976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78980c9320 con 0x55f8763b8390 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 10 client.17514 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f87679ab90 con 0x55f8763b8390 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78837fe640 1 -- 172.21.15.92:0/2837703976 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c004d20 con 0x55f8763b8390 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78837fe640 1 -- 172.21.15.92:0/2837703976 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f78980f1070 con 0x55f8763b8390 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 10 client.17514 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 20 client.17514 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8763979f0 0x55f87679e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 4 client.17514 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.242+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8766f8df0 con 0x55f8763979f0 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.243+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8763979f0 0x55f87679e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.243+0000 7f78837fe640 10 client.17514 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.243+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8763979f0 0x55f87679e150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca4d64a614e369fc server_cookie=46a42b0a6d7c1f98 in_seq=0 out_seq=0 2024-09-21T02:11:55.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78837fe640 1 -- 172.21.15.92:0/2837703976 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8766f8df0 con 0x55f8763979f0 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78837fe640 10 client.17514 handle_command_reply: tid=0 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 client.17514 shutdown 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 msgr2=0x55f87678c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678c320 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f78980ca180 tx=0x7f78980c9b10 comp rx=0 tx=0).stop 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 shutdown_connections 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f8763979f0 0x55f87679e150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f8767981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678c320 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2837703976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f87678bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 >> 172.21.15.92:0/2837703976 conn(0x55f8763cd460 msgr2=0x55f876783880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78837fe640 0 client.17514 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 shutdown_connections 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.244+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2837703976 wait complete. 2024-09-21T02:11:55.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.245+0000 7f78b0c5d740 20 client.17514 trim_cache size 0 max 16384 2024-09-21T02:11:55.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.257+0000 7f78b0c5d740 1 Processor -- start 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.257+0000 7f78b0c5d740 1 -- start start 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876782090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f87677dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41af280 con 0x55f8763b8390 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b0dc0 con 0x55f8763b01b0 2024-09-21T02:11:55.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41af950 con 0x55f8763d9070 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53292/0 (socket says 172.21.15.92:53292) 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 -- 172.21.15.92:0/2926532200 learned_addr learned my addr 172.21.15.92:0/2926532200 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a8c86640 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f87677dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876782090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 -- 172.21.15.92:0/2926532200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 msgr2=0x55f87677dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f87677dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 -- 172.21.15.92:0/2926532200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 msgr2=0x55f876782090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876782090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 -- 172.21.15.92:0/2926532200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87678d0a0 con 0x55f8763b8390 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678da10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f789803bfd0 tx=0x7f7898032ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2314c794c62355b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f7889ffb640 1 -- 172.21.15.92:0/2926532200 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898049070 con 0x55f8763b8390 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f7889ffb640 1 -- 172.21.15.92:0/2926532200 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78980cd6c0 con 0x55f8763b8390 2024-09-21T02:11:55.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.258+0000 7f7889ffb640 1 -- 172.21.15.92:0/2926532200 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980f11e0 con 0x55f8763b8390 2024-09-21T02:11:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2926532200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 msgr2=0x55f87678da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678da10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f789803bfd0 tx=0x7f7898032ab0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2926532200 shutdown_connections 2024-09-21T02:11:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f87677dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b8390 0x55f87678da10 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2926532200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b01b0 0x55f876782090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2926532200 >> 172.21.15.92:0/2926532200 conn(0x55f8763cd460 msgr2=0x55f87678d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2926532200 shutdown_connections 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2926532200 wait complete. 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.259+0000 7f78b0c5d740 1 -- start start 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 Processor -- start 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 0x55f876794aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b8390 0x55f87678ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f876790640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b06f0 con 0x55f8763b01b0 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41b0020 con 0x55f8763b8390 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41ad260 con 0x55f8763d9070 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 0x55f876794aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78b0c5d740 10 client.0 _ll_register_callbacks cb 0x7f7898005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 0x55f876794aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53308/0 (socket says 172.21.15.92:53308) 2024-09-21T02:11:55.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78a3fff640 1 -- 172.21.15.92:0/2923815078 learned_addr learned my addr 172.21.15.92:0/2923815078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78a8c86640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f876790640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.260+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b8390 0x55f87678ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a3fff640 1 -- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 msgr2=0x55f876790640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f876790640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a3fff640 1 -- 172.21.15.92:0/2923815078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b8390 msgr2=0x55f87678ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b8390 0x55f87678ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a3fff640 1 -- 172.21.15.92:0/2923815078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f876807b50 con 0x55f8763b01b0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b8390 0x55f87678ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78a3fff640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 0x55f876794aa0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7894005fd0 tx=0x7f78940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07e06ff02064b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78837fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78837fe640 1 -- 172.21.15.92:0/2923815078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789401ce20 con 0x55f8763b01b0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78837fe640 1 -- 172.21.15.92:0/2923815078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7894005020 con 0x55f8763b01b0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78980cd6c0 con 0x55f8763b01b0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78837fe640 1 -- 172.21.15.92:0/2923815078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7894005220 con 0x55f8763b01b0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78b0c5d740 10 client.17520 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f876808c80 con 0x55f8763b01b0 2024-09-21T02:11:55.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78837fe640 1 -- 172.21.15.92:0/2923815078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78980cd6c0 con 0x55f8763b01b0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.261+0000 7f78837fe640 1 -- 172.21.15.92:0/2923815078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7894019040 con 0x55f8763b01b0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 10 client.17520 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 10 client.17520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 20 client.17520 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f87636cda0 0x55f87680b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 4 client.17520 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8766f48a0 con 0x55f87636cda0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f87636cda0 0x55f87680b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78837fe640 10 client.17520 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78a37fe640 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f87636cda0 0x55f87680b450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9160c77e8c46171 server_cookie=1964e1fb76cd60e6 in_seq=0 out_seq=0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78837fe640 1 -- 172.21.15.92:0/2923815078 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8766f48a0 con 0x55f87636cda0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78837fe640 10 client.17520 handle_command_reply: tid=0 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.262+0000 7f78b0c5d740 1 client.17520 shutdown 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 msgr2=0x55f876794aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 0x55f876794aa0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7894005fd0 tx=0x7f78940046c0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 shutdown_connections 2024-09-21T02:11:55.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55f87636cda0 0x55f87680b450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8763d9070 0x55f876790640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8763b8390 0x55f87678ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78837fe640 0 client.17520 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 --2- 172.21.15.92:0/2923815078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8763b01b0 0x55f876794aa0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 >> 172.21.15.92:0/2923815078 conn(0x55f8763cd460 msgr2=0x55f876781e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 shutdown_connections 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 1 -- 172.21.15.92:0/2923815078 wait complete. 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78b0c5d740 20 client.17520 trim_cache size 0 max 16384 2024-09-21T02:11:55.276 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f788c0b65c0 msgr2=0x7f788c0b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f788c0b65c0 0x7f788c0b8ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55f87677f320 tx=0x7f789400b390 comp rx=0 tx=0).stop 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 msgr2=0x7f78a419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.263+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a419aa40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f789802a930 tx=0x7f7898004270 comp rx=0 tx=0).stop 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 shutdown_connections 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f788c0b65c0 0x7f788c0b8ad0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a410de70 0x7f78a419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a41052a0 0x7f78a419aa40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 --2- 172.21.15.92:0/3079627436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a41048c0 0x7f78a419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 >> 172.21.15.92:0/3079627436 conn(0x7f78a4079790 msgr2=0x7f78a4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 shutdown_connections 2024-09-21T02:11:55.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:55.264+0000 7f78aa6e9640 1 -- 172.21.15.92:0/3079627436 wait complete. 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 -- 172.21.15.92:0/291623574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50109540 msgr2=0x7fec5010b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 --2- 172.21.15.92:0/291623574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50109540 0x7fec5010b960 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fec40009ca0 tx=0x7fec4002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 -- 172.21.15.92:0/291623574 shutdown_connections 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 --2- 172.21.15.92:0/291623574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50109540 0x7fec5010b960 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 --2- 172.21.15.92:0/291623574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50106be0 0x7fec50109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 --2- 172.21.15.92:0/291623574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50101a90 0x7fec50101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 -- 172.21.15.92:0/291623574 >> 172.21.15.92:0/291623574 conn(0x7fec500fd690 msgr2=0x7fec500ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 -- 172.21.15.92:0/291623574 shutdown_connections 2024-09-21T02:11:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 -- 172.21.15.92:0/291623574 wait complete. 2024-09-21T02:11:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.856+0000 7fec555c6640 1 Processor -- start 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 -- start start 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50101a90 0x7fec50104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 0x7fec50102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50109540 0x7fec50103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec5010f2f0 con 0x7fec50106be0 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec5010f170 con 0x7fec50109540 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec555c6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec5010f470 con 0x7fec50101a90 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 0x7fec50102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 0x7fec50102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53326/0 (socket says 172.21.15.92:53326) 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 -- 172.21.15.92:0/125209425 learned_addr learned my addr 172.21.15.92:0/125209425 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50109540 0x7fec50103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4effd640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50101a90 0x7fec50104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 -- 172.21.15.92:0/125209425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50101a90 msgr2=0x7fec50104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50101a90 0x7fec50104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 -- 172.21.15.92:0/125209425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50109540 msgr2=0x7fec50103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50109540 0x7fec50103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 -- 172.21.15.92:0/125209425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec501036c0 con 0x7fec50106be0 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4effd640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50101a90 0x7fec50104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50109540 0x7fec50103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.857+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 0x7fec50102ae0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c920 tx=0x7fec3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.858+0000 7fec47fff640 1 -- 172.21.15.92:0/125209425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38007c70 con 0x7fec50106be0 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.858+0000 7fec47fff640 1 -- 172.21.15.92:0/125209425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38007e10 con 0x7fec50106be0 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.858+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec501a3f90 con 0x7fec50106be0 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.858+0000 7fec47fff640 1 -- 172.21.15.92:0/125209425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38018820 con 0x7fec50106be0 2024-09-21T02:11:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.858+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec501a4340 con 0x7fec50106be0 2024-09-21T02:11:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.860+0000 7fec5c61c740 1 Processor -- start 2024-09-21T02:11:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.860+0000 7fec5c61c740 1 -- start start 2024-09-21T02:11:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.860+0000 7fec47fff640 1 -- 172.21.15.92:0/125209425 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fec3800f450 con 0x7fec50106be0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583102f0 0x55d5586e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 0x55d5586e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec5c61c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d5586536f0 con 0x55d5583184d0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec5c61c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d558642b20 con 0x55d5583102f0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec5c61c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d558646930 con 0x55d5583391b0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 0x55d5586e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 0x55d5586e2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53328/0 (socket says 172.21.15.92:53328) 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 -- 172.21.15.92:0/163105612 learned_addr learned my addr 172.21.15.92:0/163105612 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4effd640 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583102f0 0x55d5586e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 -- 172.21.15.92:0/163105612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 msgr2=0x55d5586e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 -- 172.21.15.92:0/163105612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583102f0 msgr2=0x55d5586e14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583102f0 0x55d5586e14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 -- 172.21.15.92:0/163105612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d558608530 con 0x55d5583184d0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.861+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 0x55d5586e2010 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fec380ed6b0 tx=0x7fec380ed7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa9ecb4c9428c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.862+0000 7fec457fa640 1 -- 172.21.15.92:0/163105612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380fb070 con 0x55d5583184d0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.862+0000 7fec47fff640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec3c0b6510 0x7fec3c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.862+0000 7fec457fa640 1 -- 172.21.15.92:0/163105612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec380e7e00 con 0x55d5583184d0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.862+0000 7fec457fa640 1 -- 172.21.15.92:0/163105612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380f3920 con 0x55d5583184d0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.862+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec3c0b6510 0x7fec3c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.862+0000 7fec47fff640 1 -- 172.21.15.92:0/125209425 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fec380e7540 con 0x7fec50106be0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- 172.21.15.92:0/163105612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 msgr2=0x55d5586e2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec3c0b6510 0x7fec3c0b8a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec40005cc0 tx=0x7fec40031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 0x55d5586e2010 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fec380ed6b0 tx=0x7fec380ed7a0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- 172.21.15.92:0/163105612 shutdown_connections 2024-09-21T02:11:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583184d0 0x55d5586e2010 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583102f0 0x55d5586e14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- 172.21.15.92:0/163105612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- 172.21.15.92:0/163105612 >> 172.21.15.92:0/163105612 conn(0x55d55832d300 msgr2=0x55d5586dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- 172.21.15.92:0/163105612 shutdown_connections 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- 172.21.15.92:0/163105612 wait complete. 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- start start 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 Processor -- start 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586ea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583184d0 0x55d5586eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583391b0 0x55d5586eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d55857aab0 con 0x55d5583102f0 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5585e8730 con 0x55d5583391b0 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.863+0000 7fec5c61c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d5586e2b30 con 0x55d5583184d0 2024-09-21T02:11:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec5c61c740 10 client.0 _ll_register_callbacks cb 0x55d55863c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586ea630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53334/0 (socket says 172.21.15.92:53334) 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 -- 172.21.15.92:0/2975740335 learned_addr learned my addr 172.21.15.92:0/2975740335 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583184d0 0x55d5586eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583391b0 0x55d5586eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 -- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583184d0 msgr2=0x55d5586eacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583184d0 0x55d5586eacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 -- 172.21.15.92:0/2975740335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583391b0 msgr2=0x55d5586eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583391b0 0x55d5586eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 -- 172.21.15.92:0/2975740335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5583a4790 con 0x55d5583102f0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583391b0 0x55d5586eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec4effd640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586ea630 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fec48010a50 tx=0x7fec48010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f43928f9d8a7c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec367fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec367fc640 1 -- 172.21.15.92:0/2975740335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4801d060 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec367fc640 1 -- 172.21.15.92:0/2975740335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fec4800eb20 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec380e7e00 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec367fc640 1 -- 172.21.15.92:0/2975740335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec4800ed20 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec5c61c740 10 client.17532 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.864+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d55841d620 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec367fc640 1 -- 172.21.15.92:0/2975740335 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec380e7e00 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec367fc640 1 -- 172.21.15.92:0/2975740335 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fec48017420 con 0x55d5583102f0 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec5c61c740 10 client.17532 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec5c61c740 10 client.17532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec5c61c740 20 client.17532 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582f7d80 0x55d5586f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec5c61c740 4 client.17532 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d558655640 con 0x55d5582f7d80 2024-09-21T02:11:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.865+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582f7d80 0x55d5586f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582f7d80 0x55d5586f7ec0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f5ccf07cec5b49b server_cookie=ba4b18d1c76b6efc in_seq=0 out_seq=0 2024-09-21T02:11:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec367fc640 10 client.17532 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec367fc640 1 -- 172.21.15.92:0/2975740335 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d558655640 con 0x55d5582f7d80 2024-09-21T02:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec367fc640 10 client.17532 handle_command_reply: tid=0 2024-09-21T02:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 client.17532 shutdown 2024-09-21T02:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 msgr2=0x55d5586ea630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586ea630 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fec48010a50 tx=0x7fec48010f10 comp rx=0 tx=0).stop 2024-09-21T02:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 shutdown_connections 2024-09-21T02:11:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582f7d80 0x55d5586f7ec0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583391b0 0x55d5586eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583184d0 0x55d5586eacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2975740335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586ea630 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec367fc640 0 client.17532 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.866+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 >> 172.21.15.92:0/2975740335 conn(0x55d55832d300 msgr2=0x55d5586daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.867+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 shutdown_connections 2024-09-21T02:11:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.867+0000 7fec5c61c740 1 -- 172.21.15.92:0/2975740335 wait complete. 2024-09-21T02:11:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.867+0000 7fec5c61c740 20 client.17532 trim_cache size 0 max 16384 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.881+0000 7fec5c61c740 1 Processor -- start 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.881+0000 7fec5c61c740 1 -- start start 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec5c61c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec501ad1d0 con 0x55d5583102f0 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec5c61c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec501aed10 con 0x55d5583184d0 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec5c61c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec501ad8a0 con 0x55d5583391b0 2024-09-21T02:11:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53338/0 (socket says 172.21.15.92:53338) 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 -- 172.21.15.92:0/2759028085 learned_addr learned my addr 172.21.15.92:0/2759028085 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 -- 172.21.15.92:0/2759028085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 msgr2=0x55d5586db6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 -- 172.21.15.92:0/2759028085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 msgr2=0x55d5586f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 -- 172.21.15.92:0/2759028085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5586ebbd0 con 0x55d5583102f0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec4effd640 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586dc220 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec4800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1784a8afaee041f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.882+0000 7fec44ff9640 1 -- 172.21.15.92:0/2759028085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48030070 con 0x55d5583102f0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec44ff9640 1 -- 172.21.15.92:0/2759028085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec48017a60 con 0x55d5583102f0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec44ff9640 1 -- 172.21.15.92:0/2759028085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4802bb40 con 0x55d5583102f0 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 -- 172.21.15.92:0/2759028085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 msgr2=0x55d5586dc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586dc220 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec4800e830 comp rx=0 tx=0).stop 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 -- 172.21.15.92:0/2759028085 shutdown_connections 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583391b0 0x55d5586db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2759028085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583102f0 0x55d5586dc220 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 -- 172.21.15.92:0/2759028085 >> 172.21.15.92:0/2759028085 conn(0x55d55832d300 msgr2=0x55d5586daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 -- 172.21.15.92:0/2759028085 shutdown_connections 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.883+0000 7fec5c61c740 1 -- 172.21.15.92:0/2759028085 wait complete. 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 -- start start 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 Processor -- start 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583102f0 0x55d5586ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586efc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 0x55d5586f0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec501ae640 con 0x55d5583391b0 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec501adf70 con 0x55d5583184d0 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec501ab1b0 con 0x55d5583102f0 2024-09-21T02:11:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec5c61c740 10 client.0 _ll_register_callbacks cb 0x7fec4800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 0x55d5586f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 0x55d5586f0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53340/0 (socket says 172.21.15.92:53340) 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 -- 172.21.15.92:0/2290780821 learned_addr learned my addr 172.21.15.92:0/2290780821 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4e7fc640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4effd640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583102f0 0x55d5586ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 -- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583102f0 msgr2=0x55d5586ef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583102f0 0x55d5586ef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 -- 172.21.15.92:0/2290780821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 msgr2=0x55d5586efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.884+0000 7fec4f7fe640 1 -- 172.21.15.92:0/2290780821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5586f0ed0 con 0x55d5583391b0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec4effd640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583102f0 0x55d5586ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec4f7fe640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 0x55d5586f0770 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fec40005e30 tx=0x7fec40003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8444550136b44e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec367fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec367fc640 1 -- 172.21.15.92:0/2290780821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40057060 con 0x55d5583391b0 2024-09-21T02:11:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec367fc640 1 -- 172.21.15.92:0/2290780821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fec40042030 con 0x55d5583391b0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec48017a60 con 0x55d5583391b0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec367fc640 1 -- 172.21.15.92:0/2290780821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec40004310 con 0x55d5583391b0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 10 client.17538 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5586f2260 con 0x55d5583391b0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec367fc640 1 -- 172.21.15.92:0/2290780821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec48017a60 con 0x55d5583391b0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec367fc640 1 -- 172.21.15.92:0/2290780821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fec400305c0 con 0x55d5583391b0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 10 client.17538 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 20 client.17538 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582cd1c0 0x55d5587648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 4 client.17538 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.885+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5586510f0 con 0x55d5582cd1c0 2024-09-21T02:11:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.886+0000 7fec4effd640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582cd1c0 0x55d5587648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.886+0000 7fec367fc640 10 client.17538 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.886+0000 7fec4effd640 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582cd1c0 0x55d5587648c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702fcb35e2cf1aee server_cookie=ac157af0bc2158b9 in_seq=0 out_seq=0 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec367fc640 1 -- 172.21.15.92:0/2290780821 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5586510f0 con 0x55d5582cd1c0 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec367fc640 10 client.17538 handle_command_reply: tid=0 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 client.17538 shutdown 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 msgr2=0x55d5586f0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 0x55d5586f0770 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fec40005e30 tx=0x7fec40003a60 comp rx=0 tx=0).stop 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 shutdown_connections 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d5582cd1c0 0x55d5587648c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d5583391b0 0x55d5586f0770 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec367fc640 0 client.17538 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5583184d0 0x55d5586efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 --2- 172.21.15.92:0/2290780821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5583102f0 0x55d5586ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 >> 172.21.15.92:0/2290780821 conn(0x55d55832d300 msgr2=0x55d5586e4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 shutdown_connections 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 1 -- 172.21.15.92:0/2290780821 wait complete. 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec5c61c740 20 client.17538 trim_cache size 0 max 16384 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:11:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec3c0b6510 msgr2=0x7fec3c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec555c6640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec3c0b6510 0x7fec3c0b8a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec40005cc0 tx=0x7fec40031af0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 msgr2=0x7fec50102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.901+0000 7fec555c6640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 0x7fec50102ae0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c920 tx=0x7fec3800cde0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 shutdown_connections 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fec3c0b6510 0x7fec3c0b8a20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec50109540 0x7fec50103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec50106be0 0x7fec50102ae0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 --2- 172.21.15.92:0/125209425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec50101a90 0x7fec50104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 >> 172.21.15.92:0/125209425 conn(0x7fec500fd690 msgr2=0x7fec50107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 shutdown_connections 2024-09-21T02:11:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:11:55.902+0000 7fec555c6640 1 -- 172.21.15.92:0/125209425 wait complete. 2024-09-21T02:11:56.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:11:56.318 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 -- 172.21.15.92:0/3641739588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc1083a0 msgr2=0x7f03fc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 --2- 172.21.15.92:0/3641739588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc1083a0 0x7f03fc1087b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009900 tx=0x7f03f002a410 comp rx=0 tx=0).stop 2024-09-21T02:11:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 -- 172.21.15.92:0/3641739588 shutdown_connections 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 --2- 172.21.15.92:0/3641739588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc108cf0 0x7f03fc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 --2- 172.21.15.92:0/3641739588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc1083a0 0x7f03fc1087b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 --2- 172.21.15.92:0/3641739588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03fc102a60 0x7f03fc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 -- 172.21.15.92:0/3641739588 >> 172.21.15.92:0/3641739588 conn(0x7f03fc079730 msgr2=0x7f03fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 -- 172.21.15.92:0/3641739588 shutdown_connections 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.792+0000 7f04046be640 1 -- 172.21.15.92:0/3641739588 wait complete. 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.793+0000 7f04046be640 1 Processor -- start 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.793+0000 7f04046be640 1 -- start start 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.793+0000 7f04046be640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 0x7f03fc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.793+0000 7f04046be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc1083a0 0x7f03fc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f04046be640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03fc108cf0 0x7f03fc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f04046be640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1129b0 con 0x7f03fc102a60 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f04046be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc112830 con 0x7f03fc1083a0 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f04046be640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc112b30 con 0x7f03fc108cf0 2024-09-21T02:11:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 0x7f03fc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 0x7f03fc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53354/0 (socket says 172.21.15.92:53354) 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 -- 172.21.15.92:0/1464117336 learned_addr learned my addr 172.21.15.92:0/1464117336 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f0402c5b640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03fc108cf0 0x7f03fc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f0401c59640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc1083a0 0x7f03fc1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 -- 172.21.15.92:0/1464117336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03fc108cf0 msgr2=0x7f03fc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03fc108cf0 0x7f03fc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 -- 172.21.15.92:0/1464117336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc1083a0 msgr2=0x7f03fc1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc1083a0 0x7f03fc1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 -- 172.21.15.92:0/1464117336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc19f2b0 con 0x7f03fc102a60 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f040245a640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 0x7f03fc199340 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0078f0 tx=0x7f03ec007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f03fb7fe640 1 -- 172.21.15.92:0/1464117336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec019020 con 0x7f03fc102a60 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f03fb7fe640 1 -- 172.21.15.92:0/1464117336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03ec0054d0 con 0x7f03fc102a60 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f03fb7fe640 1 -- 172.21.15.92:0/1464117336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00e650 con 0x7f03fc102a60 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc19f5a0 con 0x7f03fc102a60 2024-09-21T02:11:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.794+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc06ae60 con 0x7f03fc102a60 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 Processor -- start 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 -- start start 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f03fb7fe640 1 -- 172.21.15.92:0/1464117336 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f03ec005020 con 0x7f03fc102a60 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f4f572cea0 con 0x55f4f540f070 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f4f571c2d0 con 0x55f4f53e61b0 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f040ac32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f4f57200e0 con 0x55f4f53ee390 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.797+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53366/0 (socket says 172.21.15.92:53366) 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 -- 172.21.15.92:0/825348145 learned_addr learned my addr 172.21.15.92:0/825348145 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0401c59640 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f040245a640 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 -- 172.21.15.92:0/825348145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 msgr2=0x55f4f57b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 -- 172.21.15.92:0/825348145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 msgr2=0x55f4f57b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 -- 172.21.15.92:0/825348145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f547a650 con 0x55f4f540f070 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f0402c5b640 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57b66a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009ee0 tx=0x7f03f4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff8e56f3b5ad397 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f03f8ff9640 1 -- 172.21.15.92:0/825348145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4016070 con 0x55f4f540f070 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f03f8ff9640 1 -- 172.21.15.92:0/825348145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f4004d20 con 0x55f4f540f070 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f03f8ff9640 1 -- 172.21.15.92:0/825348145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4012720 con 0x55f4f540f070 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.798+0000 7f03fb7fe640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f03e40b6590 0x7f03e40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040245a640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f03e40b6590 0x7f03e40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f03fb7fe640 1 -- 172.21.15.92:0/1464117336 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f03ec0e8a40 con 0x7f03fc102a60 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 -- 172.21.15.92:0/825348145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 msgr2=0x55f4f57b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040245a640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f03e40b6590 0x7f03e40b8aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f4f57b5320 tx=0x7f03ec004a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57b66a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009ee0 tx=0x7f03f4010740 comp rx=0 tx=0).stop 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 -- 172.21.15.92:0/825348145 shutdown_connections 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 --2- 172.21.15.92:0/825348145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57b66a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.799+0000 7f040ac32740 1 -- 172.21.15.92:0/825348145 >> 172.21.15.92:0/825348145 conn(0x55f4f5403460 msgr2=0x55f4f57b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 -- 172.21.15.92:0/825348145 shutdown_connections 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 -- 172.21.15.92:0/825348145 wait complete. 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 -- start start 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 Processor -- start 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f4f557d700 con 0x55f4f540f070 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f4f56b7160 con 0x55f4f53e61b0 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f4f57c4e80 con 0x55f4f53ee390 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f040ac32740 10 client.0 _ll_register_callbacks cb 0x55f4f56d09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.800+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53382/0 (socket says 172.21.15.92:53382) 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 -- 172.21.15.92:0/580234369 learned_addr learned my addr 172.21.15.92:0/580234369 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0401c59640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 -- 172.21.15.92:0/580234369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 msgr2=0x55f4f57c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 -- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 msgr2=0x55f4f57c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 -- 172.21.15.92:0/580234369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f56e4c90 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f0402c5b640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57ce1a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x55f4f57b8b40 tx=0x7f03f4005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f5a35969985fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f03e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f03e1ffb640 1 -- 172.21.15.92:0/580234369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4016030 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f03e1ffb640 1 -- 172.21.15.92:0/580234369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f03f4012d50 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f4004d20 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f03e1ffb640 1 -- 172.21.15.92:0/580234369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03f4004b80 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f040ac32740 10 client.17550 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.801+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4f57d0b60 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f03e1ffb640 1 -- 172.21.15.92:0/580234369 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f4004d20 con 0x55f4f540f070 2024-09-21T02:11:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f03e1ffb640 1 -- 172.21.15.92:0/580234369 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f03f400f040 con 0x55f4f540f070 2024-09-21T02:11:56.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f040ac32740 10 client.17550 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:56.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f040ac32740 10 client.17550 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:11:56.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f040ac32740 20 client.17550 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:56.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f040ac32740 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f53cd9f0 0x55f4f57d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f040ac32740 4 client.17550 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:56.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f572edf0 con 0x55f4f53cd9f0 2024-09-21T02:11:56.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.802+0000 7f0401c59640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f53cd9f0 0x55f4f57d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.803+0000 7f03e1ffb640 10 client.17550 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:56.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.803+0000 7f0401c59640 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f53cd9f0 0x55f4f57d3ec0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b5188b9de8a7a81 server_cookie=dc7290b65930f9c in_seq=0 out_seq=0 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.803+0000 7f03e1ffb640 1 -- 172.21.15.92:0/580234369 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4f572edf0 con 0x55f4f53cd9f0 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.803+0000 7f03e1ffb640 10 client.17550 handle_command_reply: tid=0 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.803+0000 7f040ac32740 1 client.17550 shutdown 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 msgr2=0x55f4f57ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57ce1a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x55f4f57b8b40 tx=0x7f03f4005a80 comp rx=0 tx=0).stop 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 shutdown_connections 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f53cd9f0 0x55f4f57d3ec0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f540f070 0x55f4f57ce1a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53ee390 0x55f4f57c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f03e1ffb640 0 client.17550 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 --2- 172.21.15.92:0/580234369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 >> 172.21.15.92:0/580234369 conn(0x55f4f5403460 msgr2=0x55f4f57b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:56.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 shutdown_connections 2024-09-21T02:11:56.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 1 -- 172.21.15.92:0/580234369 wait complete. 2024-09-21T02:11:56.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.804+0000 7f040ac32740 20 client.17550 trim_cache size 0 max 16384 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 Processor -- start 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 -- start start 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53cd9f0 0x55f4f57b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53e61b0 0x55f4f57c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ad2a0 con 0x55f4f53ee390 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1aede0 con 0x55f4f53cd9f0 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f040ac32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ad970 con 0x55f4f53e61b0 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53390/0 (socket says 172.21.15.92:53390) 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f0402c5b640 1 -- 172.21.15.92:0/2249788800 learned_addr learned my addr 172.21.15.92:0/2249788800 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53cd9f0 0x55f4f57b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.817+0000 7f0401c59640 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53e61b0 0x55f4f57c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f0402c5b640 1 -- 172.21.15.92:0/2249788800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53e61b0 msgr2=0x55f4f57c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53e61b0 0x55f4f57c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f0402c5b640 1 -- 172.21.15.92:0/2249788800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53cd9f0 msgr2=0x55f4f57b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53cd9f0 0x55f4f57b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f0402c5b640 1 -- 172.21.15.92:0/2249788800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f57b8be0 con 0x55f4f53ee390 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57ce110 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009700 tx=0x7f03f4012470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f824ddea983af5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f03e3fff640 1 -- 172.21.15.92:0/2249788800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4016050 con 0x55f4f53ee390 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f03e3fff640 1 -- 172.21.15.92:0/2249788800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f4024040 con 0x55f4f53ee390 2024-09-21T02:11:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.818+0000 7f03e3fff640 1 -- 172.21.15.92:0/2249788800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4012b50 con 0x55f4f53ee390 2024-09-21T02:11:56.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 -- 172.21.15.92:0/2249788800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 msgr2=0x55f4f57ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57ce110 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009700 tx=0x7f03f4012470 comp rx=0 tx=0).stop 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 -- 172.21.15.92:0/2249788800 shutdown_connections 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57ce110 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53e61b0 0x55f4f57c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 --2- 172.21.15.92:0/2249788800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53cd9f0 0x55f4f57b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 -- 172.21.15.92:0/2249788800 >> 172.21.15.92:0/2249788800 conn(0x55f4f5403460 msgr2=0x55f4f57bb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 -- 172.21.15.92:0/2249788800 shutdown_connections 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 -- 172.21.15.92:0/2249788800 wait complete. 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 -- start start 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.819+0000 7f040ac32740 1 Processor -- start 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53cd9f0 0x55f4f57d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ae710 con 0x55f4f53ee390 2024-09-21T02:11:56.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ae040 con 0x55f4f53e61b0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ab280 con 0x55f4f53cd9f0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 10 client.0 _ll_register_callbacks cb 0x7f03f4009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57d5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53396/0 (socket says 172.21.15.92:53396) 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 -- 172.21.15.92:0/2452766724 learned_addr learned my addr 172.21.15.92:0/2452766724 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0401c59640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53cd9f0 0x55f4f57d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 -- 172.21.15.92:0/2452766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53cd9f0 msgr2=0x55f4f57d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53cd9f0 0x55f4f57d7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 -- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 msgr2=0x55f4f57d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 -- 172.21.15.92:0/2452766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f57d5730 con 0x55f4f53ee390 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f0402c5b640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57d5070 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f03f4012910 tx=0x7f03f40021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc11de478214eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f03e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f03e1ffb640 1 -- 172.21.15.92:0/2452766724 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4016070 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f03e1ffb640 1 -- 172.21.15.92:0/2452766724 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f03f4004d20 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.820+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f4024040 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f03e1ffb640 1 -- 172.21.15.92:0/2452766724 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03f4009890 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 10 client.17556 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4f583e4b0 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f03e1ffb640 1 -- 172.21.15.92:0/2452766724 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f4024040 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f03e1ffb640 1 -- 172.21.15.92:0/2452766724 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f03f4029420 con 0x55f4f53ee390 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 10 client.17556 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 10 client.17556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 20 client.17556 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f540f070 0x55f4f5840b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 4 client.17556 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4f572a8a0 con 0x55f4f540f070 2024-09-21T02:11:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.821+0000 7f0401c59640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f540f070 0x55f4f5840b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f03e1ffb640 10 client.17556 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f0401c59640 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f540f070 0x55f4f5840b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2597174a2e98433f server_cookie=df62ff45bb4211dd in_seq=0 out_seq=0 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f03e1ffb640 1 -- 172.21.15.92:0/2452766724 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4f572a8a0 con 0x55f4f540f070 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f03e1ffb640 10 client.17556 handle_command_reply: tid=0 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 client.17556 shutdown 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 msgr2=0x55f4f57d5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57d5070 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f03f4012910 tx=0x7f03f40021f0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 shutdown_connections 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55f4f540f070 0x55f4f5840b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f4f53ee390 0x55f4f57d5070 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f53e61b0 0x55f4f57d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 --2- 172.21.15.92:0/2452766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f4f53cd9f0 0x55f4f57d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 >> 172.21.15.92:0/2452766724 conn(0x55f4f5403460 msgr2=0x55f4f57c3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f03e1ffb640 0 client.17556 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:11:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 shutdown_connections 2024-09-21T02:11:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.822+0000 7f040ac32740 1 -- 172.21.15.92:0/2452766724 wait complete. 2024-09-21T02:11:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f040ac32740 20 client.17556 trim_cache size 0 max 16384 2024-09-21T02:11:56.828 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f03e40b6590 msgr2=0x7f03e40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f03e40b6590 0x7f03e40b8aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f4f57b5320 tx=0x7f03ec004a30 comp rx=0 tx=0).stop 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 msgr2=0x7f03fc199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 0x7f03fc199340 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0078f0 tx=0x7f03ec007db0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 shutdown_connections 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f03e40b6590 0x7f03e40b8aa0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03fc108cf0 0x7f03fc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc1083a0 0x7f03fc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 --2- 172.21.15.92:0/1464117336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03fc102a60 0x7f03fc199340 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 >> 172.21.15.92:0/1464117336 conn(0x7f03fc079730 msgr2=0x7f03fc1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 shutdown_connections 2024-09-21T02:11:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:56.823+0000 7f04046be640 1 -- 172.21.15.92:0/1464117336 wait complete. 2024-09-21T02:11:57.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:56 smithi136 ceph-mon[34754]: pgmap v569: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:11:57.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:56 smithi092 ceph-mon[31190]: pgmap v569: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:11:57.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:56 smithi099 ceph-mon[34404]: pgmap v569: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:11:57.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:11:57.768 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 -- 172.21.15.92:0/3626591948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 msgr2=0x7f21e8109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 --2- 172.21.15.92:0/3626591948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e8109970 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009900 tx=0x7f21dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 -- 172.21.15.92:0/3626591948 shutdown_connections 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 --2- 172.21.15.92:0/3626591948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e8109eb0 0x7f21e81106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 --2- 172.21.15.92:0/3626591948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e8109970 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 --2- 172.21.15.92:0/3626591948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e8102080 0x7f21e8102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 -- 172.21.15.92:0/3626591948 >> 172.21.15.92:0/3626591948 conn(0x7f21e8079ec0 msgr2=0x7f21e807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 -- 172.21.15.92:0/3626591948 shutdown_connections 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.298+0000 7f21edb21640 1 -- 172.21.15.92:0/3626591948 wait complete. 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 Processor -- start 2024-09-21T02:11:58.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 -- start start 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e8102080 0x7f21e819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e8109eb0 0x7f21e819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21e8113bb0 con 0x7f21e8109560 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e8113a30 con 0x7f21e8102080 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21edb21640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21e8113d30 con 0x7f21e8109eb0 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.299+0000 7f21e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53410/0 (socket says 172.21.15.92:53410) 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 -- 172.21.15.92:0/153839770 learned_addr learned my addr 172.21.15.92:0/153839770 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e7fff640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e8109eb0 0x7f21e819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e77fe640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e8102080 0x7f21e819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 -- 172.21.15.92:0/153839770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e8109eb0 msgr2=0x7f21e819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e8109eb0 0x7f21e819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 -- 172.21.15.92:0/153839770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e8102080 msgr2=0x7f21e819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e8102080 0x7f21e819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 -- 172.21.15.92:0/153839770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e81a0590 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e819abc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f21dc02a930 tx=0x7f21dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e4ff9640 1 -- 172.21.15.92:0/153839770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc03f030 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e4ff9640 1 -- 172.21.15.92:0/153839770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc003cb0 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e81a0820 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21e4ff9640 1 -- 172.21.15.92:0/153839770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc035070 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.300+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e81a6fe0 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 Processor -- start 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21e4ff9640 1 -- 172.21.15.92:0/153839770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f21dc042030 con 0x7f21e8109560 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 -- start start 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d8083662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808366800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f9e390 0x55d808366d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8082dcea0 con 0x55d807fbf070 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d8082cc2d0 con 0x55d807f9e390 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21f4095740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d8082d00e0 con 0x55d807f961b0 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d8083662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d8083662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53412/0 (socket says 172.21.15.92:53412) 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21e7fff640 1 -- 172.21.15.92:0/1783487724 learned_addr learned my addr 172.21.15.92:0/1783487724 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.303+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f9e390 0x55d808366d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e77fe640 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808366800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e7fff640 1 -- 172.21.15.92:0/1783487724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 msgr2=0x55d808366800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e7fff640 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808366800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e7fff640 1 -- 172.21.15.92:0/1783487724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f9e390 msgr2=0x55d808366d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e7fff640 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f9e390 0x55d808366d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e7fff640 1 -- 172.21.15.92:0/1783487724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d80802a650 con 0x55d807fbf070 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e77fe640 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808366800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:11:58.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e4ff9640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21d00b6590 0x7f21d00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e7fff640 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d8083662c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f21e000ab80 tx=0x7f21e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c20db7d9209846 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e77fe640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21d00b6590 0x7f21d00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21e4ff9640 1 -- 172.21.15.92:0/153839770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f21dc1078e0 con 0x7f21e8109560 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21cdffb640 1 -- 172.21.15.92:0/1783487724 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e001d070 con 0x55d807fbf070 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21cdffb640 1 -- 172.21.15.92:0/1783487724 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e0018af0 con 0x55d807fbf070 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.304+0000 7f21cdffb640 1 -- 172.21.15.92:0/1783487724 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0010a10 con 0x55d807fbf070 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 -- 172.21.15.92:0/1783487724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 msgr2=0x55d8083662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21e77fe640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21d00b6590 0x7f21d00b8aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55d808364f40 tx=0x7f21d80095b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d8083662c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f21e000ab80 tx=0x7f21e000f790 comp rx=0 tx=0).stop 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 -- 172.21.15.92:0/1783487724 shutdown_connections 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f9e390 0x55d808366d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808366800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 --2- 172.21.15.92:0/1783487724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d8083662c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 -- 172.21.15.92:0/1783487724 >> 172.21.15.92:0/1783487724 conn(0x55d807fb3460 msgr2=0x55d8083641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 -- 172.21.15.92:0/1783487724 shutdown_connections 2024-09-21T02:11:58.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 -- 172.21.15.92:0/1783487724 wait complete. 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.305+0000 7f21f4095740 1 -- start start 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 Processor -- start 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f961b0 0x55d808371a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f9e390 0x55d808371f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d80837ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d80812d700 con 0x55d807fbf070 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d808267160 con 0x55d807f961b0 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d808374aa0 con 0x55d807f9e390 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21f4095740 10 client.0 _ll_register_callbacks cb 0x55d8082809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d80837ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d80837ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53420/0 (socket says 172.21.15.92:53420) 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 -- 172.21.15.92:0/3892734703 learned_addr learned my addr 172.21.15.92:0/3892734703 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f961b0 0x55d808371a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f9e390 0x55d808371f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 -- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f9e390 msgr2=0x55d808371f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f9e390 0x55d808371f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 -- 172.21.15.92:0/3892734703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f961b0 msgr2=0x55d808371a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f961b0 0x55d808371a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.306+0000 7f21e7fff640 1 -- 172.21.15.92:0/3892734703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d808294c90 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21e7fff640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d80837ddf0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f21e000fc90 tx=0x7f21e0018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76245508be3a830 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3892734703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0027020 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3892734703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f21e0005040 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e0018af0 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3892734703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21e0032050 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21f4095740 10 client.17568 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d8083807e0 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3892734703 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e0018af0 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.307+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3892734703 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21e00046c0 con 0x55d807fbf070 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21f4095740 10 client.17568 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:58.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21f4095740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21f4095740 20 client.17568 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21f4095740 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f7d9f0 0x55d808383bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21f4095740 4 client.17568 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8082dedf0 con 0x55d807f7d9f0 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f7d9f0 0x55d808383bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f7d9f0 0x55d808383bf0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847e1de255c19cc6 server_cookie=23a00a8e31aa5ee3 in_seq=0 out_seq=0 2024-09-21T02:11:58.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.308+0000 7f21c6ffd640 10 client.17568 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3892734703 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8082dedf0 con 0x55d807f7d9f0 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21c6ffd640 10 client.17568 handle_command_reply: tid=0 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 client.17568 shutdown 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 msgr2=0x55d80837ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d80837ddf0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f21e000fc90 tx=0x7f21e0018670 comp rx=0 tx=0).stop 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 shutdown_connections 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f7d9f0 0x55d808383bf0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807fbf070 0x55d80837ddf0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f9e390 0x55d808371f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21c6ffd640 0 client.17568 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:58.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 --2- 172.21.15.92:0/3892734703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807f961b0 0x55d808371a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 >> 172.21.15.92:0/3892734703 conn(0x55d807fb3460 msgr2=0x55d8083694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 shutdown_connections 2024-09-21T02:11:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.309+0000 7f21f4095740 1 -- 172.21.15.92:0/3892734703 wait complete. 2024-09-21T02:11:58.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.310+0000 7f21f4095740 20 client.17568 trim_cache size 0 max 16384 2024-09-21T02:11:58.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.322+0000 7f21f4095740 1 Processor -- start 2024-09-21T02:11:58.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.322+0000 7f21f4095740 1 -- start start 2024-09-21T02:11:58.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21f4095740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808369ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21f4095740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d80837dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21f4095740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21f4095740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81af480 con 0x55d807f9e390 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21f4095740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81b0fc0 con 0x55d807fbf070 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21f4095740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81afb50 con 0x55d807f961b0 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d80837dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d80837dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53432/0 (socket says 172.21.15.92:53432) 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3078746677 learned_addr learned my addr 172.21.15.92:0/3078746677 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808369ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e7fff640 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3078746677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 msgr2=0x55d808369ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808369ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3078746677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 msgr2=0x55d8083717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3078746677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d808364aa0 con 0x55d807f9e390 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e7fff640 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:58.325 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d80837dd60 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cd750 tx=0x7f21dc03b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f153e0e3b0c242 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:58.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.323+0000 7f21cd7fa640 1 -- 172.21.15.92:0/3078746677 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc049070 con 0x55d807f9e390 2024-09-21T02:11:58.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21cd7fa640 1 -- 172.21.15.92:0/3078746677 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc0cd560 con 0x55d807f9e390 2024-09-21T02:11:58.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21cd7fa640 1 -- 172.21.15.92:0/3078746677 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0d2530 con 0x55d807f9e390 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 -- 172.21.15.92:0/3078746677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 msgr2=0x55d80837dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d80837dd60 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cd750 tx=0x7f21dc03b6b0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 -- 172.21.15.92:0/3078746677 shutdown_connections 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d80837dd60 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 --2- 172.21.15.92:0/3078746677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808369ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 -- 172.21.15.92:0/3078746677 >> 172.21.15.92:0/3078746677 conn(0x55d807fb3460 msgr2=0x55d8083693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:58.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 -- 172.21.15.92:0/3078746677 shutdown_connections 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.324+0000 7f21f4095740 1 -- 172.21.15.92:0/3078746677 wait complete. 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 -- start start 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 Processor -- start 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808387540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d808384a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81b08f0 con 0x55d807f9e390 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81b0220 con 0x55d807fbf070 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81ad460 con 0x55d807f961b0 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21f4095740 10 client.0 _ll_register_callbacks cb 0x7f21e001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d808384a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21e6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d808384a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53446/0 (socket says 172.21.15.92:53446) 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.325+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3360075177 learned_addr learned my addr 172.21.15.92:0/3360075177 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808387540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 msgr2=0x55d808387540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808387540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e7fff640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3360075177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 msgr2=0x55d8083850b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083850b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e6ffd640 1 -- 172.21.15.92:0/3360075177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d808385820 con 0x55d807f9e390 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808387540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21e6ffd640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d808384a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21dc105730 tx=0x7f21dc0ca1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d311ace50e989230 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3360075177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc049050 con 0x55d807f9e390 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3360075177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f21dc03bbe0 con 0x55d807f9e390 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc0cd560 con 0x55d807f9e390 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3360075177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21dc0cd1c0 con 0x55d807f9e390 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21f4095740 10 client.17574 fetch_fsmap learned FSMap version 29 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d8083ee3f0 con 0x55d807f9e390 2024-09-21T02:11:58.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.326+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3360075177 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc0cd560 con 0x55d807f9e390 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3360075177 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21dc0d2100 con 0x55d807f9e390 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21f4095740 10 client.17574 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21f4095740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21f4095740 20 client.17574 populate_metadata read hostname 'smithi092' 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21f4095740 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f52da0 0x55d8083f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21f4095740 4 client.17574 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8082da8a0 con 0x55d807f52da0 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f52da0 0x55d8083f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21c6ffd640 10 client.17574 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.327+0000 7f21e77fe640 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f52da0 0x55d8083f0ad0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c399d6ee46f744 server_cookie=40e52e0a88cff7b3 in_seq=0 out_seq=0 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21c6ffd640 1 -- 172.21.15.92:0/3360075177 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8082da8a0 con 0x55d807f52da0 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21c6ffd640 10 client.17574 handle_command_reply: tid=0 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 client.17574 shutdown 2024-09-21T02:11:58.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 msgr2=0x55d808384a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d808384a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21dc105730 tx=0x7f21dc0ca1a0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 shutdown_connections 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d807f52da0 0x55d8083f0ad0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d807fbf070 0x55d8083850b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d807f9e390 0x55d808384a10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21c6ffd640 0 client.17574 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 --2- 172.21.15.92:0/3360075177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d807f961b0 0x55d808387540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 >> 172.21.15.92:0/3360075177 conn(0x55d807fb3460 msgr2=0x55d8083717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 shutdown_connections 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 1 -- 172.21.15.92:0/3360075177 wait complete. 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.328+0000 7f21f4095740 20 client.17574 trim_cache size 0 max 16384 2024-09-21T02:11:58.333 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:11:58.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21d00b6590 msgr2=0x7f21d00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21d00b6590 0x7f21d00b8aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55d808364f40 tx=0x7f21d80095b0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 msgr2=0x7f21e819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e819abc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f21dc02a930 tx=0x7f21dc004270 comp rx=0 tx=0).stop 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 shutdown_connections 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f21d00b6590 0x7f21d00b8aa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e8109eb0 0x7f21e819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21e8109560 0x7f21e819abc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 --2- 172.21.15.92:0/153839770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e8102080 0x7f21e819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 >> 172.21.15.92:0/153839770 conn(0x7f21e8079ec0 msgr2=0x7f21e8107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 shutdown_connections 2024-09-21T02:11:58.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:11:58.329+0000 7f21edb21640 1 -- 172.21.15.92:0/153839770 wait complete. 2024-09-21T02:11:59.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:11:58 smithi136 ceph-mon[34754]: pgmap v570: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:11:59.046 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:11:58 smithi092 ceph-mon[31190]: pgmap v570: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:11:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:11:58 smithi099 ceph-mon[34404]: pgmap v570: 97 pgs: 97 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:12:00.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:12:00.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:12:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 -- 172.21.15.92:0/989232185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 msgr2=0x7ff74410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 --2- 172.21.15.92:0/989232185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74410ecb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009ca0 tx=0x7ff73c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 -- 172.21.15.92:0/989232185 shutdown_connections 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 --2- 172.21.15.92:0/989232185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74410ecb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 --2- 172.21.15.92:0/989232185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff744107b30 0x7ff744107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 --2- 172.21.15.92:0/989232185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7441021f0 0x7ff744102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 -- 172.21.15.92:0/989232185 >> 172.21.15.92:0/989232185 conn(0x7ff744077c60 msgr2=0x7ff7440780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 -- 172.21.15.92:0/989232185 shutdown_connections 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 -- 172.21.15.92:0/989232185 wait complete. 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.860+0000 7ff749a02640 1 Processor -- start 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 -- start start 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7441021f0 0x7ff74419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff744107b30 0x7ff74419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff744113ad0 con 0x7ff744108480 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff744113950 con 0x7ff7441021f0 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff749a02640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff744113c50 con 0x7ff744107b30 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53456/0 (socket says 172.21.15.92:53456) 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 -- 172.21.15.92:0/1217083711 learned_addr learned my addr 172.21.15.92:0/1217083711 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff744107b30 0x7ff74419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7441021f0 0x7ff74419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 -- 172.21.15.92:0/1217083711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff744107b30 msgr2=0x7ff74419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff744107b30 0x7ff74419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 -- 172.21.15.92:0/1217083711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7441021f0 msgr2=0x7ff74419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7441021f0 0x7ff74419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.861+0000 7ff7437fe640 1 -- 172.21.15.92:0/1217083711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7441a0410 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.862+0000 7ff7437fe640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74419fce0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff73c005cc0 tx=0x7ff73c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.862+0000 7ff748a00640 1 -- 172.21.15.92:0/1217083711 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c003bf0 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.862+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7441a06a0 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.862+0000 7ff748a00640 1 -- 172.21.15.92:0/1217083711 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73c003d90 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.862+0000 7ff748a00640 1 -- 172.21.15.92:0/1217083711 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c02fbc0 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.862+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441a6e60 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.864+0000 7ff74ff76740 1 Processor -- start 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.864+0000 7ff748a00640 1 -- 172.21.15.92:0/1217083711 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff73c02fd60 con 0x7ff744108480 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 -- start start 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945869350 0x555945c10430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945848670 0x555945c10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555945b86e60 con 0x555945840490 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555945b76290 con 0x555945869350 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff74ff76740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555945b7a0a0 con 0x555945848670 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c10970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53462/0 (socket says 172.21.15.92:53462) 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 -- 172.21.15.92:0/1916075513 learned_addr learned my addr 172.21.15.92:0/1916075513 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945848670 0x555945c10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945869350 0x555945c10430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 -- 172.21.15.92:0/1916075513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945848670 msgr2=0x555945c10eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945848670 0x555945c10eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 -- 172.21.15.92:0/1916075513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945869350 msgr2=0x555945c10430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945869350 0x555945c10430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 -- 172.21.15.92:0/1916075513 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559458d4930 con 0x555945840490 2024-09-21T02:12:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945869350 0x555945c10430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.865+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c10970 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff73400bd10 tx=0x7ff73400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f145f9f88d2f632 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.866+0000 7ff748a00640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7300b6510 0x7ff7300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.866+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7300b6510 0x7ff7300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.866+0000 7ff748a00640 1 -- 172.21.15.92:0/1217083711 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff73c108770 con 0x7ff744108480 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.866+0000 7ff739ffb640 1 -- 172.21.15.92:0/1916075513 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734015070 con 0x555945840490 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.866+0000 7ff739ffb640 1 -- 172.21.15.92:0/1916075513 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73401a910 con 0x555945840490 2024-09-21T02:12:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.866+0000 7ff739ffb640 1 -- 172.21.15.92:0/1916075513 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734011980 con 0x555945840490 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 -- 172.21.15.92:0/1916075513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 msgr2=0x555945c10970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7300b6510 0x7ff7300b8a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff744107890 tx=0x7ff72c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c10970 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff73400bd10 tx=0x7ff73400f730 comp rx=0 tx=0).stop 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 -- 172.21.15.92:0/1916075513 shutdown_connections 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945848670 0x555945c10eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c10970 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1916075513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945869350 0x555945c10430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 -- 172.21.15.92:0/1916075513 >> 172.21.15.92:0/1916075513 conn(0x55594585d740 msgr2=0x555945c0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:00.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 -- 172.21.15.92:0/1916075513 shutdown_connections 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 -- 172.21.15.92:0/1916075513 wait complete. 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.867+0000 7ff74ff76740 1 -- start start 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 Processor -- start 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945840490 0x555945c1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 0x555945c1c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c27f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5559459d7890 con 0x555945848670 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555945b10d40 con 0x555945840490 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555945c1ec10 con 0x555945869350 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff74ff76740 10 client.0 _ll_register_callbacks cb 0x555945b74c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 0x555945c1c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 0x555945c1c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53468/0 (socket says 172.21.15.92:53468) 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 -- 172.21.15.92:0/2278783637 learned_addr learned my addr 172.21.15.92:0/2278783637 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff742ffd640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945840490 0x555945c1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff7437fe640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c27f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 -- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 msgr2=0x555945c27f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c27f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 -- 172.21.15.92:0/2278783637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945840490 msgr2=0x555945c1c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945840490 0x555945c1c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.868+0000 7ff73bfff640 1 -- 172.21.15.92:0/2278783637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555945c28540 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff73bfff640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 0x555945c1c600 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00fb80 tx=0x7ff72c00c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdc087b45f42207 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff722ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff722ffd640 1 -- 172.21.15.92:0/2278783637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c011df0 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff722ffd640 1 -- 172.21.15.92:0/2278783637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff72c01f410 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff73401a910 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff722ffd640 1 -- 172.21.15.92:0/2278783637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff72c02d070 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 10 client.17586 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555945c2ae40 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff722ffd640 1 -- 172.21.15.92:0/2278783637 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73401a910 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff722ffd640 1 -- 172.21.15.92:0/2278783637 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff72c027730 con 0x555945848670 2024-09-21T02:12:00.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 10 client.17586 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 20 client.17586 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555945827cd0 0x555945c2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 4 client.17586 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.869+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555945b88db0 con 0x555945827cd0 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.870+0000 7ff742ffd640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555945827cd0 0x555945c2e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.870+0000 7ff742ffd640 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555945827cd0 0x555945c2e390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376b5cba757cf98e server_cookie=492cf069a604cca7 in_seq=0 out_seq=0 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.870+0000 7ff722ffd640 10 client.17586 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff722ffd640 1 -- 172.21.15.92:0/2278783637 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555945b88db0 con 0x555945827cd0 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff722ffd640 10 client.17586 handle_command_reply: tid=0 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 client.17586 shutdown 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 msgr2=0x555945c1c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 0x555945c1c600 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00fb80 tx=0x7ff72c00c3b0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 shutdown_connections 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555945827cd0 0x555945c2e390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c27f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945848670 0x555945c1c600 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 --2- 172.21.15.92:0/2278783637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945840490 0x555945c1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 >> 172.21.15.92:0/2278783637 conn(0x55594585d740 msgr2=0x555945c135c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff722ffd640 0 client.17586 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 shutdown_connections 2024-09-21T02:12:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 1 -- 172.21.15.92:0/2278783637 wait complete. 2024-09-21T02:12:00.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.871+0000 7ff74ff76740 20 client.17586 trim_cache size 0 max 16384 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.888+0000 7ff74ff76740 1 Processor -- start 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.888+0000 7ff74ff76740 1 -- start start 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c15690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c15bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff74ff76740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441af4f0 con 0x555945840490 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff74ff76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b1030 con 0x555945848670 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff74ff76740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441afbc0 con 0x555945869350 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c27e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53472/0 (socket says 172.21.15.92:53472) 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 -- 172.21.15.92:0/1887661697 learned_addr learned my addr 172.21.15.92:0/1887661697 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff7437fe640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c15bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c15690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 -- 172.21.15.92:0/1887661697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 msgr2=0x555945c15bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c15bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 -- 172.21.15.92:0/1887661697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 msgr2=0x555945c15690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c15690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 -- 172.21.15.92:0/1887661697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555945c28370 con 0x555945840490 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff73bfff640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c15690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff7437fe640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.889+0000 7ff742ffd640 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c27e30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff7340115f0 tx=0x7ff734011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735f91834118191a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff7397fa640 1 -- 172.21.15.92:0/1887661697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734031070 con 0x555945840490 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff7397fa640 1 -- 172.21.15.92:0/1887661697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff734004240 con 0x555945840490 2024-09-21T02:12:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff7397fa640 1 -- 172.21.15.92:0/1887661697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734015040 con 0x555945840490 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 -- 172.21.15.92:0/1887661697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 msgr2=0x555945c27e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c27e30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff7340115f0 tx=0x7ff734011620 comp rx=0 tx=0).stop 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 -- 172.21.15.92:0/1887661697 shutdown_connections 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c15690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 --2- 172.21.15.92:0/1887661697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c27e30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.890+0000 7ff74ff76740 1 -- 172.21.15.92:0/1887661697 >> 172.21.15.92:0/1887661697 conn(0x55594585d740 msgr2=0x555945c1a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.891+0000 7ff74ff76740 1 -- 172.21.15.92:0/1887661697 shutdown_connections 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.891+0000 7ff74ff76740 1 -- 172.21.15.92:0/1887661697 wait complete. 2024-09-21T02:12:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.891+0000 7ff74ff76740 1 -- start start 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.891+0000 7ff74ff76740 1 Processor -- start 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.891+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c32d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c33910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b0960 con 0x555945840490 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b0290 con 0x555945848670 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441ad4d0 con 0x555945869350 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c32d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 10 client.0 _ll_register_callbacks cb 0x7ff72c00c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c32d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53482/0 (socket says 172.21.15.92:53482) 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 -- 172.21.15.92:0/383183415 learned_addr learned my addr 172.21.15.92:0/383183415 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff73bfff640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff7437fe640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c33910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 -- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 msgr2=0x555945c33910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c33910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 -- 172.21.15.92:0/383183415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 msgr2=0x555945c332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 -- 172.21.15.92:0/383183415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555945c34080 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff742ffd640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c32d80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff73402f040 tx=0x7ff734005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efce22ffcd9abf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff722ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff722ffd640 1 -- 172.21.15.92:0/383183415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734039030 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff722ffd640 1 -- 172.21.15.92:0/383183415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff734011d90 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.892+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff734004240 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff722ffd640 1 -- 172.21.15.92:0/383183415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff734031040 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 10 client.17592 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555945c9d580 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff722ffd640 1 -- 172.21.15.92:0/383183415 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff734004240 con 0x555945840490 2024-09-21T02:12:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff722ffd640 1 -- 172.21.15.92:0/383183415 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff734034070 con 0x555945840490 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 10 client.17592 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 20 client.17592 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5559457fd080 0x555945c9fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 4 client.17592 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555945b84860 con 0x5559457fd080 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.893+0000 7ff73bfff640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5559457fd080 0x555945c9fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.894+0000 7ff722ffd640 10 client.17592 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.894+0000 7ff73bfff640 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5559457fd080 0x555945c9fa90 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fbe1f22f6e44b54 server_cookie=5cc33a7db75ec82c in_seq=0 out_seq=0 2024-09-21T02:12:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.895+0000 7ff722ffd640 1 -- 172.21.15.92:0/383183415 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555945b84860 con 0x5559457fd080 2024-09-21T02:12:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.895+0000 7ff722ffd640 10 client.17592 handle_command_reply: tid=0 2024-09-21T02:12:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.895+0000 7ff74ff76740 1 client.17592 shutdown 2024-09-21T02:12:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 msgr2=0x555945c32d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c32d80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff73402f040 tx=0x7ff734005850 comp rx=0 tx=0).stop 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 shutdown_connections 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5559457fd080 0x555945c9fa90 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555945869350 0x555945c33910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555945848670 0x555945c332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 --2- 172.21.15.92:0/383183415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555945840490 0x555945c32d80 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 >> 172.21.15.92:0/383183415 conn(0x55594585d740 msgr2=0x555945c13530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff722ffd640 0 client.17592 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 shutdown_connections 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 1 -- 172.21.15.92:0/383183415 wait complete. 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff74ff76740 20 client.17592 trim_cache size 0 max 16384 2024-09-21T02:12:00.900 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:12:00.901 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:12:00.901 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "0b7521ce-80b1-4138-a4f9-de46c10a80b9", 2024-09-21T02:12:00.901 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:12:00.901 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:12:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7300b6510 msgr2=0x7ff7300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff749a02640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7300b6510 0x7ff7300b8a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff744107890 tx=0x7ff72c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 msgr2=0x7ff74419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.896+0000 7ff749a02640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74419fce0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff73c005cc0 tx=0x7ff73c02f380 comp rx=0 tx=0).stop 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 shutdown_connections 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7300b6510 0x7ff7300b8a20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff744108480 0x7ff74419fce0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff744107b30 0x7ff74419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 --2- 172.21.15.92:0/1217083711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7441021f0 0x7ff74419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 >> 172.21.15.92:0/1217083711 conn(0x7ff744077c60 msgr2=0x7ff744105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 shutdown_connections 2024-09-21T02:12:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:00.897+0000 7ff749a02640 1 -- 172.21.15.92:0/1217083711 wait complete. 2024-09-21T02:12:01.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:00 smithi092 ceph-mon[31190]: pgmap v571: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:12:01.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:00 smithi099 ceph-mon[34404]: pgmap v571: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:12:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:00 smithi136 ceph-mon[34754]: pgmap v571: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:12:01.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:12:01.838 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:01 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:12:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:01 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:01 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:12:02.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:01 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:01 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:12:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:01 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:01 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:12:02.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:01 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:01 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:12:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:01 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:01 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:12:02.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:01 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:02.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 -- 172.21.15.92:0/3089070859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109570 msgr2=0x7f78a8109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 --2- 172.21.15.92:0/3089070859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109570 0x7f78a8109980 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f789c009900 tx=0x7f789c02a410 comp rx=0 tx=0).stop 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 -- 172.21.15.92:0/3089070859 shutdown_connections 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 --2- 172.21.15.92:0/3089070859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109ec0 0x7f78a81106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 --2- 172.21.15.92:0/3089070859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109570 0x7f78a8109980 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 --2- 172.21.15.92:0/3089070859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8103c30 0x7f78a8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 -- 172.21.15.92:0/3089070859 >> 172.21.15.92:0/3089070859 conn(0x7f78a807b8d0 msgr2=0x7f78a807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 -- 172.21.15.92:0/3089070859 shutdown_connections 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.333+0000 7f78afe3f640 1 -- 172.21.15.92:0/3089070859 wait complete. 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 Processor -- start 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 -- start start 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8103c30 0x7f78a819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109570 0x7f78a819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 0x7f78a81a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8113ae0 con 0x7f78a8109ec0 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8113960 con 0x7f78a8109570 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78afe3f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8113c60 con 0x7f78a8103c30 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78ae3dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 0x7f78a81a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78ae3dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 0x7f78a81a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53498/0 (socket says 172.21.15.92:53498) 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78ae3dc640 1 -- 172.21.15.92:0/304792005 learned_addr learned my addr 172.21.15.92:0/304792005 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78adbdb640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8103c30 0x7f78a819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78ad3da640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109570 0x7f78a819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.334+0000 7f78ae3dc640 1 -- 172.21.15.92:0/304792005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8103c30 msgr2=0x7f78a819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78ae3dc640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8103c30 0x7f78a819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78ae3dc640 1 -- 172.21.15.92:0/304792005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109570 msgr2=0x7f78a819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78ae3dc640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109570 0x7f78a819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78ae3dc640 1 -- 172.21.15.92:0/304792005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a81a2720 con 0x7f78a8109ec0 2024-09-21T02:12:02.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78ad3da640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109570 0x7f78a819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78ae3dc640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 0x7f78a81a1ff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f789800d750 tx=0x7f789800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78a6ffd640 1 -- 172.21.15.92:0/304792005 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898017070 con 0x7f78a8109ec0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a81a2a10 con 0x7f78a8109ec0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78a6ffd640 1 -- 172.21.15.92:0/304792005 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78980054d0 con 0x7f78a8109ec0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78a6ffd640 1 -- 172.21.15.92:0/304792005 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898004c60 con 0x7f78a8109ec0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.335+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a81a9170 con 0x7f78a8109ec0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.337+0000 7f78b63b3740 1 Processor -- start 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78a6ffd640 1 -- 172.21.15.92:0/304792005 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f7898010460 con 0x7f78a8109ec0 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 -- start start 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b98b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b98bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b98c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556b6b901e50 con 0x556b6b5e4070 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b6b8f1280 con 0x556b6b5c3390 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78b63b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b6b8f5090 con 0x556b6b5bb1b0 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b98b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b98b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53506/0 (socket says 172.21.15.92:53506) 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.338+0000 7f78ad3da640 1 -- 172.21.15.92:0/2021709404 learned_addr learned my addr 172.21.15.92:0/2021709404 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78adbdb640 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b98bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78a6ffd640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f78900b6510 0x7f78900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ad3da640 1 -- 172.21.15.92:0/2021709404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 msgr2=0x556b6b98bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b98bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ad3da640 1 -- 172.21.15.92:0/2021709404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 msgr2=0x556b6b98c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b98c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ad3da640 1 -- 172.21.15.92:0/2021709404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6b64f650 con 0x556b6b5e4070 2024-09-21T02:12:02.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78a6ffd640 1 -- 172.21.15.92:0/304792005 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f78980e6d00 con 0x7f78a8109ec0 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ae3dc640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f78900b6510 0x7f78900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b98b650 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f789c000c00 tx=0x7f789c005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ad791c8120442d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f788ffff640 1 -- 172.21.15.92:0/2021709404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c035050 con 0x556b6b5e4070 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f788ffff640 1 -- 172.21.15.92:0/2021709404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c0316a0 con 0x556b6b5e4070 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.339+0000 7f788ffff640 1 -- 172.21.15.92:0/2021709404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0046c0 con 0x556b6b5e4070 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 -- 172.21.15.92:0/2021709404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 msgr2=0x556b6b98b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b98b650 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f789c000c00 tx=0x7f789c005ea0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78ae3dc640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f78900b6510 0x7f78900b8a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f789800d720 tx=0x7f7898094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 -- 172.21.15.92:0/2021709404 shutdown_connections 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b98c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b98bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2021709404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b98b650 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 -- 172.21.15.92:0/2021709404 >> 172.21.15.92:0/2021709404 conn(0x556b6b5d8460 msgr2=0x556b6b989540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:02.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 -- 172.21.15.92:0/2021709404 shutdown_connections 2024-09-21T02:12:02.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.340+0000 7f78b63b3740 1 -- 172.21.15.92:0/2021709404 wait complete. 2024-09-21T02:12:02.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 -- start start 2024-09-21T02:12:02.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 Processor -- start 2024-09-21T02:12:02.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5bb1b0 0x556b6b996d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b997250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556b6b752700 con 0x556b6b5e4070 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b6b88c110 con 0x556b6b5bb1b0 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b6b999dd0 con 0x556b6b5c3390 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.341+0000 7f78b63b3740 10 client.0 _ll_register_callbacks cb 0x556b6b8a5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53512/0 (socket says 172.21.15.92:53512) 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 -- 172.21.15.92:0/1766691239 learned_addr learned my addr 172.21.15.92:0/1766691239 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78adbdb640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b997250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5bb1b0 0x556b6b996d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 -- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 msgr2=0x556b6b997250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b997250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 -- 172.21.15.92:0/1766691239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5bb1b0 msgr2=0x556b6b996d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5bb1b0 0x556b6b996d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 -- 172.21.15.92:0/1766691239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6b8b9c40 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78ad3da640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9a3120 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x556b6b97b6f0 tx=0x7f789c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733a5a76ab2a8831 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f788d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f788d7fa640 1 -- 172.21.15.92:0/1766691239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c035070 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f788d7fa640 1 -- 172.21.15.92:0/1766691239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f789c031b10 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789c0316a0 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f788d7fa640 1 -- 172.21.15.92:0/1766691239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f789c058050 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78b63b3740 10 client.17604 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.342+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b6b9a5ae0 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f788d7fa640 1 -- 172.21.15.92:0/1766691239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c0316a0 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f788d7fa640 1 -- 172.21.15.92:0/1766691239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f789c004b10 con 0x556b6b5e4070 2024-09-21T02:12:02.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78b63b3740 10 client.17604 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78b63b3740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78b63b3740 20 client.17604 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78b63b3740 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b5a29f0 0x556b6b9a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78b63b3740 4 client.17604 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6b903da0 con 0x556b6b5a29f0 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.343+0000 7f78adbdb640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b5a29f0 0x556b6b9a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.344+0000 7f78adbdb640 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b5a29f0 0x556b6b9a8e40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee191297a7ce5e2 server_cookie=24621ec0ba1a0dc3 in_seq=0 out_seq=0 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.344+0000 7f788d7fa640 10 client.17604 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.344+0000 7f788d7fa640 1 -- 172.21.15.92:0/1766691239 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b6b903da0 con 0x556b6b5a29f0 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.344+0000 7f788d7fa640 10 client.17604 handle_command_reply: tid=0 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.344+0000 7f78b63b3740 1 client.17604 shutdown 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 msgr2=0x556b6b9a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9a3120 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x556b6b97b6f0 tx=0x7f789c02a410 comp rx=0 tx=0).stop 2024-09-21T02:12:02.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 shutdown_connections 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b5a29f0 0x556b6b9a8e40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9a3120 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b997250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f788d7fa640 0 client.17604 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 --2- 172.21.15.92:0/1766691239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5bb1b0 0x556b6b996d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 >> 172.21.15.92:0/1766691239 conn(0x556b6b5d8460 msgr2=0x556b6b98e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 shutdown_connections 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 1 -- 172.21.15.92:0/1766691239 wait complete. 2024-09-21T02:12:02.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.345+0000 7f78b63b3740 20 client.17604 trim_cache size 0 max 16384 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 Processor -- start 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 -- start start 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 0x556b6b98d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b998940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5e4070 0x556b6b988b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81b1590 con 0x556b6b5bb1b0 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81b30d0 con 0x556b6b5e4070 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78b63b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81b1c60 con 0x556b6b5c3390 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 0x556b6b98d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 0x556b6b98d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53520/0 (socket says 172.21.15.92:53520) 2024-09-21T02:12:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.362+0000 7f78ad3da640 1 -- 172.21.15.92:0/2283396936 learned_addr learned my addr 172.21.15.92:0/2283396936 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78ad3da640 1 -- 172.21.15.92:0/2283396936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 msgr2=0x556b6b998940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78adbdb640 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b998940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b998940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78ad3da640 1 -- 172.21.15.92:0/2283396936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5e4070 msgr2=0x556b6b988b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5e4070 0x556b6b988b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78ad3da640 1 -- 172.21.15.92:0/2283396936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6b98d940 con 0x556b6b5bb1b0 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 0x556b6b98d040 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f789c006010 tx=0x7f789c02ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab2a7fb439dc26e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f788f7fe640 1 -- 172.21.15.92:0/2283396936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c035070 con 0x556b6b5bb1b0 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f788f7fe640 1 -- 172.21.15.92:0/2283396936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789c032e70 con 0x556b6b5bb1b0 2024-09-21T02:12:02.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.363+0000 7f788f7fe640 1 -- 172.21.15.92:0/2283396936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c004920 con 0x556b6b5bb1b0 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 -- 172.21.15.92:0/2283396936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 msgr2=0x556b6b98d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 0x556b6b98d040 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f789c006010 tx=0x7f789c02ab60 comp rx=0 tx=0).stop 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 -- 172.21.15.92:0/2283396936 shutdown_connections 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5e4070 0x556b6b988b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5c3390 0x556b6b998940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2283396936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5bb1b0 0x556b6b98d040 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 -- 172.21.15.92:0/2283396936 >> 172.21.15.92:0/2283396936 conn(0x556b6b5d8460 msgr2=0x556b6b998230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 -- 172.21.15.92:0/2283396936 shutdown_connections 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 -- 172.21.15.92:0/2283396936 wait complete. 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 -- start start 2024-09-21T02:12:02.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.364+0000 7f78b63b3740 1 Processor -- start 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b9ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b9a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81b2a00 con 0x556b6b5e4070 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81b2330 con 0x556b6b5c3390 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81af570 con 0x556b6b5bb1b0 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78b63b3740 10 client.0 _ll_register_callbacks cb 0x7f78a0009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53522/0 (socket says 172.21.15.92:53522) 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 -- 172.21.15.92:0/2606557941 learned_addr learned my addr 172.21.15.92:0/2606557941 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78adbdb640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b9a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b9ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 -- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 msgr2=0x556b6b9ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b9ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 -- 172.21.15.92:0/2606557941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 msgr2=0x556b6b9a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b9a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.365+0000 7f78ad3da640 1 -- 172.21.15.92:0/2606557941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6b9aa940 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f78ad3da640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9aa1b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x556b6b989bd0 tx=0x7f789c031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00a53a42bcce1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f788d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f788d7fa640 1 -- 172.21.15.92:0/2606557941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c035070 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f788d7fa640 1 -- 172.21.15.92:0/2606557941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f789c0048c0 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b6b8ff850 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f788d7fa640 1 -- 172.21.15.92:0/2606557941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f789c06f070 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f78b63b3740 10 client.17610 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b6ba135b0 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f788d7fa640 1 -- 172.21.15.92:0/2606557941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556b6b8ff850 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.366+0000 7f788d7fa640 1 -- 172.21.15.92:0/2606557941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f789c06b4b0 con 0x556b6b5e4070 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78b63b3740 10 client.17610 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:02.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78b63b3740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78b63b3740 20 client.17610 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b577da0 0x556b6ba15c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78b63b3740 4 client.17610 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b6b8fcc80 con 0x556b6b577da0 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78adbdb640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b577da0 0x556b6ba15c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f78adbdb640 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b577da0 0x556b6ba15c00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c07768f2bdf7bc server_cookie=6ba6607d417991cc in_seq=0 out_seq=0 2024-09-21T02:12:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.367+0000 7f788d7fa640 10 client.17610 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f788d7fa640 1 -- 172.21.15.92:0/2606557941 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x556b6b8fcc80 con 0x556b6b577da0 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f788d7fa640 10 client.17610 handle_command_reply: tid=0 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 client.17610 shutdown 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 msgr2=0x556b6b9aa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9aa1b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x556b6b989bd0 tx=0x7f789c031620 comp rx=0 tx=0).stop 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 shutdown_connections 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556b6b577da0 0x556b6ba15c00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556b6b5e4070 0x556b6b9aa1b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6b5c3390 0x556b6b9a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f788d7fa640 0 client.17610 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 --2- 172.21.15.92:0/2606557941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b6b5bb1b0 0x556b6b9ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 >> 172.21.15.92:0/2606557941 conn(0x556b6b5d8460 msgr2=0x556b6b98f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 shutdown_connections 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 1 -- 172.21.15.92:0/2606557941 wait complete. 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.368+0000 7f78b63b3740 20 client.17610 trim_cache size 0 max 16384 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "0b7521ce-80b1-4138-a4f9-de46c10a80b9", 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:02.380 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:12:02.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f78900b6510 msgr2=0x7f78900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f78900b6510 0x7f78900b8a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f789800d720 tx=0x7f7898094040 comp rx=0 tx=0).stop 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 msgr2=0x7f78a81a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 0x7f78a81a1ff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f789800d750 tx=0x7f789800dc10 comp rx=0 tx=0).stop 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 shutdown_connections 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f78900b6510 0x7f78900b8a20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8109ec0 0x7f78a81a1ff0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a8109570 0x7f78a819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 --2- 172.21.15.92:0/304792005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8103c30 0x7f78a819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 >> 172.21.15.92:0/304792005 conn(0x7f78a807b8d0 msgr2=0x7f78a8105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 shutdown_connections 2024-09-21T02:12:02.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:02.369+0000 7f78afe3f640 1 -- 172.21.15.92:0/304792005 wait complete. 2024-09-21T02:12:03.020 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:02 smithi092 ceph-mon[31190]: pgmap v572: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:03.046 INFO:tasks.cephfs.filesystem:scrub status for tag:0b7521ce-80b1-4138-a4f9-de46c10a80b9 - {'path': '/', 'tag': '0b7521ce-80b1-4138-a4f9-de46c10a80b9', 'options': 'recursive,force'} 2024-09-21T02:12:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:02 smithi099 ceph-mon[34404]: pgmap v572: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:02 smithi136 ceph-mon[34754]: pgmap v572: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:04.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:03 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:12:04.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:03 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:12:04.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:03 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:12:05.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:04 smithi092 ceph-mon[31190]: pgmap v573: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:12:05.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:04 smithi099 ceph-mon[34404]: pgmap v573: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:12:05.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:04 smithi136 ceph-mon[34754]: pgmap v573: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-09-21T02:12:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:06 smithi092 ceph-mon[31190]: pgmap v574: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:12:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:06 smithi099 ceph-mon[34404]: pgmap v574: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:12:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:06 smithi136 ceph-mon[34754]: pgmap v574: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 -- 172.21.15.92:0/52824193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc103620 msgr2=0x7f31dc103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 --2- 172.21.15.92:0/52824193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc103620 0x7f31dc103a30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009900 tx=0x7f31d002a410 comp rx=0 tx=0).stop 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 -- 172.21.15.92:0/52824193 shutdown_connections 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 --2- 172.21.15.92:0/52824193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 0x7f31dc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 --2- 172.21.15.92:0/52824193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc103620 0x7f31dc103a30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 --2- 172.21.15.92:0/52824193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc1025a0 0x7f31dc102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 -- 172.21.15.92:0/52824193 >> 172.21.15.92:0/52824193 conn(0x7f31dc0fdee0 msgr2=0x7f31dc100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 -- 172.21.15.92:0/52824193 shutdown_connections 2024-09-21T02:12:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.005+0000 7f31e2825640 1 -- 172.21.15.92:0/52824193 wait complete. 2024-09-21T02:12:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.006+0000 7f31e2825640 1 Processor -- start 2024-09-21T02:12:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.006+0000 7f31e2825640 1 -- start start 2024-09-21T02:12:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 0x7f31dc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc103620 0x7f31dc199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 0x7f31dc19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc10e200 con 0x7f31dc1025a0 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc10e080 con 0x7f31dc103fc0 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc10e380 con 0x7f31dc103620 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 0x7f31dc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 0x7f31dc198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35862/0 (socket says 172.21.15.92:35862) 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 -- 172.21.15.92:0/1702918262 learned_addr learned my addr 172.21.15.92:0/1702918262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 0x7f31dc19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31db7fe640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc103620 0x7f31dc199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 -- 172.21.15.92:0/1702918262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc103620 msgr2=0x7f31dc199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc103620 0x7f31dc199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 -- 172.21.15.92:0/1702918262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 msgr2=0x7f31dc19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 0x7f31dc19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 -- 172.21.15.92:0/1702918262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc19a8f0 con 0x7f31dc1025a0 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 0x7f31dc19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31db7fe640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc103620 0x7f31dc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31dbfff640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 0x7f31dc198eb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00c970 tx=0x7f31cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31d97fa640 1 -- 172.21.15.92:0/1702918262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc007bb0 con 0x7f31dc1025a0 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc19abe0 con 0x7f31dc1025a0 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31d97fa640 1 -- 172.21.15.92:0/1702918262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31cc007d50 con 0x7f31dc1025a0 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31d97fa640 1 -- 172.21.15.92:0/1702918262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00f450 con 0x7f31dc1025a0 2024-09-21T02:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.007+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1a12d0 con 0x7f31dc1025a0 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 Processor -- start 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 -- start start 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ffc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b881ffc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b881f6c6f0 con 0x55b881c314d0 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b881f5bb20 con 0x55b881c292f0 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e9871740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b881f5f930 con 0x55b881c521b0 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31d97fa640 1 -- 172.21.15.92:0/1702918262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f31cc004f00 con 0x7f31dc1025a0 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35872/0 (socket says 172.21.15.92:35872) 2024-09-21T02:12:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31db7fe640 1 -- 172.21.15.92:0/412306670 learned_addr learned my addr 172.21.15.92:0/412306670 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.010+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ffc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31db7fe640 1 -- 172.21.15.92:0/412306670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 msgr2=0x55b881ffc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31db7fe640 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ffc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31db7fe640 1 -- 172.21.15.92:0/412306670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 msgr2=0x55b881ffc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31db7fe640 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b881ffc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31db7fe640 1 -- 172.21.15.92:0/412306670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b881f21530 con 0x55b881c314d0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31db7fe640 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffd220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f31d0005940 tx=0x7f31d0031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7b2461dd376ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31c27fc640 1 -- 172.21.15.92:0/412306670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d003f070 con 0x55b881c314d0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31c27fc640 1 -- 172.21.15.92:0/412306670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d0031cd0 con 0x55b881c314d0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31c27fc640 1 -- 172.21.15.92:0/412306670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0035030 con 0x55b881c314d0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31d97fa640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31c40b6510 0x7f31c40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31c40b6510 0x7f31c40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.011+0000 7f31d97fa640 1 -- 172.21.15.92:0/1702918262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f31cc0e70f0 con 0x7f31dc1025a0 2024-09-21T02:12:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 -- 172.21.15.92:0/412306670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 msgr2=0x55b881ffd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffd220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f31d0005940 tx=0x7f31d0031870 comp rx=0 tx=0).stop 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31c40b6510 0x7f31c40b8a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f31d4008e20 tx=0x7f31d40083b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 -- 172.21.15.92:0/412306670 shutdown_connections 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffd220 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b881ffc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 --2- 172.21.15.92:0/412306670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ffc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 -- 172.21.15.92:0/412306670 >> 172.21.15.92:0/412306670 conn(0x55b881c46300 msgr2=0x55b881ff58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 -- 172.21.15.92:0/412306670 shutdown_connections 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 -- 172.21.15.92:0/412306670 wait complete. 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.012+0000 7f31e9871740 1 -- start start 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 Processor -- start 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b8820057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c314d0 0x55b882005e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 0x55b8820069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b881e93ab0 con 0x55b881c521b0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b881f01730 con 0x55b881c314d0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b881ffdce0 con 0x55b881c292f0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e9871740 10 client.0 _ll_register_callbacks cb 0x55b881f556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e0dc2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 0x55b8820069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e0dc2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 0x55b8820069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35882/0 (socket says 172.21.15.92:35882) 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31e0dc2640 1 -- 172.21.15.92:0/3018593331 learned_addr learned my addr 172.21.15.92:0/3018593331 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31dbfff640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b8820057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.013+0000 7f31db7fe640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c314d0 0x55b882005e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e0dc2640 1 -- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 msgr2=0x55b8820057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b8820057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e0dc2640 1 -- 172.21.15.92:0/3018593331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c314d0 msgr2=0x55b882005e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c314d0 0x55b882005e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e0dc2640 1 -- 172.21.15.92:0/3018593331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b881cbd790 con 0x55b881c521b0 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31db7fe640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c314d0 0x55b882005e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 0x55b8820069e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f31d4010d70 tx=0x7f31d4004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18546dd23c9ad0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3018593331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4010e60 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31d0031cd0 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3018593331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f31d401d620 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3018593331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31d401d820 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 10 client.17622 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b881d36620 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3018593331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d0031cd0 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3018593331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31d4030b60 con 0x55b881c521b0 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 10 client.17622 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 20 client.17622 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.014+0000 7f31e9871740 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881c10d80 0x55b882012fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.015+0000 7f31e9871740 4 client.17622 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.015+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b881f6e640 con 0x55b881c10d80 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.015+0000 7f31dbfff640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881c10d80 0x55b882012fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.015+0000 7f31dbfff640 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881c10d80 0x55b882012fb0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47542644ae1b6e9 server_cookie=2939d8880486dce6 in_seq=0 out_seq=0 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.015+0000 7f31bb7fe640 10 client.17622 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.016+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3018593331 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b881f6e640 con 0x55b881c10d80 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.016+0000 7f31bb7fe640 10 client.17622 handle_command_reply: tid=0 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.016+0000 7f31e9871740 1 client.17622 shutdown 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 msgr2=0x55b8820069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 0x55b8820069e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f31d4010d70 tx=0x7f31d4004a90 comp rx=0 tx=0).stop 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 shutdown_connections 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881c10d80 0x55b882012fb0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c521b0 0x55b8820069e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c314d0 0x55b882005e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 --2- 172.21.15.92:0/3018593331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b8820057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31bb7fe640 0 client.17622 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 >> 172.21.15.92:0/3018593331 conn(0x55b881c46300 msgr2=0x55b881ff3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 shutdown_connections 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 1 -- 172.21.15.92:0/3018593331 wait complete. 2024-09-21T02:12:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.017+0000 7f31e9871740 20 client.17622 trim_cache size 0 max 16384 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 Processor -- start 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 -- start start 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b882006f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ff3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1a8e80 con 0x55b881c314d0 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1aa9c0 con 0x55b881c292f0 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31e9871740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1a9550 con 0x55b881c521b0 2024-09-21T02:12:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.031+0000 7f31db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35898/0 (socket says 172.21.15.92:35898) 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 -- 172.21.15.92:0/2436051855 learned_addr learned my addr 172.21.15.92:0/2436051855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ff3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31dbfff640 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b882006f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 -- 172.21.15.92:0/2436051855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 msgr2=0x55b881ff3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ff3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 -- 172.21.15.92:0/2436051855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 msgr2=0x55b882006f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b882006f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 -- 172.21.15.92:0/2436051855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8820061e0 con 0x55b881c314d0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31db7fe640 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffcef0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009a20 tx=0x7f31d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f169118d0c5e0385 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31c1ffb640 1 -- 172.21.15.92:0/2436051855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d003f070 con 0x55b881c314d0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31c1ffb640 1 -- 172.21.15.92:0/2436051855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d0031b90 con 0x55b881c314d0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.032+0000 7f31c1ffb640 1 -- 172.21.15.92:0/2436051855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0035040 con 0x55b881c314d0 2024-09-21T02:12:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 -- 172.21.15.92:0/2436051855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 msgr2=0x55b881ffcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffcef0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009a20 tx=0x7f31d0005fd0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 -- 172.21.15.92:0/2436051855 shutdown_connections 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c521b0 0x55b881ff3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b881ffcef0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 --2- 172.21.15.92:0/2436051855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c292f0 0x55b882006f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 -- 172.21.15.92:0/2436051855 >> 172.21.15.92:0/2436051855 conn(0x55b881c46300 msgr2=0x55b882000ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 -- 172.21.15.92:0/2436051855 shutdown_connections 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 -- 172.21.15.92:0/2436051855 wait complete. 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 -- start start 2024-09-21T02:12:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.033+0000 7f31e9871740 1 Processor -- start 2024-09-21T02:12:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b88200a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b88200b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c521b0 0x55b88200bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1aa2f0 con 0x55b881c314d0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1a9c20 con 0x55b881c521b0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1a6e60 con 0x55b881c292f0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e9871740 10 client.0 _ll_register_callbacks cb 0x7f31d4008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b88200b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b88200b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35914/0 (socket says 172.21.15.92:35914) 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 -- 172.21.15.92:0/3269326765 learned_addr learned my addr 172.21.15.92:0/3269326765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31e0dc2640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c521b0 0x55b88200bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31dbfff640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b88200a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 -- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 msgr2=0x55b88200a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b88200a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 -- 172.21.15.92:0/3269326765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c521b0 msgr2=0x55b88200bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c521b0 0x55b88200bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 -- 172.21.15.92:0/3269326765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88200c320 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31db7fe640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b88200b050 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f31d002a450 tx=0x7f31d0009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acca67bd859e093 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.034+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3269326765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d003f070 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31d0031b90 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3269326765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f31d0003b20 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3269326765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31d0035040 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 10 client.17628 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b88200d640 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3269326765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d0031b90 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3269326765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31d003b8f0 con 0x55b881c314d0 2024-09-21T02:12:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 10 client.17628 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 20 client.17628 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881be61c0 0x55b88207f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 4 client.17628 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b881f6a0f0 con 0x55b881be61c0 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.035+0000 7f31dbfff640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881be61c0 0x55b88207f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.036+0000 7f31bb7fe640 10 client.17628 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.036+0000 7f31dbfff640 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881be61c0 0x55b88207f980 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd84b1b8af50e0a9 server_cookie=189a22e67c6a6504 in_seq=0 out_seq=0 2024-09-21T02:12:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31bb7fe640 1 -- 172.21.15.92:0/3269326765 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b881f6a0f0 con 0x55b881be61c0 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31bb7fe640 10 client.17628 handle_command_reply: tid=0 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 client.17628 shutdown 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 msgr2=0x55b88200b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b88200b050 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f31d002a450 tx=0x7f31d0009520 comp rx=0 tx=0).stop 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 shutdown_connections 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b881be61c0 0x55b88207f980 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b881c521b0 0x55b88200bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31bb7fe640 0 client.17628 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b881c314d0 0x55b88200b050 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 --2- 172.21.15.92:0/3269326765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b881c292f0 0x55b88200a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.049+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 >> 172.21.15.92:0/3269326765 conn(0x55b881c46300 msgr2=0x55b882000ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 shutdown_connections 2024-09-21T02:12:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e9871740 1 -- 172.21.15.92:0/3269326765 wait complete. 2024-09-21T02:12:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e9871740 20 client.17628 trim_cache size 0 max 16384 2024-09-21T02:12:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:12:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:12:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:12:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31c40b6510 msgr2=0x7f31c40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31c40b6510 0x7f31c40b8a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f31d4008e20 tx=0x7f31d40083b0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 msgr2=0x7f31dc198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 0x7f31dc198eb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00c970 tx=0x7f31cc00ce30 comp rx=0 tx=0).stop 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 shutdown_connections 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f31c40b6510 0x7f31c40b8a20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31dc103fc0 0x7f31dc19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31dc103620 0x7f31dc199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 --2- 172.21.15.92:0/1702918262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31dc1025a0 0x7f31dc198eb0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 >> 172.21.15.92:0/1702918262 conn(0x7f31dc0fdee0 msgr2=0x7f31dc106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 shutdown_connections 2024-09-21T02:12:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:09.050+0000 7f31e2825640 1 -- 172.21.15.92:0/1702918262 wait complete. 2024-09-21T02:12:09.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:08 smithi099 ceph-mon[34404]: pgmap v575: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:08 smithi136 ceph-mon[34754]: pgmap v575: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:08 smithi092 ceph-mon[31190]: pgmap v575: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:10 smithi099 ceph-mon[34404]: pgmap v576: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:12:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:10 smithi136 ceph-mon[34754]: pgmap v576: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:12:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:10 smithi092 ceph-mon[31190]: pgmap v576: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:12:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:12 smithi099 ceph-mon[34404]: pgmap v577: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:12:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:12 smithi136 ceph-mon[34754]: pgmap v577: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:12:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:12 smithi092 ceph-mon[31190]: pgmap v577: 97 pgs: 97 active+clean; 1.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:12:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:14 smithi099 ceph-mon[34404]: pgmap v578: 97 pgs: 97 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:12:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:14 smithi136 ceph-mon[34754]: pgmap v578: 97 pgs: 97 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:12:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:14 smithi092 ceph-mon[31190]: pgmap v578: 97 pgs: 97 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:12:17.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:16 smithi099 ceph-mon[34404]: pgmap v579: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:12:17.107 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:16 smithi092 ceph-mon[31190]: pgmap v579: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:12:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:16 smithi136 ceph-mon[34754]: pgmap v579: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:12:19.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:18 smithi099 ceph-mon[34404]: pgmap v580: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:18 smithi136 ceph-mon[34754]: pgmap v580: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:19.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:18 smithi092 ceph-mon[31190]: pgmap v580: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:20 smithi099 ceph-mon[34404]: pgmap v581: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:12:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:20 smithi136 ceph-mon[34754]: pgmap v581: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:12:21.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:20 smithi092 ceph-mon[31190]: pgmap v581: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:12:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2177772332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e181036c0 msgr2=0x7f7e18103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2177772332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e181036c0 0x7f7e18103b30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009900 tx=0x7f7e0802a410 comp rx=0 tx=0).stop 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2177772332 shutdown_connections 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2177772332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181050e0 0x7f7e1810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2177772332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e18104740 0x7f7e18104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2177772332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e181036c0 0x7f7e18103b30 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2177772332 >> 172.21.15.92:0/2177772332 conn(0x7f7e180fefe0 msgr2=0x7f7e181014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2177772332 shutdown_connections 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.157+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2177772332 wait complete. 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 Processor -- start 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 -- start start 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181036c0 0x7f7e1819c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 0x7f7e1819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e181050e0 0x7f7e1819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1810f300 con 0x7f7e18104740 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1810f180 con 0x7f7e181036c0 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e1d49b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1810f480 con 0x7f7e181050e0 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e167fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 0x7f7e1819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e167fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 0x7f7e1819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39812/0 (socket says 172.21.15.92:39812) 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e167fc640 1 -- 172.21.15.92:0/2216709176 learned_addr learned my addr 172.21.15.92:0/2216709176 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.158+0000 7f7e177fe640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e181050e0 0x7f7e1819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181036c0 0x7f7e1819c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e167fc640 1 -- 172.21.15.92:0/2216709176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e181050e0 msgr2=0x7f7e1819d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e167fc640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e181050e0 0x7f7e1819d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e167fc640 1 -- 172.21.15.92:0/2216709176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181036c0 msgr2=0x7f7e1819c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e167fc640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181036c0 0x7f7e1819c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e167fc640 1 -- 172.21.15.92:0/2216709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1819dbc0 con 0x7f7e18104740 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181036c0 0x7f7e1819c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e167fc640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 0x7f7e1819c7f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000e9f0 tx=0x7f7e0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e0ffff640 1 -- 172.21.15.92:0/2216709176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000cd80 con 0x7f7e18104740 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e0ffff640 1 -- 172.21.15.92:0/2216709176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e000054d0 con 0x7f7e18104740 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1819deb0 con 0x7f7e18104740 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e0ffff640 1 -- 172.21.15.92:0/2216709176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00010690 con 0x7f7e18104740 2024-09-21T02:12:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.159+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a45a0 con 0x7f7e18104740 2024-09-21T02:12:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 Processor -- start 2024-09-21T02:12:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 -- start start 2024-09-21T02:12:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e0ffff640 1 -- 172.21.15.92:0/2216709176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f7e00004e90 con 0x7f7e18104740 2024-09-21T02:12:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e3001b0 0x556d4e6a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2df4d0 0x556d4e6a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556d4e61a6f0 con 0x556d4e2d72f0 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d4e609b20 con 0x556d4e2df4d0 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e24413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556d4e60d930 con 0x556d4e3001b0 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e16ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39828/0 (socket says 172.21.15.92:39828) 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.162+0000 7f7e16ffd640 1 -- 172.21.15.92:0/2650209990 learned_addr learned my addr 172.21.15.92:0/2650209990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e177fe640 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e3001b0 0x556d4e6a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e167fc640 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2df4d0 0x556d4e6a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e16ffd640 1 -- 172.21.15.92:0/2650209990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e3001b0 msgr2=0x556d4e6a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e3001b0 0x556d4e6a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e16ffd640 1 -- 172.21.15.92:0/2650209990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2df4d0 msgr2=0x556d4e6a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2df4d0 0x556d4e6a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e16ffd640 1 -- 172.21.15.92:0/2650209990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4e5cf530 con 0x556d4e2d72f0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e167fc640 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2df4d0 0x556d4e6a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a84c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7e0802a930 tx=0x7f7e08031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec92591d45eb5e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e0d7fa640 1 -- 172.21.15.92:0/2650209990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08035030 con 0x556d4e2d72f0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e0ffff640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e040b6510 0x7f7e040b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e0d7fa640 1 -- 172.21.15.92:0/2650209990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e080318d0 con 0x556d4e2d72f0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e0d7fa640 1 -- 172.21.15.92:0/2650209990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e080046c0 con 0x556d4e2d72f0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.163+0000 7f7e177fe640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e040b6510 0x7f7e040b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e0ffff640 1 -- 172.21.15.92:0/2216709176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7e000e77c0 con 0x7f7e18104740 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 -- 172.21.15.92:0/2650209990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 msgr2=0x556d4e6a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e177fe640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e040b6510 0x7f7e040b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005fd0 tx=0x7f7e10009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a84c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7e0802a930 tx=0x7f7e08031620 comp rx=0 tx=0).stop 2024-09-21T02:12:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 -- 172.21.15.92:0/2650209990 shutdown_connections 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2df4d0 0x556d4e6a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a84c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 --2- 172.21.15.92:0/2650209990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e3001b0 0x556d4e6a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.164+0000 7f7e24413740 1 -- 172.21.15.92:0/2650209990 >> 172.21.15.92:0/2650209990 conn(0x556d4e2f4300 msgr2=0x556d4e6a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 -- 172.21.15.92:0/2650209990 shutdown_connections 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 -- 172.21.15.92:0/2650209990 wait complete. 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 -- start start 2024-09-21T02:12:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 Processor -- start 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2d72f0 0x556d4e6b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2df4d0 0x556d4e6b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 0x556d4e6b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556d4e541ab0 con 0x556d4e3001b0 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d4e5af730 con 0x556d4e2d72f0 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556d4e6a9ad0 con 0x556d4e2df4d0 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e24413740 10 client.0 _ll_register_callbacks cb 0x556d4e6036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 0x556d4e6b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 0x556d4e6b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39830/0 (socket says 172.21.15.92:39830) 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e177fe640 1 -- 172.21.15.92:0/3913558239 learned_addr learned my addr 172.21.15.92:0/3913558239 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.165+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2d72f0 0x556d4e6b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e167fc640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2df4d0 0x556d4e6b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e177fe640 1 -- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2df4d0 msgr2=0x556d4e6b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e177fe640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2df4d0 0x556d4e6b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e177fe640 1 -- 172.21.15.92:0/3913558239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2d72f0 msgr2=0x556d4e6b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e177fe640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2d72f0 0x556d4e6b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e177fe640 1 -- 172.21.15.92:0/3913558239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4e36b790 con 0x556d4e3001b0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e177fe640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 0x556d4e6b27c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7e10004070 tx=0x7f7e10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd3abfde63479b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7dfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/3913558239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1000f970 con 0x556d4e3001b0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e080318d0 con 0x556d4e3001b0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/3913558239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7e1000fb10 con 0x556d4e3001b0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/3913558239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e1000fd10 con 0x556d4e3001b0 2024-09-21T02:12:22.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e24413740 10 client.17640 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d4e3e4620 con 0x556d4e3001b0 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.166+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/3913558239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e080318d0 con 0x556d4e3001b0 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/3913558239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7e10026020 con 0x556d4e3001b0 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e24413740 10 client.17640 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e24413740 10 client.17640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e24413740 20 client.17640 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e24413740 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2bed80 0x556d4e6bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e24413740 4 client.17640 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d4e61c640 con 0x556d4e2bed80 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.167+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2bed80 0x556d4e6bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.168+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2bed80 0x556d4e6bed30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cccae5f108b995 server_cookie=448b4b1d17b4da4a in_seq=0 out_seq=0 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.168+0000 7f7dfe7fc640 10 client.17640 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.168+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/3913558239 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d4e61c640 con 0x556d4e2bed80 2024-09-21T02:12:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.168+0000 7f7dfe7fc640 10 client.17640 handle_command_reply: tid=0 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.168+0000 7f7e24413740 1 client.17640 shutdown 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 msgr2=0x556d4e6b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 0x556d4e6b27c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f7e10004070 tx=0x7f7e10004ab0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 shutdown_connections 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2bed80 0x556d4e6bed30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e3001b0 0x556d4e6b27c0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2df4d0 0x556d4e6b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7dfe7fc640 0 client.17640 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 --2- 172.21.15.92:0/3913558239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2d72f0 0x556d4e6b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 >> 172.21.15.92:0/3913558239 conn(0x556d4e2f4300 msgr2=0x556d4e6a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 shutdown_connections 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 1 -- 172.21.15.92:0/3913558239 wait complete. 2024-09-21T02:12:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.169+0000 7f7e24413740 20 client.17640 trim_cache size 0 max 16384 2024-09-21T02:12:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.183+0000 7f7e24413740 1 Processor -- start 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.183+0000 7f7e24413740 1 -- start start 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e24413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e24413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e24413740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e24413740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ad220 con 0x556d4e2d72f0 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e24413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181aed60 con 0x556d4e2bed80 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e24413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ad8f0 con 0x556d4e2f4300 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39832/0 (socket says 172.21.15.92:39832) 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 -- 172.21.15.92:0/1759615023 learned_addr learned my addr 172.21.15.92:0/1759615023 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e167fc640 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 -- 172.21.15.92:0/1759615023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 msgr2=0x556d4e6afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 -- 172.21.15.92:0/1759615023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 msgr2=0x556d4e6b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.184+0000 7f7e177fe640 1 -- 172.21.15.92:0/1759615023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4e6a4010 con 0x556d4e2d72f0 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.185+0000 7f7e177fe640 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a7df0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009690 tx=0x7f7e10011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372eca4a650b54f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.185+0000 7f7e0cff9640 1 -- 172.21.15.92:0/1759615023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10003db0 con 0x556d4e2d72f0 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.185+0000 7f7e0cff9640 1 -- 172.21.15.92:0/1759615023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1000f970 con 0x556d4e2d72f0 2024-09-21T02:12:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.185+0000 7f7e0cff9640 1 -- 172.21.15.92:0/1759615023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10028ae0 con 0x556d4e2d72f0 2024-09-21T02:12:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- 172.21.15.92:0/1759615023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 msgr2=0x556d4e6a7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a7df0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009690 tx=0x7f7e10011f10 comp rx=0 tx=0).stop 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- 172.21.15.92:0/1759615023 shutdown_connections 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6a7df0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- 172.21.15.92:0/1759615023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- 172.21.15.92:0/1759615023 >> 172.21.15.92:0/1759615023 conn(0x556d4e6bf060 msgr2=0x556d4e6b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- 172.21.15.92:0/1759615023 shutdown_connections 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- 172.21.15.92:0/1759615023 wait complete. 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- start start 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 Processor -- start 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ae690 con 0x556d4e2d72f0 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181adfc0 con 0x556d4e2bed80 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181ab200 con 0x556d4e2f4300 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e24413740 10 client.0 _ll_register_callbacks cb 0x7f7e10009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e167fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e167fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39844/0 (socket says 172.21.15.92:39844) 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.186+0000 7f7e167fc640 1 -- 172.21.15.92:0/245583860 learned_addr learned my addr 172.21.15.92:0/245583860 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e177fe640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e167fc640 1 -- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 msgr2=0x556d4e6b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e167fc640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e167fc640 1 -- 172.21.15.92:0/245583860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 msgr2=0x556d4e6b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e167fc640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e167fc640 1 -- 172.21.15.92:0/245583860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4e6b9450 con 0x556d4e2d72f0 2024-09-21T02:12:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e177fe640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e167fc640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6b8b80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7e000a9f00 tx=0x7f7e000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6000081a4429503b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7dfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/245583860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e000a90a0 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/245583860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7e000a9280 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1000f970 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/245583860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e000b2200 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e24413740 10 client.17646 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d4e729d70 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.187+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/245583860 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1000f970 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/245583860 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7e000a9480 con 0x556d4e2d72f0 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e24413740 10 client.17646 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e24413740 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e24413740 20 client.17646 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e24413740 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2df4d0 0x556d4e72c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e24413740 4 client.17646 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:12:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d4e6180f0 con 0x556d4e2df4d0 2024-09-21T02:12:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2df4d0 0x556d4e72c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7dfe7fc640 10 client.17646 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.188+0000 7f7e16ffd640 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2df4d0 0x556d4e72c760 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782ef899c6cc6e0a server_cookie=b77f554db3785bf6 in_seq=0 out_seq=0 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7dfe7fc640 1 -- 172.21.15.92:0/245583860 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d4e6180f0 con 0x556d4e2df4d0 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7dfe7fc640 10 client.17646 handle_command_reply: tid=0 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 client.17646 shutdown 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 msgr2=0x556d4e6b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6b8b80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7e000a9f00 tx=0x7f7e000048f0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 shutdown_connections 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d4e2df4d0 0x556d4e72c760 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d4e2f4300 0x556d4e6b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d4e2d72f0 0x556d4e6b8b80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 --2- 172.21.15.92:0/245583860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d4e2bed80 0x556d4e6b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7dfe7fc640 0 client.17646 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 >> 172.21.15.92:0/245583860 conn(0x556d4e6bf060 msgr2=0x556d4e6a25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 shutdown_connections 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 1 -- 172.21.15.92:0/245583860 wait complete. 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.203+0000 7f7e24413740 20 client.17646 trim_cache size 0 max 16384 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:12:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:12:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e040b6510 msgr2=0x7f7e040b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e040b6510 0x7f7e040b8a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005fd0 tx=0x7f7e10009500 comp rx=0 tx=0).stop 2024-09-21T02:12:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 msgr2=0x7f7e1819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 0x7f7e1819c7f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000e9f0 tx=0x7f7e0000edf0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 shutdown_connections 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f7e040b6510 0x7f7e040b8a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e181050e0 0x7f7e1819d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e18104740 0x7f7e1819c7f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 --2- 172.21.15.92:0/2216709176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e181036c0 0x7f7e1819c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 >> 172.21.15.92:0/2216709176 conn(0x7f7e180fefe0 msgr2=0x7f7e180755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 shutdown_connections 2024-09-21T02:12:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:22.204+0000 7f7e1d49b640 1 -- 172.21.15.92:0/2216709176 wait complete. 2024-09-21T02:12:23.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:22 smithi099 ceph-mon[34404]: pgmap v582: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:22 smithi136 ceph-mon[34754]: pgmap v582: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:23.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:22 smithi092 ceph-mon[31190]: pgmap v582: 97 pgs: 97 active+clean; 1.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:25.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:24 smithi136 ceph-mon[34754]: pgmap v583: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-09-21T02:12:25.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:24 smithi092 ceph-mon[31190]: pgmap v583: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-09-21T02:12:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:24 smithi099 ceph-mon[34404]: pgmap v583: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-09-21T02:12:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:26 smithi136 ceph-mon[34754]: pgmap v584: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:12:27.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:26 smithi092 ceph-mon[31190]: pgmap v584: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:12:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:26 smithi099 ceph-mon[34404]: pgmap v584: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:12:29.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:28 smithi099 ceph-mon[34404]: pgmap v585: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:28 smithi136 ceph-mon[34754]: pgmap v585: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:29.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:28 smithi092 ceph-mon[31190]: pgmap v585: 97 pgs: 97 active+clean; 1.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:30 smithi136 ceph-mon[34754]: pgmap v586: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:12:31.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:30 smithi092 ceph-mon[31190]: pgmap v586: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:12:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:30 smithi099 ceph-mon[34404]: pgmap v586: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:12:33.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:12:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:32 smithi136 ceph-mon[34754]: pgmap v587: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:33.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:32 smithi092 ceph-mon[31190]: pgmap v587: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:33.318 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:32 smithi099 ceph-mon[34404]: pgmap v587: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 -- 172.21.15.92:0/1558565790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 msgr2=0x7f00e01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 --2- 172.21.15.92:0/1558565790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e01087b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009900 tx=0x7f00dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 -- 172.21.15.92:0/1558565790 shutdown_connections 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 --2- 172.21.15.92:0/1558565790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 0x7f00e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 --2- 172.21.15.92:0/1558565790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e01087b0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 --2- 172.21.15.92:0/1558565790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 0x7f00e0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 -- 172.21.15.92:0/1558565790 >> 172.21.15.92:0/1558565790 conn(0x7f00e0079730 msgr2=0x7f00e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 -- 172.21.15.92:0/1558565790 shutdown_connections 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.822+0000 7f00e838a640 1 -- 172.21.15.92:0/1558565790 wait complete. 2024-09-21T02:12:33.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 Processor -- start 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 -- start start 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 0x7f00e0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 0x7f00e019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0112960 con 0x7f00e01083a0 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01127e0 con 0x7f00e0108cf0 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e838a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0112ae0 con 0x7f00e0102a60 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e5925640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e5925640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57838/0 (socket says 172.21.15.92:57838) 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e5925640 1 -- 172.21.15.92:0/2918544697 learned_addr learned my addr 172.21.15.92:0/2918544697 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.823+0000 7f00e6126640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 0x7f00e0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e6927640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 0x7f00e019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e5925640 1 -- 172.21.15.92:0/2918544697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 msgr2=0x7f00e0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e5925640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 0x7f00e0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e5925640 1 -- 172.21.15.92:0/2918544697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 msgr2=0x7f00e019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e5925640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 0x7f00e019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e5925640 1 -- 172.21.15.92:0/2918544697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e019f300 con 0x7f00e01083a0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e6126640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 0x7f00e0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e6927640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 0x7f00e019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e5925640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e0199930 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f00dc031a90 tx=0x7f00dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00d37fe640 1 -- 172.21.15.92:0/2918544697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc03f030 con 0x7f00e01083a0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e019f590 con 0x7f00e01083a0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00d37fe640 1 -- 172.21.15.92:0/2918544697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc003cb0 con 0x7f00e01083a0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01a5d50 con 0x7f00e01083a0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.824+0000 7f00d37fe640 1 -- 172.21.15.92:0/2918544697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc035070 con 0x7f00e01083a0 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.826+0000 7f00ee8fe740 1 Processor -- start 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 -- start start 2024-09-21T02:12:33.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00d37fe640 1 -- 172.21.15.92:0/2918544697 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f00dc042030 con 0x7f00e01083a0 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab96b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 0x559cab96bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5a3390 0x559cab96c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559cab8e1e50 con 0x559cab59b1b0 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559cab8d1280 con 0x559cab5a3390 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559cab8d5090 con 0x559cab5c4070 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00e6126640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 0x559cab96bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00e6126640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 0x559cab96bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57846/0 (socket says 172.21.15.92:57846) 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00e6126640 1 -- 172.21.15.92:0/3032155962 learned_addr learned my addr 172.21.15.92:0/3032155962 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00e5925640 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5a3390 0x559cab96c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.827+0000 7f00e6927640 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab96b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6126640 1 -- 172.21.15.92:0/3032155962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 msgr2=0x559cab96b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6126640 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab96b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6126640 1 -- 172.21.15.92:0/3032155962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5a3390 msgr2=0x559cab96c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6126640 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5a3390 0x559cab96c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6126640 1 -- 172.21.15.92:0/3032155962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cab62f650 con 0x559cab59b1b0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00d37fe640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f00cc0b6510 0x7f00cc0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6126640 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 0x559cab96bb90 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f00d400cca0 tx=0x7f00d400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bb54e17096f81c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00e6927640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f00cc0b6510 0x7f00cc0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00d37fe640 1 -- 172.21.15.92:0/2918544697 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f00dc1076d0 con 0x7f00e01083a0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00d0d9a640 1 -- 172.21.15.92:0/3032155962 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4005220 con 0x559cab59b1b0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00d0d9a640 1 -- 172.21.15.92:0/3032155962 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d4004c50 con 0x559cab59b1b0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.828+0000 7f00d0d9a640 1 -- 172.21.15.92:0/3032155962 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4010790 con 0x559cab59b1b0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3032155962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 msgr2=0x559cab96bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00e6927640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f00cc0b6510 0x7f00cc0b8a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f00d8005fd0 tx=0x7f00d80083f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 0x559cab96bb90 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f00d400cca0 tx=0x7f00d400e720 comp rx=0 tx=0).stop 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3032155962 shutdown_connections 2024-09-21T02:12:33.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5a3390 0x559cab96c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab59b1b0 0x559cab96bb90 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3032155962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab96b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3032155962 >> 172.21.15.92:0/3032155962 conn(0x559cab5b8460 msgr2=0x559cab969540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3032155962 shutdown_connections 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3032155962 wait complete. 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.829+0000 7f00ee8fe740 1 -- start start 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 Processor -- start 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab9773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab9778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab982d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559cab732700 con 0x559cab5a3390 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559cab86c110 con 0x559cab59b1b0 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559cab971bc0 con 0x559cab5c4070 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00ee8fe740 10 client.0 _ll_register_callbacks cb 0x559cab885990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:33.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab9778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab9778e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57848/0 (socket says 172.21.15.92:57848) 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 -- 172.21.15.92:0/1219251780 learned_addr learned my addr 172.21.15.92:0/1219251780 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e6126640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab9773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e6927640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab982d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 -- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 msgr2=0x559cab982d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab982d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 -- 172.21.15.92:0/1219251780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 msgr2=0x559cab9773a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab9773a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 -- 172.21.15.92:0/1219251780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cab899c40 con 0x559cab5a3390 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e6927640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab982d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00e5925640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab9778e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0c9db0 tx=0x7f00dc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5297083329c322 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.830+0000 7f00c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00c9ffb640 1 -- 172.21.15.92:0/1219251780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc049070 con 0x559cab5a3390 2024-09-21T02:12:33.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00c9ffb640 1 -- 172.21.15.92:0/1219251780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f00dc0d6d70 con 0x559cab5a3390 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d4004c50 con 0x559cab5a3390 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00c9ffb640 1 -- 172.21.15.92:0/1219251780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00dc0cd5b0 con 0x559cab5a3390 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 10 client.17658 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559cab985a30 con 0x559cab5a3390 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00c9ffb640 1 -- 172.21.15.92:0/1219251780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d4004c50 con 0x559cab5a3390 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00c9ffb640 1 -- 172.21.15.92:0/1219251780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f00dc04c060 con 0x559cab5a3390 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 10 client.17658 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 20 client.17658 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab5829f0 0x559cab988e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 4 client.17658 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.831+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cab8e3da0 con 0x559cab5829f0 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.832+0000 7f00e6126640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab5829f0 0x559cab988e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.832+0000 7f00c9ffb640 10 client.17658 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.832+0000 7f00e6126640 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab5829f0 0x559cab988e20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e443b1209fd168 server_cookie=a0873a5199f141e9 in_seq=0 out_seq=0 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00c9ffb640 1 -- 172.21.15.92:0/1219251780 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cab8e3da0 con 0x559cab5829f0 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00c9ffb640 10 client.17658 handle_command_reply: tid=0 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 client.17658 shutdown 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 msgr2=0x559cab9778e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab9778e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0c9db0 tx=0x7f00dc0057b0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 shutdown_connections 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab5829f0 0x559cab988e20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5c4070 0x559cab982d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab9778e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00c9ffb640 0 client.17658 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/1219251780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab9773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 >> 172.21.15.92:0/1219251780 conn(0x559cab5b8460 msgr2=0x559cab96d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 shutdown_connections 2024-09-21T02:12:33.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.833+0000 7f00ee8fe740 1 -- 172.21.15.92:0/1219251780 wait complete. 2024-09-21T02:12:33.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.834+0000 7f00ee8fe740 20 client.17658 trim_cache size 0 max 16384 2024-09-21T02:12:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.851+0000 7f00ee8fe740 1 Processor -- start 2024-09-21T02:12:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.851+0000 7f00ee8fe740 1 -- start start 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab97aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5a3390 0x559cab974390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 0x559cab9748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b1650 con 0x559cab5c4070 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b3190 con 0x559cab59b1b0 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b1d20 con 0x559cab5a3390 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 0x559cab9748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 0x559cab9748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57856/0 (socket says 172.21.15.92:57856) 2024-09-21T02:12:33.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 -- 172.21.15.92:0/3237792342 learned_addr learned my addr 172.21.15.92:0/3237792342 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e5925640 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5a3390 0x559cab974390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6126640 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab97aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 -- 172.21.15.92:0/3237792342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5a3390 msgr2=0x559cab974390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5a3390 0x559cab974390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 -- 172.21.15.92:0/3237792342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 msgr2=0x559cab97aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab97aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6927640 1 -- 172.21.15.92:0/3237792342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cab9756a0 con 0x559cab5c4070 2024-09-21T02:12:33.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.852+0000 7f00e6126640 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab97aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:33.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.853+0000 7f00e6927640 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 0x559cab9748d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f00d80107e0 tx=0x7f00d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881f2350c814bdf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.853+0000 7f00cbfff640 1 -- 172.21.15.92:0/3237792342 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d801cb30 con 0x559cab5c4070 2024-09-21T02:12:33.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.853+0000 7f00cbfff640 1 -- 172.21.15.92:0/3237792342 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d801ccd0 con 0x559cab5c4070 2024-09-21T02:12:33.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.853+0000 7f00cbfff640 1 -- 172.21.15.92:0/3237792342 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d801a670 con 0x559cab5c4070 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3237792342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 msgr2=0x559cab9748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 0x559cab9748d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f00d80107e0 tx=0x7f00d8012000 comp rx=0 tx=0).stop 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3237792342 shutdown_connections 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5c4070 0x559cab9748d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab5a3390 0x559cab974390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3237792342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab59b1b0 0x559cab97aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3237792342 >> 172.21.15.92:0/3237792342 conn(0x559cab5b8460 msgr2=0x559cab97a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3237792342 shutdown_connections 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3237792342 wait complete. 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 -- start start 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 Processor -- start 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab59b1b0 0x559cab97f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab97f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.854+0000 7f00ee8fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5c4070 0x559cab9ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b2ac0 con 0x559cab5a3390 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01b23f0 con 0x559cab5c4070 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01af630 con 0x559cab59b1b0 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 10 client.0 _ll_register_callbacks cb 0x7f00dc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab97f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab97f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57864/0 (socket says 172.21.15.92:57864) 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 -- 172.21.15.92:0/3196065472 learned_addr learned my addr 172.21.15.92:0/3196065472 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:33.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e6927640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5c4070 0x559cab9ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e6126640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab59b1b0 0x559cab97f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 -- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab59b1b0 msgr2=0x559cab97f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab59b1b0 0x559cab97f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 -- 172.21.15.92:0/3196065472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5c4070 msgr2=0x559cab9ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5c4070 0x559cab9ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 -- 172.21.15.92:0/3196065472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cab9ece70 con 0x559cab5a3390 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e6927640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5c4070 0x559cab9ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00e5925640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab97f900 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0d7250 tx=0x7f00dc0ccc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2a958dc2d8aeb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00c9ffb640 1 -- 172.21.15.92:0/3196065472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc049070 con 0x559cab5a3390 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00c9ffb640 1 -- 172.21.15.92:0/3196065472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f00dc03bc50 con 0x559cab5a3390 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d801ccd0 con 0x559cab5a3390 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00c9ffb640 1 -- 172.21.15.92:0/3196065472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00dc0ccf50 con 0x559cab5a3390 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 10 client.17664 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:33.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.855+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559cab9f4180 con 0x559cab5a3390 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00c9ffb640 1 -- 172.21.15.92:0/3196065472 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d801ccd0 con 0x559cab5a3390 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00c9ffb640 1 -- 172.21.15.92:0/3196065472 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f00dc0cc4b0 con 0x559cab5a3390 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00ee8fe740 10 client.17664 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00ee8fe740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00ee8fe740 20 client.17664 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab557da0 0x559cab9f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00ee8fe740 4 client.17664 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cab8df850 con 0x559cab557da0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.856+0000 7f00e6126640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab557da0 0x559cab9f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00e6126640 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab557da0 0x559cab9f6800 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1cda3230020842 server_cookie=ea3c2a4e7f0e2898 in_seq=0 out_seq=0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00c9ffb640 10 client.17664 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00c9ffb640 1 -- 172.21.15.92:0/3196065472 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x559cab8df850 con 0x559cab557da0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00c9ffb640 10 client.17664 handle_command_reply: tid=0 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 client.17664 shutdown 2024-09-21T02:12:33.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 msgr2=0x559cab97f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab97f900 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0d7250 tx=0x7f00dc0ccc70 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 shutdown_connections 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559cab557da0 0x559cab9f6800 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559cab5c4070 0x559cab9ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559cab5a3390 0x559cab97f900 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 --2- 172.21.15.92:0/3196065472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cab59b1b0 0x559cab97f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00c9ffb640 0 client.17664 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 >> 172.21.15.92:0/3196065472 conn(0x559cab5b8460 msgr2=0x559cab96d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 shutdown_connections 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 1 -- 172.21.15.92:0/3196065472 wait complete. 2024-09-21T02:12:33.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.857+0000 7f00ee8fe740 20 client.17664 trim_cache size 0 max 16384 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:12:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:33.871 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f00cc0b6510 msgr2=0x7f00cc0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f00cc0b6510 0x7f00cc0b8a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f00d8005fd0 tx=0x7f00d80083f0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 msgr2=0x7f00e0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e0199930 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f00dc031a90 tx=0x7f00dc004270 comp rx=0 tx=0).stop 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 shutdown_connections 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f00cc0b6510 0x7f00cc0b8a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0108cf0 0x7f00e019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00e01083a0 0x7f00e0199930 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 --2- 172.21.15.92:0/2918544697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e0102a60 0x7f00e0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 >> 172.21.15.92:0/2918544697 conn(0x7f00e0079730 msgr2=0x7f00e0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 shutdown_connections 2024-09-21T02:12:33.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:33.858+0000 7f00e838a640 1 -- 172.21.15.92:0/2918544697 wait complete. 2024-09-21T02:12:34.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:12:34.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:12:34.909 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:35.140 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:34 smithi092 ceph-mon[31190]: pgmap v588: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:12:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:34 smithi136 ceph-mon[34754]: pgmap v588: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:12:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 -- 172.21.15.92:0/4125117994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 msgr2=0x7f462c102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 --2- 172.21.15.92:0/4125117994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c102a40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f461c009900 tx=0x7f461c02a410 comp rx=0 tx=0).stop 2024-09-21T02:12:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 -- 172.21.15.92:0/4125117994 shutdown_connections 2024-09-21T02:12:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 --2- 172.21.15.92:0/4125117994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c103ff0 0x7f462c10a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 --2- 172.21.15.92:0/4125117994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103650 0x7f462c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 --2- 172.21.15.92:0/4125117994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c102a40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 -- 172.21.15.92:0/4125117994 >> 172.21.15.92:0/4125117994 conn(0x7f462c0fdf30 msgr2=0x7f462c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 -- 172.21.15.92:0/4125117994 shutdown_connections 2024-09-21T02:12:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.303+0000 7f4632430640 1 -- 172.21.15.92:0/4125117994 wait complete. 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 Processor -- start 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 -- start start 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c103650 0x7f462c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103ff0 0x7f462c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10e240 con 0x7f462c1025d0 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10e0c0 con 0x7f462c103ff0 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f4632430640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f462c10e3c0 con 0x7f462c103650 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.304+0000 7f462bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57882/0 (socket says 172.21.15.92:57882) 2024-09-21T02:12:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 -- 172.21.15.92:0/1893297349 learned_addr learned my addr 172.21.15.92:0/1893297349 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462b7fe640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c103650 0x7f462c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f46309cd640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103ff0 0x7f462c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 -- 172.21.15.92:0/1893297349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c103650 msgr2=0x7f462c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c103650 0x7f462c199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 -- 172.21.15.92:0/1893297349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103ff0 msgr2=0x7f462c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103ff0 0x7f462c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 -- 172.21.15.92:0/1893297349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c19a8d0 con 0x7f462c1025d0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f46309cd640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103ff0 0x7f462c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f462bfff640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c198e90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f461c031440 tx=0x7f461c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f46297fa640 1 -- 172.21.15.92:0/1893297349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c03f070 con 0x7f462c1025d0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f46297fa640 1 -- 172.21.15.92:0/1893297349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c031c50 con 0x7f462c1025d0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c19ab60 con 0x7f462c1025d0 2024-09-21T02:12:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f46297fa640 1 -- 172.21.15.92:0/1893297349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c035030 con 0x7f462c1025d0 2024-09-21T02:12:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.305+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c19ae70 con 0x7f462c1025d0 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.308+0000 7f4639479740 1 Processor -- start 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.308+0000 7f4639479740 1 -- start start 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f4639479740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 0x5644bedb6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f4639479740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9e62f0 0x5644bedb74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f4639479740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46297fa640 1 -- 172.21.15.92:0/1893297349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f461c042070 con 0x7f462c1025d0 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f4639479740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644bed296f0 con 0x5644bea0f1b0 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f4639479740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5644bed18b20 con 0x5644be9ee4d0 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f4639479740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644bed1c930 con 0x5644be9e62f0 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 0x5644bedb6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 0x5644bedb6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57886/0 (socket says 172.21.15.92:57886) 2024-09-21T02:12:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 -- 172.21.15.92:0/3677633381 learned_addr learned my addr 172.21.15.92:0/3677633381 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f462bfff640 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9e62f0 0x5644bedb74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 -- 172.21.15.92:0/3677633381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9e62f0 msgr2=0x5644bedb74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9e62f0 0x5644bedb74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 -- 172.21.15.92:0/3677633381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 msgr2=0x5644bedb8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f46309cd640 1 -- 172.21.15.92:0/3677633381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644becde530 con 0x5644bea0f1b0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.309+0000 7f462bfff640 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9e62f0 0x5644bedb74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f46309cd640 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 0x5644bedb6e50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4624009fb0 tx=0x7f4624010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79616fb5f29274c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f46127fc640 1 -- 172.21.15.92:0/3677633381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624016070 con 0x5644bea0f1b0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f46127fc640 1 -- 172.21.15.92:0/3677633381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4624012bc0 con 0x5644bea0f1b0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f46297fa640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46140b6510 0x7f46140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f46127fc640 1 -- 172.21.15.92:0/3677633381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462401bcb0 con 0x5644bea0f1b0 2024-09-21T02:12:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f462bfff640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46140b6510 0x7f46140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.310+0000 7f46297fa640 1 -- 172.21.15.92:0/1893297349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f461c107ca0 con 0x7f462c1025d0 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 -- 172.21.15.92:0/3677633381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 msgr2=0x5644bedb6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f462bfff640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46140b6510 0x7f46140b8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f461c0cb080 tx=0x7f461c10b780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 0x5644bedb6e50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4624009fb0 tx=0x7f4624010800 comp rx=0 tx=0).stop 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 -- 172.21.15.92:0/3677633381 shutdown_connections 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9e62f0 0x5644bedb74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 --2- 172.21.15.92:0/3677633381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644bea0f1b0 0x5644bedb6e50 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 -- 172.21.15.92:0/3677633381 >> 172.21.15.92:0/3677633381 conn(0x5644bea03300 msgr2=0x5644bedb28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 -- 172.21.15.92:0/3677633381 shutdown_connections 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 -- 172.21.15.92:0/3677633381 wait complete. 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.311+0000 7f4639479740 1 -- start start 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 Processor -- start 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9e62f0 0x5644bedc05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 0x5644bedc0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644bea0f1b0 0x5644bedc1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644bec50ab0 con 0x5644be9ee4d0 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5644becbe730 con 0x5644be9e62f0 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644bedb8b30 con 0x5644bea0f1b0 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f4639479740 10 client.0 _ll_register_callbacks cb 0x5644bed126b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 0x5644bedc0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 0x5644bedc0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57896/0 (socket says 172.21.15.92:57896) 2024-09-21T02:12:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f46309cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9e62f0 0x5644bedc05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f462b7fe640 1 -- 172.21.15.92:0/3074411808 learned_addr learned my addr 172.21.15.92:0/3074411808 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.312+0000 7f46309cd640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644bea0f1b0 0x5644bedc1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f462b7fe640 1 -- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644bea0f1b0 msgr2=0x5644bedc1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644bea0f1b0 0x5644bedc1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f462b7fe640 1 -- 172.21.15.92:0/3074411808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9e62f0 msgr2=0x5644bedc05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9e62f0 0x5644bedc05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f462b7fe640 1 -- 172.21.15.92:0/3074411808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644bea7a790 con 0x5644be9ee4d0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f46309cd640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9e62f0 0x5644bedc05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 0x5644bedc0ca0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4620010820 tx=0x7f4620010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343a9b76167f0afb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f460b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f460b7fe640 1 -- 172.21.15.92:0/3074411808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462001e060 con 0x5644be9ee4d0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f460b7fe640 1 -- 172.21.15.92:0/3074411808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f462000fb90 con 0x5644be9ee4d0 2024-09-21T02:12:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4624012bc0 con 0x5644be9ee4d0 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f460b7fe640 1 -- 172.21.15.92:0/3074411808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f462000fd90 con 0x5644be9ee4d0 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f4639479740 10 client.17676 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5644beaf3620 con 0x5644be9ee4d0 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.313+0000 7f460b7fe640 1 -- 172.21.15.92:0/3074411808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4624012bc0 con 0x5644be9ee4d0 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f460b7fe640 1 -- 172.21.15.92:0/3074411808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f462000e460 con 0x5644be9ee4d0 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f4639479740 10 client.17676 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f4639479740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f4639479740 20 client.17676 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f4639479740 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644be9cdd80 0x5644bedcdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f4639479740 4 client.17676 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644bed2b640 con 0x5644be9cdd80 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.314+0000 7f46309cd640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644be9cdd80 0x5644bedcdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f460b7fe640 10 client.17676 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f46309cd640 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644be9cdd80 0x5644bedcdd70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8cf6a53ef617939 server_cookie=ac798f418f74ab8a in_seq=0 out_seq=0 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f460b7fe640 1 -- 172.21.15.92:0/3074411808 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644bed2b640 con 0x5644be9cdd80 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f460b7fe640 10 client.17676 handle_command_reply: tid=0 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 client.17676 shutdown 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 msgr2=0x5644bedc0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 0x5644bedc0ca0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4620010820 tx=0x7f4620010ce0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 shutdown_connections 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644be9cdd80 0x5644bedcdd70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644bea0f1b0 0x5644bedc1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.315+0000 7f4639479740 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9ee4d0 0x5644bedc0ca0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.316+0000 7f460b7fe640 0 client.17676 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.316+0000 7f4639479740 1 --2- 172.21.15.92:0/3074411808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9e62f0 0x5644bedc05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.316+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 >> 172.21.15.92:0/3074411808 conn(0x5644bea03300 msgr2=0x5644bedb0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.316+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 shutdown_connections 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.316+0000 7f4639479740 1 -- 172.21.15.92:0/3074411808 wait complete. 2024-09-21T02:12:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.316+0000 7f4639479740 20 client.17676 trim_cache size 0 max 16384 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 Processor -- start 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 -- start start 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9cdd80 0x5644bedbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedb6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1a9f50 con 0x5644be9e62f0 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1aba90 con 0x5644be9ee4d0 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f4639479740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1aa620 con 0x5644be9cdd80 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedb6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedb6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57912/0 (socket says 172.21.15.92:57912) 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f462b7fe640 1 -- 172.21.15.92:0/3768990148 learned_addr learned my addr 172.21.15.92:0/3768990148 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.330+0000 7f46309cd640 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9cdd80 0x5644bedbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f462b7fe640 1 -- 172.21.15.92:0/3768990148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9cdd80 msgr2=0x5644bedbf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9cdd80 0x5644bedbf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f46309cd640 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f462b7fe640 1 -- 172.21.15.92:0/3768990148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 msgr2=0x5644bedb7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f462b7fe640 1 -- 172.21.15.92:0/3768990148 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644bedc89b0 con 0x5644be9e62f0 2024-09-21T02:12:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f46309cd640 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f462b7fe640 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedb6df0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f462000e850 tx=0x7f4620010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6668f9a954c099 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f4611ffb640 1 -- 172.21.15.92:0/3768990148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462000eac0 con 0x5644be9e62f0 2024-09-21T02:12:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f4611ffb640 1 -- 172.21.15.92:0/3768990148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462000ec60 con 0x5644be9e62f0 2024-09-21T02:12:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.331+0000 7f4611ffb640 1 -- 172.21.15.92:0/3768990148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46200048c0 con 0x5644be9e62f0 2024-09-21T02:12:35.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:34 smithi099 ceph-mon[34404]: pgmap v588: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:12:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 -- 172.21.15.92:0/3768990148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 msgr2=0x5644bedb6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedb6df0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f462000e850 tx=0x7f4620010430 comp rx=0 tx=0).stop 2024-09-21T02:12:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 -- 172.21.15.92:0/3768990148 shutdown_connections 2024-09-21T02:12:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9ee4d0 0x5644bedb7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedb6df0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 --2- 172.21.15.92:0/3768990148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9cdd80 0x5644bedbf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.332+0000 7f4639479740 1 -- 172.21.15.92:0/3768990148 >> 172.21.15.92:0/3768990148 conn(0x5644bea03300 msgr2=0x5644bedbeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.333+0000 7f4639479740 1 -- 172.21.15.92:0/3768990148 shutdown_connections 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.333+0000 7f4639479740 1 -- 172.21.15.92:0/3768990148 wait complete. 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.333+0000 7f4639479740 1 -- start start 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.333+0000 7f4639479740 1 Processor -- start 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9cdd80 0x5644bedbd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedbda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9ee4d0 0x5644bee32800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ab3c0 con 0x5644be9e62f0 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1aacf0 con 0x5644be9cdd80 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1a7f30 con 0x5644be9ee4d0 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 10 client.0 _ll_register_callbacks cb 0x7f462000a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedbda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedbda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57922/0 (socket says 172.21.15.92:57922) 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 -- 172.21.15.92:0/2377317409 learned_addr learned my addr 172.21.15.92:0/2377317409 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f46309cd640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9cdd80 0x5644bedbd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f46309cd640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9ee4d0 0x5644bee32800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 -- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9ee4d0 msgr2=0x5644bee32800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9ee4d0 0x5644bee32800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 -- 172.21.15.92:0/2377317409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9cdd80 msgr2=0x5644bedbd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9cdd80 0x5644bedbd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 -- 172.21.15.92:0/2377317409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644bee32f60 con 0x5644be9e62f0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f46309cd640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9ee4d0 0x5644bee32800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f462b7fe640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedbda80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4620010820 tx=0x7f462000acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaa6c100a901bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f460b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f460b7fe640 1 -- 172.21.15.92:0/2377317409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462001e040 con 0x5644be9e62f0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f460b7fe640 1 -- 172.21.15.92:0/2377317409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4620004220 con 0x5644be9e62f0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.334+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462000ec60 con 0x5644be9e62f0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f460b7fe640 1 -- 172.21.15.92:0/2377317409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f462003d050 con 0x5644be9e62f0 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 10 client.17682 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5644bee3a1c0 con 0x5644be9e62f0 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f460b7fe640 1 -- 172.21.15.92:0/2377317409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462000ec60 con 0x5644be9e62f0 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f460b7fe640 1 -- 172.21.15.92:0/2377317409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4620004660 con 0x5644be9e62f0 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 10 client.17682 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 20 client.17682 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644bea0f1b0 0x5644bee3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 4 client.17682 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644bed270f0 con 0x5644bea0f1b0 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.335+0000 7f46309cd640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644bea0f1b0 0x5644bee3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.336+0000 7f460b7fe640 10 client.17682 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.336+0000 7f46309cd640 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644bea0f1b0 0x5644bee3ca10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e96f169f55565a server_cookie=9c4e4e0c409560fd in_seq=0 out_seq=0 2024-09-21T02:12:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f460b7fe640 1 -- 172.21.15.92:0/2377317409 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644bed270f0 con 0x5644bea0f1b0 2024-09-21T02:12:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f460b7fe640 10 client.17682 handle_command_reply: tid=0 2024-09-21T02:12:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 client.17682 shutdown 2024-09-21T02:12:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 msgr2=0x5644bedbda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedbda80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4620010820 tx=0x7f462000acb0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 shutdown_connections 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5644bea0f1b0 0x5644bee3ca10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644be9ee4d0 0x5644bee32800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644be9e62f0 0x5644bedbda80 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 --2- 172.21.15.92:0/2377317409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be9cdd80 0x5644bedbd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 >> 172.21.15.92:0/2377317409 conn(0x5644bea03300 msgr2=0x5644bedc05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f460b7fe640 0 client.17682 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 shutdown_connections 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 1 -- 172.21.15.92:0/2377317409 wait complete. 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.351+0000 7f4639479740 20 client.17682 trim_cache size 0 max 16384 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:12:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46140b6510 msgr2=0x7f46140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46140b6510 0x7f46140b8a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f461c0cb080 tx=0x7f461c10b780 comp rx=0 tx=0).stop 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 msgr2=0x7f462c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c198e90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f461c031440 tx=0x7f461c003c10 comp rx=0 tx=0).stop 2024-09-21T02:12:35.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 shutdown_connections 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46140b6510 0x7f46140b8a20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f462c103ff0 0x7f462c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c103650 0x7f462c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 --2- 172.21.15.92:0/1893297349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c1025d0 0x7f462c198e90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 >> 172.21.15.92:0/1893297349 conn(0x7f462c0fdf30 msgr2=0x7f462c106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 shutdown_connections 2024-09-21T02:12:35.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:35.352+0000 7f4632430640 1 -- 172.21.15.92:0/1893297349 wait complete. 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 -- 172.21.15.92:0/1831450143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 msgr2=0x7fccb01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/1831450143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb01087e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009900 tx=0x7fcca402a410 comp rx=0 tx=0).stop 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 -- 172.21.15.92:0/1831450143 shutdown_connections 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/1831450143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccb0108d20 0x7fccb010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/1831450143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb01087e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/1831450143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0102ae0 0x7fccb0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 -- 172.21.15.92:0/1831450143 >> 172.21.15.92:0/1831450143 conn(0x7fccb00797b0 msgr2=0x7fccb0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 -- 172.21.15.92:0/1831450143 shutdown_connections 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.351+0000 7fccb5cf0640 1 -- 172.21.15.92:0/1831450143 wait complete. 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 Processor -- start 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 -- start start 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccb0102ae0 0x7fccb019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0108d20 0x7fccb019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01129e0 con 0x7fccb01083d0 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0112860 con 0x7fccb0108d20 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccb5cf0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0112b60 con 0x7fccb0102ae0 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57944/0 (socket says 172.21.15.92:57944) 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccaeffd640 1 -- 172.21.15.92:0/3980950572 learned_addr learned my addr 172.21.15.92:0/3980950572 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.352+0000 7fccaffff640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0108d20 0x7fccb019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaeffd640 1 -- 172.21.15.92:0/3980950572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccb0102ae0 msgr2=0x7fccb019a4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:12:35.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaeffd640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccb0102ae0 0x7fccb019a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaeffd640 1 -- 172.21.15.92:0/3980950572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0108d20 msgr2=0x7fccb019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaeffd640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0108d20 0x7fccb019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaeffd640 1 -- 172.21.15.92:0/3980950572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb01a0430 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaffff640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0108d20 0x7fccb019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccaeffd640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb019aa60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcca402a930 tx=0x7fcca4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca403f030 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca4004490 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb01a06c0 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4035070 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.353+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb00ff550 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.354+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb0103d10 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.356+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fcca4042030 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.357+0000 7fccacff9640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcc980b6510 0x7fcc980b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.357+0000 7fccaf7fe640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcc980b6510 0x7fcc980b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.357+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fcca4107a60 con 0x7fccb01083d0 2024-09-21T02:12:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.357+0000 7fccaf7fe640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcc980b6510 0x7fcc980b8a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009a60 tx=0x7fcca0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.360+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcca40ce2d0 con 0x7fccb01083d0 2024-09-21T02:12:35.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.579+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccb0102ef0 con 0x7fccb01083d0 2024-09-21T02:12:35.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.580+0000 7fccacff9640 1 -- 172.21.15.92:0/3980950572 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7fcca40d26b0 con 0x7fccb01083d0 2024-09-21T02:12:35.581 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:12:35.581 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:12:35.581 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcc980b6510 msgr2=0x7fcc980b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcc980b6510 0x7fcc980b8a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009a60 tx=0x7fcca0005f90 comp rx=0 tx=0).stop 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 msgr2=0x7fccb019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb019aa60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcca402a930 tx=0x7fcca4031a20 comp rx=0 tx=0).stop 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 shutdown_connections 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fcc980b6510 0x7fcc980b8a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb0108d20 0x7fccb019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb01083d0 0x7fccb019aa60 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 --2- 172.21.15.92:0/3980950572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccb0102ae0 0x7fccb019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 >> 172.21.15.92:0/3980950572 conn(0x7fccb00797b0 msgr2=0x7fccb0101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 shutdown_connections 2024-09-21T02:12:35.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:35.582+0000 7fccb5cf0640 1 -- 172.21.15.92:0/3980950572 wait complete. 2024-09-21T02:12:36.183 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:35 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3980950572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:12:36.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:12:36.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:35 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3980950572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:12:36.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:35 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3980950572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:12:36.456 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 -- 172.21.15.92:0/3775819351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158108cf0 msgr2=0x7f915810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 --2- 172.21.15.92:0/3775819351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158108cf0 0x7f915810f520 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f9150009ca0 tx=0x7f915002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 -- 172.21.15.92:0/3775819351 shutdown_connections 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 --2- 172.21.15.92:0/3775819351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158108cf0 0x7f915810f520 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 --2- 172.21.15.92:0/3775819351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91581083a0 0x7f91581087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 --2- 172.21.15.92:0/3775819351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158102a60 0x7f9158102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 -- 172.21.15.92:0/3775819351 >> 172.21.15.92:0/3775819351 conn(0x7f9158079730 msgr2=0x7f9158079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 -- 172.21.15.92:0/3775819351 shutdown_connections 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.927+0000 7f915e615640 1 -- 172.21.15.92:0/3775819351 wait complete. 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 Processor -- start 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 -- start start 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 0x7f91580ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91581083a0 0x7f91580ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158108cf0 0x7f91581000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9158112960 con 0x7f9158102a60 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91581127e0 con 0x7f9158108cf0 2024-09-21T02:12:36.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915e615640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9158112ae0 con 0x7f91581083a0 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 0x7f91580ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 0x7f91580ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57950/0 (socket says 172.21.15.92:57950) 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 -- 172.21.15.92:0/371323765 learned_addr learned my addr 172.21.15.92:0/371323765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f914ffff640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91581083a0 0x7f91580ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915cbb2640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158108cf0 0x7f91581000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 -- 172.21.15.92:0/371323765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91581083a0 msgr2=0x7f91580ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91581083a0 0x7f91580ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 -- 172.21.15.92:0/371323765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158108cf0 msgr2=0x7f91581000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158108cf0 0x7f91581000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f9157fff640 1 -- 172.21.15.92:0/371323765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91581a8760 con 0x7f9158102a60 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.928+0000 7f915cbb2640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158108cf0 0x7f91581000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.929+0000 7f9157fff640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 0x7f91580ff670 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f914800c890 tx=0x7f914800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.929+0000 7f9155ffb640 1 -- 172.21.15.92:0/371323765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148004220 con 0x7f9158102a60 2024-09-21T02:12:36.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.929+0000 7f9155ffb640 1 -- 172.21.15.92:0/371323765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91480043c0 con 0x7f9158102a60 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.929+0000 7f9155ffb640 1 -- 172.21.15.92:0/371323765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148004a00 con 0x7f9158102a60 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.929+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91581a8a50 con 0x7f9158102a60 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.929+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91581a8f90 con 0x7f9158102a60 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.931+0000 7f9164b89740 1 Processor -- start 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.931+0000 7f9155ffb640 1 -- 172.21.15.92:0/371323765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f91480079b0 con 0x7f9158102a60 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.931+0000 7f9164b89740 1 -- start start 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9164b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d82152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9164b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8215800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9164b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8215d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9164b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f1d818bea0 con 0x55f1d7e4d390 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9164b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f1d817b2d0 con 0x55f1d7e6e070 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9164b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1d817f0e0 con 0x55f1d7e451b0 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8215d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8215d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57962/0 (socket says 172.21.15.92:57962) 2024-09-21T02:12:36.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 -- 172.21.15.92:0/2951032171 learned_addr learned my addr 172.21.15.92:0/2951032171 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f915cbb2640 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d82152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9157fff640 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8215800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 -- 172.21.15.92:0/2951032171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 msgr2=0x55f1d8215800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8215800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 -- 172.21.15.92:0/2951032171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 msgr2=0x55f1d82152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d82152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 -- 172.21.15.92:0/2951032171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d7ed9650 con 0x55f1d7e4d390 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9155ffb640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91440b6510 0x7f91440b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f915cbb2640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91440b6510 0x7f91440b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f914ffff640 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8215d40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f914000cac0 tx=0x7f914000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5e4bbc4807310b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.932+0000 7f9155ffb640 1 -- 172.21.15.92:0/371323765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f91480e7110 con 0x7f9158102a60 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.933+0000 7f914e7fc640 1 -- 172.21.15.92:0/2951032171 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140005020 con 0x55f1d7e4d390 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.933+0000 7f914e7fc640 1 -- 172.21.15.92:0/2951032171 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91400047a0 con 0x55f1d7e4d390 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.933+0000 7f914e7fc640 1 -- 172.21.15.92:0/2951032171 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140010690 con 0x55f1d7e4d390 2024-09-21T02:12:36.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 -- 172.21.15.92:0/2951032171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 msgr2=0x55f1d8215d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f915cbb2640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91440b6510 0x7f91440b8a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f91500094e0 tx=0x7f9150009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8215d40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f914000cac0 tx=0x7f914000cf80 comp rx=0 tx=0).stop 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 -- 172.21.15.92:0/2951032171 shutdown_connections 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8215d40 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8215800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 --2- 172.21.15.92:0/2951032171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d82152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 -- 172.21.15.92:0/2951032171 >> 172.21.15.92:0/2951032171 conn(0x55f1d7e62460 msgr2=0x55f1d82131b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 -- 172.21.15.92:0/2951032171 shutdown_connections 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 -- 172.21.15.92:0/2951032171 wait complete. 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 -- start start 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.934+0000 7f9164b89740 1 Processor -- start 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d82208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8220e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d8221380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f1d7fdc700 con 0x55f1d7e4d390 2024-09-21T02:12:36.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f1d8116160 con 0x55f1d7e6e070 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1d8223aa0 con 0x55f1d7e451b0 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9164b89740 10 client.0 _ll_register_callbacks cb 0x55f1d812f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8220e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8220e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57974/0 (socket says 172.21.15.92:57974) 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 -- 172.21.15.92:0/1857901376 learned_addr learned my addr 172.21.15.92:0/1857901376 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f915cbb2640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d8221380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f9157fff640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d82208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 -- 172.21.15.92:0/1857901376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 msgr2=0x55f1d82208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d82208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 -- 172.21.15.92:0/1857901376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 msgr2=0x55f1d8221380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d8221380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.935+0000 7f914ffff640 1 -- 172.21.15.92:0/1857901376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d8143c90 con 0x55f1d7e4d390 2024-09-21T02:12:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.936+0000 7f914ffff640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8220e20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f914000ef50 tx=0x7f91400046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6ca34e8efb704e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.936+0000 7f91377fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.936+0000 7f91377fe640 1 -- 172.21.15.92:0/1857901376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914002a060 con 0x55f1d7e4d390 2024-09-21T02:12:36.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.936+0000 7f91377fe640 1 -- 172.21.15.92:0/1857901376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9140005230 con 0x55f1d7e4d390 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.936+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91400047a0 con 0x55f1d7e4d390 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f91377fe640 1 -- 172.21.15.92:0/1857901376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9140004d10 con 0x55f1d7e4d390 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 10 client.17700 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1d822f750 con 0x55f1d7e4d390 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f91377fe640 1 -- 172.21.15.92:0/1857901376 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91400047a0 con 0x55f1d7e4d390 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f91377fe640 1 -- 172.21.15.92:0/1857901376 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f914001a410 con 0x55f1d7e4d390 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 10 client.17700 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 10 client.17700 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 20 client.17700 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e2c9f0 0x55f1d8232ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 4 client.17700 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d818ddf0 con 0x55f1d7e2c9f0 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.937+0000 7f9157fff640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e2c9f0 0x55f1d8232ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.938+0000 7f91377fe640 10 client.17700 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.938+0000 7f9157fff640 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e2c9f0 0x55f1d8232ab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8519ca41abf64f81 server_cookie=53a4795557a3715 in_seq=0 out_seq=0 2024-09-21T02:12:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.938+0000 7f91377fe640 1 -- 172.21.15.92:0/1857901376 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1d818ddf0 con 0x55f1d7e2c9f0 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.938+0000 7f91377fe640 10 client.17700 handle_command_reply: tid=0 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.938+0000 7f9164b89740 1 client.17700 shutdown 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 msgr2=0x55f1d8220e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8220e20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f914000ef50 tx=0x7f91400046a0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 shutdown_connections 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e2c9f0 0x55f1d8232ab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e6e070 0x55f1d8221380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e4d390 0x55f1d8220e20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f91377fe640 0 client.17700 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 --2- 172.21.15.92:0/1857901376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d82208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 >> 172.21.15.92:0/1857901376 conn(0x55f1d7e62460 msgr2=0x55f1d8218450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 shutdown_connections 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 1 -- 172.21.15.92:0/1857901376 wait complete. 2024-09-21T02:12:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.939+0000 7f9164b89740 20 client.17700 trim_cache size 0 max 16384 2024-09-21T02:12:36.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 Processor -- start 2024-09-21T02:12:36.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 -- start start 2024-09-21T02:12:36.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8216cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8222510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d8212760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91581b1740 con 0x55f1d7e6e070 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91581b3280 con 0x55f1d7e4d390 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9164b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91581b1e10 con 0x55f1d7e451b0 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f915cbb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d8212760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f915cbb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d8212760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57988/0 (socket says 172.21.15.92:57988) 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f915cbb2640 1 -- 172.21.15.92:0/3333452136 learned_addr learned my addr 172.21.15.92:0/3333452136 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f9157fff640 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8216cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.952+0000 7f914ffff640 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8222510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f915cbb2640 1 -- 172.21.15.92:0/3333452136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 msgr2=0x55f1d8216cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f915cbb2640 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8216cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f915cbb2640 1 -- 172.21.15.92:0/3333452136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 msgr2=0x55f1d8222510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f915cbb2640 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8222510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f915cbb2640 1 -- 172.21.15.92:0/3333452136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d8223410 con 0x55f1d7e6e070 2024-09-21T02:12:36.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f914ffff640 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8222510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:36.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f915cbb2640 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d8212760 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9150032ec0 tx=0x7f9150040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e31512b76cb19b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f914dffb640 1 -- 172.21.15.92:0/3333452136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500318e0 con 0x55f1d7e6e070 2024-09-21T02:12:36.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f914dffb640 1 -- 172.21.15.92:0/3333452136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9150036050 con 0x55f1d7e6e070 2024-09-21T02:12:36.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.953+0000 7f914dffb640 1 -- 172.21.15.92:0/3333452136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150030890 con 0x55f1d7e6e070 2024-09-21T02:12:36.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 -- 172.21.15.92:0/3333452136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 msgr2=0x55f1d8212760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d8212760 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9150032ec0 tx=0x7f9150040000 comp rx=0 tx=0).stop 2024-09-21T02:12:36.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 -- 172.21.15.92:0/3333452136 shutdown_connections 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d8212760 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8222510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 --2- 172.21.15.92:0/3333452136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8216cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 -- 172.21.15.92:0/3333452136 >> 172.21.15.92:0/3333452136 conn(0x55f1d7e62460 msgr2=0x55f1d8221e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 -- 172.21.15.92:0/3333452136 shutdown_connections 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 -- 172.21.15.92:0/3333452136 wait complete. 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.954+0000 7f9164b89740 1 -- start start 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 Processor -- start 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8224d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8225280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d82257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91581b2bb0 con 0x55f1d7e6e070 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91581b24e0 con 0x55f1d7e4d390 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91581af720 con 0x55f1d7e451b0 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9164b89740 10 client.0 _ll_register_callbacks cb 0x7f9140014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d82257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d82257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57996/0 (socket says 172.21.15.92:57996) 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 -- 172.21.15.92:0/4101526031 learned_addr learned my addr 172.21.15.92:0/4101526031 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f914ffff640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8225280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f9157fff640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8224d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 -- 172.21.15.92:0/4101526031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 msgr2=0x55f1d8224d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8224d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 -- 172.21.15.92:0/4101526031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 msgr2=0x55f1d8225280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8225280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.955+0000 7f915cbb2640 1 -- 172.21.15.92:0/4101526031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d822ccc0 con 0x55f1d7e6e070 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f915cbb2640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d82257c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f915002a900 tx=0x7f9150030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa82a3e1a3a27fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f91377fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f91377fe640 1 -- 172.21.15.92:0/4101526031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500318e0 con 0x55f1d7e6e070 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f91377fe640 1 -- 172.21.15.92:0/4101526031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f915002f870 con 0x55f1d7e6e070 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9150036050 con 0x55f1d7e6e070 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f91377fe640 1 -- 172.21.15.92:0/4101526031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9150062070 con 0x55f1d7e6e070 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 10 client.17706 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:36.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1d829d2c0 con 0x55f1d7e6e070 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f91377fe640 1 -- 172.21.15.92:0/4101526031 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9150036050 con 0x55f1d7e6e070 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f91377fe640 1 -- 172.21.15.92:0/4101526031 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f91500044b0 con 0x55f1d7e6e070 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 10 client.17706 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 10 client.17706 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 20 client.17706 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e01da0 0x55f1d829f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 4 client.17706 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.956+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1d81898a0 con 0x55f1d7e01da0 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9157fff640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e01da0 0x55f1d829f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f91377fe640 10 client.17706 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9157fff640 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e01da0 0x55f1d829f9d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82a826c8d0762105 server_cookie=5650d5c850c4c498 in_seq=0 out_seq=0 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f91377fe640 1 -- 172.21.15.92:0/4101526031 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1d81898a0 con 0x55f1d7e01da0 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f91377fe640 10 client.17706 handle_command_reply: tid=0 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 client.17706 shutdown 2024-09-21T02:12:36.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 msgr2=0x55f1d82257c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d82257c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f915002a900 tx=0x7f9150030d80 comp rx=0 tx=0).stop 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 shutdown_connections 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55f1d7e01da0 0x55f1d829f9d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1d7e6e070 0x55f1d82257c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1d7e4d390 0x55f1d8225280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 --2- 172.21.15.92:0/4101526031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d7e451b0 0x55f1d8224d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 >> 172.21.15.92:0/4101526031 conn(0x55f1d7e62460 msgr2=0x55f1d8216a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.957+0000 7f91377fe640 0 client.17706 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 shutdown_connections 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f9164b89740 1 -- 172.21.15.92:0/4101526031 wait complete. 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f9164b89740 20 client.17706 trim_cache size 0 max 16384 2024-09-21T02:12:36.963 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91440b6510 msgr2=0x7f91440b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91440b6510 0x7f91440b8a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f91500094e0 tx=0x7f9150009360 comp rx=0 tx=0).stop 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 msgr2=0x7f91580ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 0x7f91580ff670 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f914800c890 tx=0x7f914800cd50 comp rx=0 tx=0).stop 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 shutdown_connections 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91440b6510 0x7f91440b8a20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158108cf0 0x7f91581000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91581083a0 0x7f91580ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 --2- 172.21.15.92:0/371323765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9158102a60 0x7f91580ff670 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:36.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 >> 172.21.15.92:0/371323765 conn(0x7f9158079730 msgr2=0x7f915806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:36.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 shutdown_connections 2024-09-21T02:12:36.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:36.958+0000 7f915e615640 1 -- 172.21.15.92:0/371323765 wait complete. 2024-09-21T02:12:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:36 smithi136 ceph-mon[34754]: pgmap v589: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:12:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:36 smithi092 ceph-mon[31190]: pgmap v589: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:12:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:36 smithi099 ceph-mon[34404]: pgmap v589: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:12:37.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:12:37.971 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.444+0000 7f95de015640 1 -- 172.21.15.92:0/2086317423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 msgr2=0x7f95d81026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.444+0000 7f95de015640 1 --2- 172.21.15.92:0/2086317423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d81026b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009900 tx=0x7f95c802a410 comp rx=0 tx=0).stop 2024-09-21T02:12:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.444+0000 7f95de015640 1 -- 172.21.15.92:0/2086317423 shutdown_connections 2024-09-21T02:12:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.444+0000 7f95de015640 1 --2- 172.21.15.92:0/2086317423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95d8108530 0x7f95d810ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 --2- 172.21.15.92:0/2086317423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d8107be0 0x7f95d8107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 --2- 172.21.15.92:0/2086317423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d81026b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 -- 172.21.15.92:0/2086317423 >> 172.21.15.92:0/2086317423 conn(0x7f95d8079ec0 msgr2=0x7f95d807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 -- 172.21.15.92:0/2086317423 shutdown_connections 2024-09-21T02:12:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 -- 172.21.15.92:0/2086317423 wait complete. 2024-09-21T02:12:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 Processor -- start 2024-09-21T02:12:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.445+0000 7f95de015640 1 -- start start 2024-09-21T02:12:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95de015640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d8110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95de015640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d8107be0 0x7f95d8110b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95de015640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95d8108530 0x7f95d81110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95de015640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8113bc0 con 0x7f95d81022a0 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95de015640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8113a40 con 0x7f95d8107be0 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95de015640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8113d40 con 0x7f95d8108530 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d8110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d8110650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58004/0 (socket says 172.21.15.92:58004) 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 -- 172.21.15.92:0/2733631995 learned_addr learned my addr 172.21.15.92:0/2733631995 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95d8108530 0x7f95d81110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95ceffd640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d8107be0 0x7f95d8110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 -- 172.21.15.92:0/2733631995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95d8108530 msgr2=0x7f95d81110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95d8108530 0x7f95d81110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 -- 172.21.15.92:0/2733631995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d8107be0 msgr2=0x7f95d8110b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d8107be0 0x7f95d8110b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 -- 172.21.15.92:0/2733631995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81a88b0 con 0x7f95d81022a0 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.446+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d8110650 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f95c8005940 tx=0x7f95c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.447+0000 7f95d57fa640 1 -- 172.21.15.92:0/2733631995 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c803f030 con 0x7f95d81022a0 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.447+0000 7f95d57fa640 1 -- 172.21.15.92:0/2733631995 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c8005d10 con 0x7f95d81022a0 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.447+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81a8b40 con 0x7f95d81022a0 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.447+0000 7f95d57fa640 1 -- 172.21.15.92:0/2733631995 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8035070 con 0x7f95d81022a0 2024-09-21T02:12:38.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.447+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a9020 con 0x7f95d81022a0 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.449+0000 7f95e4589740 1 Processor -- start 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.449+0000 7f95d57fa640 1 -- 172.21.15.92:0/2733631995 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f95c8042030 con 0x7f95d81022a0 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.449+0000 7f95e4589740 1 -- start start 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.449+0000 7f95e4589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d37c070 0x55d96d7232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95e4589740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d723800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95e4589740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d723d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95e4589740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d96d699ea0 con 0x55d96d3531b0 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95e4589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d96d6892d0 con 0x55d96d37c070 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95e4589740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d96d68d0e0 con 0x55d96d35b390 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d723800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d723800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58010/0 (socket says 172.21.15.92:58010) 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 -- 172.21.15.92:0/162671052 learned_addr learned my addr 172.21.15.92:0/162671052 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d7fff640 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d37c070 0x55d96d7232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95ceffd640 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d723d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 -- 172.21.15.92:0/162671052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 msgr2=0x55d96d723d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d723d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 -- 172.21.15.92:0/162671052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d37c070 msgr2=0x55d96d7232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d37c070 0x55d96d7232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 -- 172.21.15.92:0/162671052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96d3e7650 con 0x55d96d3531b0 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95ceffd640 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d723d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d57fa640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95c40b6510 0x7f95c40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.454 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95c40b6510 0x7f95c40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.450+0000 7f95d77fe640 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d723800 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f95c810dc20 tx=0x7f95c810dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe9c71f1a7c863f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.451+0000 7f95d57fa640 1 -- 172.21.15.92:0/2733631995 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f95c8107880 con 0x7f95d81022a0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.451+0000 7f95cdffb640 1 -- 172.21.15.92:0/162671052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80c9fe0 con 0x55d96d3531b0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.451+0000 7f95cdffb640 1 -- 172.21.15.92:0/162671052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c80c90f0 con 0x55d96d3531b0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.451+0000 7f95cdffb640 1 -- 172.21.15.92:0/162671052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80cd100 con 0x55d96d3531b0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 -- 172.21.15.92:0/162671052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 msgr2=0x55d96d723800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95c40b6510 0x7f95c40b8a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f95d0005fd0 tx=0x7f95d0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d723800 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f95c810dc20 tx=0x7f95c810dd10 comp rx=0 tx=0).stop 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 -- 172.21.15.92:0/162671052 shutdown_connections 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d723d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d723800 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 --2- 172.21.15.92:0/162671052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d37c070 0x55d96d7232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 -- 172.21.15.92:0/162671052 >> 172.21.15.92:0/162671052 conn(0x55d96d370460 msgr2=0x55d96d7211b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:38.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 -- 172.21.15.92:0/162671052 shutdown_connections 2024-09-21T02:12:38.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 -- 172.21.15.92:0/162671052 wait complete. 2024-09-21T02:12:38.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 -- start start 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.452+0000 7f95e4589740 1 Processor -- start 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d72ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d72ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 0x55d96d73aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d96d4ea700 con 0x55d96d37c070 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d96d624160 con 0x55d96d3531b0 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d96d731860 con 0x55d96d35b390 2024-09-21T02:12:38.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95e4589740 10 client.0 _ll_register_callbacks cb 0x55d96d63d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 0x55d96d73aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 0x55d96d73aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58020/0 (socket says 172.21.15.92:58020) 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 -- 172.21.15.92:0/2463280482 learned_addr learned my addr 172.21.15.92:0/2463280482 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d72ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95ceffd640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d72ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 -- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 msgr2=0x55d96d72ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d72ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 -- 172.21.15.92:0/2463280482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 msgr2=0x55d96d72ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d72ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.453+0000 7f95d7fff640 1 -- 172.21.15.92:0/2463280482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96d651c90 con 0x55d96d37c070 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 0x55d96d73aac0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f95d0011d30 tx=0x7f95d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ac5f1e1a802d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:38.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2463280482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0020620 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2463280482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f95d00207c0 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c80c90f0 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2463280482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95d00209c0 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 10 client.17718 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d96d73d5c0 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2463280482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c80c90f0 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2463280482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95d000f040 con 0x55d96d37c070 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 10 client.17718 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 10 client.17718 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 20 client.17718 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.454+0000 7f95e4589740 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d33a9f0 0x55d96d740b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.455+0000 7f95e4589740 4 client.17718 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.455+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96d69bdf0 con 0x55d96d33a9f0 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.455+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d33a9f0 0x55d96d740b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.455+0000 7f95b6ffd640 10 client.17718 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.455+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d33a9f0 0x55d96d740b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=311feed68a738331 server_cookie=51741a6075105b8b in_seq=0 out_seq=0 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2463280482 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d96d69bdf0 con 0x55d96d33a9f0 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95b6ffd640 10 client.17718 handle_command_reply: tid=0 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 client.17718 shutdown 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 msgr2=0x55d96d73aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 0x55d96d73aac0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f95d0011d30 tx=0x7f95d0004a40 comp rx=0 tx=0).stop 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 shutdown_connections 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d37c070 0x55d96d73aac0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d35b390 0x55d96d72ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d33a9f0 0x55d96d740b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 --2- 172.21.15.92:0/2463280482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d72ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 >> 172.21.15.92:0/2463280482 conn(0x55d96d370460 msgr2=0x55d96d726450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95b6ffd640 0 client.17718 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 shutdown_connections 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 1 -- 172.21.15.92:0/2463280482 wait complete. 2024-09-21T02:12:38.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.456+0000 7f95e4589740 20 client.17718 trim_cache size 0 max 16384 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 Processor -- start 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 -- start start 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d724a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 0x55d96d73a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d72bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81b1760 con 0x55d96d35b390 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81b32a0 con 0x55d96d3531b0 2024-09-21T02:12:38.472 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.469+0000 7f95e4589740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81b1e30 con 0x55d96d37c070 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 0x55d96d73a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 0x55d96d73a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58034/0 (socket says 172.21.15.92:58034) 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 -- 172.21.15.92:0/1739966778 learned_addr learned my addr 172.21.15.92:0/1739966778 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95d77fe640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d724a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95d7fff640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d72bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 -- 172.21.15.92:0/1739966778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 msgr2=0x55d96d72bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d72bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 -- 172.21.15.92:0/1739966778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 msgr2=0x55d96d724a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d724a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 -- 172.21.15.92:0/1739966778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96d72fd10 con 0x55d96d35b390 2024-09-21T02:12:38.473 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95d7fff640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d72bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95d77fe640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d724a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95ceffd640 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 0x55d96d73a9c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f95c000bad0 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f540e4b71a4c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95cd7fa640 1 -- 172.21.15.92:0/1739966778 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c000ede0 con 0x55d96d35b390 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95cd7fa640 1 -- 172.21.15.92:0/1739966778 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c001d070 con 0x55d96d35b390 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.470+0000 7f95cd7fa640 1 -- 172.21.15.92:0/1739966778 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0017720 con 0x55d96d35b390 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 -- 172.21.15.92:0/1739966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 msgr2=0x55d96d73a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 0x55d96d73a9c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f95c000bad0 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 -- 172.21.15.92:0/1739966778 shutdown_connections 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d72bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d35b390 0x55d96d73a9c0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 --2- 172.21.15.92:0/1739966778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d3531b0 0x55d96d724a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 -- 172.21.15.92:0/1739966778 >> 172.21.15.92:0/1739966778 conn(0x55d96d370460 msgr2=0x55d96d7242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 -- 172.21.15.92:0/1739966778 shutdown_connections 2024-09-21T02:12:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 -- 172.21.15.92:0/1739966778 wait complete. 2024-09-21T02:12:38.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.471+0000 7f95e4589740 1 -- start start 2024-09-21T02:12:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 Processor -- start 2024-09-21T02:12:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d733370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d35b390 0x55d96d733950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d7a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81b2bd0 con 0x55d96d3531b0 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81b2500 con 0x55d96d35b390 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81af740 con 0x55d96d37c070 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95e4589740 10 client.0 _ll_register_callbacks cb 0x7f95d00119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d733370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d733370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58050/0 (socket says 172.21.15.92:58050) 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 -- 172.21.15.92:0/2932844016 learned_addr learned my addr 172.21.15.92:0/2932844016 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95ceffd640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d35b390 0x55d96d733950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d7a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 -- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 msgr2=0x55d96d7a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.479 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d7a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 -- 172.21.15.92:0/2932844016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d35b390 msgr2=0x55d96d733950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d35b390 0x55d96d733950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d77fe640 1 -- 172.21.15.92:0/2932844016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96d7a4b90 con 0x55d96d3531b0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95d7fff640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d7a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.472+0000 7f95ceffd640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d35b390 0x55d96d733950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95d77fe640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d733370 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f95c8006010 tx=0x7f95c803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4824b90f25517bef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2932844016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8049040 con 0x55d96d3531b0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2932844016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f95c810d220 con 0x55d96d3531b0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c001d070 con 0x55d96d3531b0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2932844016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95c80e8050 con 0x55d96d3531b0 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 10 client.17724 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:38.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d96d7abe70 con 0x55d96d3531b0 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2932844016 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c001d070 con 0x55d96d3531b0 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2932844016 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95c80c9c00 con 0x55d96d3531b0 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 10 client.17724 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 10 client.17724 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 20 client.17724 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d30fda0 0x55d96d7ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 4 client.17724 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.473+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d96d6978a0 con 0x55d96d30fda0 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.474+0000 7f95ceffd640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d30fda0 0x55d96d7ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.474+0000 7f95b6ffd640 10 client.17724 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.474+0000 7f95ceffd640 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d30fda0 0x55d96d7ae4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6687584ae58d656 server_cookie=fe11a0baf65bfdcd in_seq=0 out_seq=0 2024-09-21T02:12:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.474+0000 7f95b6ffd640 1 -- 172.21.15.92:0/2932844016 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d96d6978a0 con 0x55d96d30fda0 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.474+0000 7f95b6ffd640 10 client.17724 handle_command_reply: tid=0 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.474+0000 7f95e4589740 1 client.17724 shutdown 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 msgr2=0x55d96d733370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d733370 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f95c8006010 tx=0x7f95c803b460 comp rx=0 tx=0).stop 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 shutdown_connections 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d96d37c070 0x55d96d7a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55d96d30fda0 0x55d96d7ae4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d96d35b390 0x55d96d733950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 --2- 172.21.15.92:0/2932844016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d96d3531b0 0x55d96d733370 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95b6ffd640 0 client.17724 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 >> 172.21.15.92:0/2932844016 conn(0x55d96d370460 msgr2=0x55d96d7242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 shutdown_connections 2024-09-21T02:12:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 1 -- 172.21.15.92:0/2932844016 wait complete. 2024-09-21T02:12:38.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95e4589740 20 client.17724 trim_cache size 0 max 16384 2024-09-21T02:12:38.483 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95c40b6510 msgr2=0x7f95c40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95de015640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95c40b6510 0x7f95c40b8a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f95d0005fd0 tx=0x7f95d0009600 comp rx=0 tx=0).stop 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 msgr2=0x7f95d8110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.475+0000 7f95de015640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d8110650 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f95c8005940 tx=0x7f95c80042f0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 shutdown_connections 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f95c40b6510 0x7f95c40b8a20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95d8108530 0x7f95d81110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d8107be0 0x7f95d8110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 --2- 172.21.15.92:0/2733631995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d81022a0 0x7f95d8110650 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:38.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 >> 172.21.15.92:0/2733631995 conn(0x7f95d8079ec0 msgr2=0x7f95d8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 shutdown_connections 2024-09-21T02:12:38.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:38.476+0000 7f95de015640 1 -- 172.21.15.92:0/2733631995 wait complete. 2024-09-21T02:12:39.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:38 smithi092 ceph-mon[31190]: pgmap v590: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:12:39.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:12:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:38 smithi136 ceph-mon[34754]: pgmap v590: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:12:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:38 smithi099 ceph-mon[34404]: pgmap v590: 97 pgs: 97 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:12:39.365 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 -- 172.21.15.92:0/1821695600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c108d00 msgr2=0x7f863c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 --2- 172.21.15.92:0/1821695600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c108d00 0x7f863c10f530 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8634009ca0 tx=0x7f863402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 -- 172.21.15.92:0/1821695600 shutdown_connections 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 --2- 172.21.15.92:0/1821695600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c108d00 0x7f863c10f530 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 --2- 172.21.15.92:0/1821695600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f863c1083b0 0x7f863c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 --2- 172.21.15.92:0/1821695600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f863c102ac0 0x7f863c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 -- 172.21.15.92:0/1821695600 >> 172.21.15.92:0/1821695600 conn(0x7f863c079790 msgr2=0x7f863c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 -- 172.21.15.92:0/1821695600 shutdown_connections 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.861+0000 7f8641017640 1 -- 172.21.15.92:0/1821695600 wait complete. 2024-09-21T02:12:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 Processor -- start 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 -- start start 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 0x7f863c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f863c1083b0 0x7f863c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f863c108d00 0x7f863c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1129c0 con 0x7f863c102ac0 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f863c112840 con 0x7f863c1083b0 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f8641017640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f863c112b40 con 0x7f863c108d00 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f863ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 0x7f863c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f863ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 0x7f863c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58070/0 (socket says 172.21.15.92:58070) 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.862+0000 7f863ad9d640 1 -- 172.21.15.92:0/2633664284 learned_addr learned my addr 172.21.15.92:0/2633664284 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863a59c640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f863c1083b0 0x7f863c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863b59e640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f863c108d00 0x7f863c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863ad9d640 1 -- 172.21.15.92:0/2633664284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f863c108d00 msgr2=0x7f863c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863ad9d640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f863c108d00 0x7f863c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863ad9d640 1 -- 172.21.15.92:0/2633664284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f863c1083b0 msgr2=0x7f863c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863ad9d640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f863c1083b0 0x7f863c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863ad9d640 1 -- 172.21.15.92:0/2633664284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c1a0450 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f863ad9d640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 0x7f863c19a4e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f862c00e9f0 tx=0x7f862c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f8623fff640 1 -- 172.21.15.92:0/2633664284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c00cd80 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f8623fff640 1 -- 172.21.15.92:0/2633664284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862c0108e0 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c1a0740 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f8623fff640 1 -- 172.21.15.92:0/2633664284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c018980 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.863+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1a6ea0 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 Processor -- start 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f8623fff640 1 -- 172.21.15.92:0/2633664284 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f862c018b20 con 0x7f863c102ac0 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 -- start start 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d59242ea0 con 0x560d58efc1b0 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d592322d0 con 0x560d58f25070 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f864758b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d592360e0 con 0x560d58f04390 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f863ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f863ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58072/0 (socket says 172.21.15.92:58072) 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f863ad9d640 1 -- 172.21.15.92:0/3853206067 learned_addr learned my addr 172.21.15.92:0/3853206067 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f863a59c640 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.866+0000 7f863b59e640 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863ad9d640 1 -- 172.21.15.92:0/3853206067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 msgr2=0x560d592ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863ad9d640 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863ad9d640 1 -- 172.21.15.92:0/3853206067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 msgr2=0x560d592cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863ad9d640 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863ad9d640 1 -- 172.21.15.92:0/3853206067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d58f90650 con 0x560d58efc1b0 2024-09-21T02:12:39.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863b59e640 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863ad9d640 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592cc800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f862c0edc10 tx=0x7f862c0ef690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f649dc50fd20d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f8623fff640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86240b6510 0x7f86240b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f863b59e640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86240b6510 0x7f86240b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f86217fa640 1 -- 172.21.15.92:0/3853206067 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c0f6070 con 0x560d58efc1b0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f86217fa640 1 -- 172.21.15.92:0/3853206067 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862c097c60 con 0x560d58efc1b0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f86217fa640 1 -- 172.21.15.92:0/3853206067 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c0ad640 con 0x560d58efc1b0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.867+0000 7f8623fff640 1 -- 172.21.15.92:0/2633664284 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f862c0e7650 con 0x7f863c102ac0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 -- 172.21.15.92:0/3853206067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 msgr2=0x560d592cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f863b59e640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86240b6510 0x7f86240b8a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f863402f500 tx=0x7f863402f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592cc800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f862c0edc10 tx=0x7f862c0ef690 comp rx=0 tx=0).stop 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 -- 172.21.15.92:0/3853206067 shutdown_connections 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592cc800 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 --2- 172.21.15.92:0/3853206067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 -- 172.21.15.92:0/3853206067 >> 172.21.15.92:0/3853206067 conn(0x560d58f19460 msgr2=0x560d592ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 -- 172.21.15.92:0/3853206067 shutdown_connections 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.868+0000 7f864758b740 1 -- 172.21.15.92:0/3853206067 wait complete. 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 -- start start 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 Processor -- start 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592d78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d59093700 con 0x560d58efc1b0 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d591cd160 con 0x560d58f25070 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d592daaa0 con 0x560d58f04390 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f864758b740 10 client.0 _ll_register_callbacks cb 0x560d591e69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f863ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592d78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f863ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592d78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58080/0 (socket says 172.21.15.92:58080) 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f863ad9d640 1 -- 172.21.15.92:0/1363566674 learned_addr learned my addr 172.21.15.92:0/1363566674 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:39.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f863a59c640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.869+0000 7f863b59e640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f863ad9d640 1 -- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 msgr2=0x560d592d7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f863ad9d640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f863ad9d640 1 -- 172.21.15.92:0/1363566674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 msgr2=0x560d592d8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f863ad9d640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592d8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f863ad9d640 1 -- 172.21.15.92:0/1363566674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d591fac90 con 0x560d58efc1b0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f863ad9d640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592d78d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ad570 tx=0x7f862c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ace46e96deaab34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f861eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f861eff5640 1 -- 172.21.15.92:0/1363566674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c01b310 con 0x560d58efc1b0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f861eff5640 1 -- 172.21.15.92:0/1363566674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f862c005470 con 0x560d58efc1b0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862c097c60 con 0x560d58efc1b0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f861eff5640 1 -- 172.21.15.92:0/1363566674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f862c01b500 con 0x560d58efc1b0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f864758b740 10 client.17736 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d592e6610 con 0x560d58efc1b0 2024-09-21T02:12:39.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f861eff5640 1 -- 172.21.15.92:0/1363566674 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862c097c60 con 0x560d58efc1b0 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.870+0000 7f861eff5640 1 -- 172.21.15.92:0/1363566674 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f862c01bc80 con 0x560d58efc1b0 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f864758b740 10 client.17736 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f864758b740 10 client.17736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f864758b740 20 client.17736 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f864758b740 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58ee39f0 0x560d592e9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f864758b740 4 client.17736 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d59244df0 con 0x560d58ee39f0 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f863a59c640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58ee39f0 0x560d592e9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f861eff5640 10 client.17736 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:12:39.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.871+0000 7f863a59c640 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58ee39f0 0x560d592e9970 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebba8c3b19b27dfd server_cookie=ecacf8976d392733 in_seq=0 out_seq=0 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f861eff5640 1 -- 172.21.15.92:0/1363566674 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d59244df0 con 0x560d58ee39f0 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f861eff5640 10 client.17736 handle_command_reply: tid=0 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 client.17736 shutdown 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 msgr2=0x560d592d78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592d78d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ad570 tx=0x7f862c000bc0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 shutdown_connections 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58ee39f0 0x560d592e9970 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592d8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 --2- 172.21.15.92:0/1363566674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58efc1b0 0x560d592d78d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 >> 172.21.15.92:0/1363566674 conn(0x560d58f19460 msgr2=0x560d592cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.872+0000 7f861eff5640 0 client.17736 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.873+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 shutdown_connections 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.873+0000 7f864758b740 1 -- 172.21.15.92:0/1363566674 wait complete. 2024-09-21T02:12:39.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.873+0000 7f864758b740 20 client.17736 trim_cache size 0 max 16384 2024-09-21T02:12:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 Processor -- start 2024-09-21T02:12:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 -- start start 2024-09-21T02:12:39.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58efc1b0 0x560d592d9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 0x560d592d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1af580 con 0x560d58f25070 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b10c0 con 0x560d58efc1b0 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f864758b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1afc50 con 0x560d58f04390 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 0x560d592d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 0x560d592d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58082/0 (socket says 172.21.15.92:58082) 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 -- 172.21.15.92:0/519115388 learned_addr learned my addr 172.21.15.92:0/519115388 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863a59c640 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863ad9d640 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58efc1b0 0x560d592d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 -- 172.21.15.92:0/519115388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 msgr2=0x560d592d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 -- 172.21.15.92:0/519115388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58efc1b0 msgr2=0x560d592d9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58efc1b0 0x560d592d9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.886+0000 7f863b59e640 1 -- 172.21.15.92:0/519115388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d592cf120 con 0x560d58f25070 2024-09-21T02:12:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.887+0000 7f863b59e640 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 0x560d592d5000 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8634031d90 tx=0x7f8634040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba2c2ade6ac005b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.887+0000 7f8620ff9640 1 -- 172.21.15.92:0/519115388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634031e80 con 0x560d58f25070 2024-09-21T02:12:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.887+0000 7f8620ff9640 1 -- 172.21.15.92:0/519115388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8634036040 con 0x560d58f25070 2024-09-21T02:12:39.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.887+0000 7f8620ff9640 1 -- 172.21.15.92:0/519115388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634004850 con 0x560d58f25070 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 -- 172.21.15.92:0/519115388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 msgr2=0x560d592d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 0x560d592d5000 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8634031d90 tx=0x7f8634040000 comp rx=0 tx=0).stop 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 -- 172.21.15.92:0/519115388 shutdown_connections 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f25070 0x560d592d5000 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58f04390 0x560d592d4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 --2- 172.21.15.92:0/519115388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58efc1b0 0x560d592d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 -- 172.21.15.92:0/519115388 >> 172.21.15.92:0/519115388 conn(0x560d58f19460 msgr2=0x560d592d8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 -- 172.21.15.92:0/519115388 shutdown_connections 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 -- 172.21.15.92:0/519115388 wait complete. 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 -- start start 2024-09-21T02:12:39.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.888+0000 7f864758b740 1 Processor -- start 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58efc1b0 0x560d592ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 0x560d592ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b09f0 con 0x560d58f04390 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b0320 con 0x560d58f25070 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ad560 con 0x560d58efc1b0 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 10 client.0 _ll_register_callbacks cb 0x7f862c0eda90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 0x560d592ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 0x560d592ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58096/0 (socket says 172.21.15.92:58096) 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 -- 172.21.15.92:0/1014702678 learned_addr learned my addr 172.21.15.92:0/1014702678 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863ad9d640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58efc1b0 0x560d592ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863b59e640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 -- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58efc1b0 msgr2=0x560d592ed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58efc1b0 0x560d592ed0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 -- 172.21.15.92:0/1014702678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 msgr2=0x560d592eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 -- 172.21.15.92:0/1014702678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d592eb1b0 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f863a59c640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 0x560d592ea580 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f863c06a890 tx=0x7f86300044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dded3dc70d34664d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f861eff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f861eff5640 1 -- 172.21.15.92:0/1014702678 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630029060 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f861eff5640 1 -- 172.21.15.92:0/1014702678 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8630003e00 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.889+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8634036040 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f861eff5640 1 -- 172.21.15.92:0/1014702678 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8630024430 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 10 client.17742 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d59353fb0 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f861eff5640 1 -- 172.21.15.92:0/1014702678 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8634036040 con 0x560d58f04390 2024-09-21T02:12:39.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f861eff5640 1 -- 172.21.15.92:0/1014702678 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8630023070 con 0x560d58f04390 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 10 client.17742 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 10 client.17742 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 20 client.17742 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58eb8da0 0x560d59356780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 4 client.17742 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d592408a0 con 0x560d58eb8da0 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.890+0000 7f863ad9d640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58eb8da0 0x560d59356780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f861eff5640 10 client.17742 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f863ad9d640 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58eb8da0 0x560d59356780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efa6dcd6a9416176 server_cookie=20b89270684a493c in_seq=0 out_seq=0 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f861eff5640 1 -- 172.21.15.92:0/1014702678 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d592408a0 con 0x560d58eb8da0 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f861eff5640 10 client.17742 handle_command_reply: tid=0 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 client.17742 shutdown 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 msgr2=0x560d592ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 0x560d592ea580 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f863c06a890 tx=0x7f86300044d0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 shutdown_connections 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x560d58eb8da0 0x560d59356780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d58f25070 0x560d592eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d58f04390 0x560d592ea580 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 --2- 172.21.15.92:0/1014702678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d58efc1b0 0x560d592ed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 >> 172.21.15.92:0/1014702678 conn(0x560d58f19460 msgr2=0x560d592cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f861eff5640 0 client.17742 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 shutdown_connections 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 1 -- 172.21.15.92:0/1014702678 wait complete. 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.891+0000 7f864758b740 20 client.17742 trim_cache size 0 max 16384 2024-09-21T02:12:39.897 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86240b6510 msgr2=0x7f86240b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86240b6510 0x7f86240b8a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f863402f500 tx=0x7f863402f310 comp rx=0 tx=0).stop 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 msgr2=0x7f863c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 0x7f863c19a4e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f862c00e9f0 tx=0x7f862c00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 shutdown_connections 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f86240b6510 0x7f86240b8a20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f863c108d00 0x7f863c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f863c1083b0 0x7f863c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 --2- 172.21.15.92:0/2633664284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f863c102ac0 0x7f863c19a4e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 >> 172.21.15.92:0/2633664284 conn(0x7f863c079790 msgr2=0x7f863c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 shutdown_connections 2024-09-21T02:12:39.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:39.892+0000 7f8641017640 1 -- 172.21.15.92:0/2633664284 wait complete. 2024-09-21T02:12:40.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:12:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:41.120 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:40 smithi092 ceph-mon[31190]: pgmap v591: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:12:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:40 smithi136 ceph-mon[34754]: pgmap v591: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:12:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:40 smithi099 ceph-mon[34404]: pgmap v591: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:12:41.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 -- 172.21.15.92:0/564805860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6801094a0 msgr2=0x7fc6801098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 --2- 172.21.15.92:0/564805860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6801094a0 0x7fc6801098b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc674009900 tx=0x7fc67402a410 comp rx=0 tx=0).stop 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 -- 172.21.15.92:0/564805860 shutdown_connections 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 --2- 172.21.15.92:0/564805860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc680109df0 0x7fc680110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 --2- 172.21.15.92:0/564805860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6801094a0 0x7fc6801098b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 --2- 172.21.15.92:0/564805860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc680103b60 0x7fc680103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 -- 172.21.15.92:0/564805860 >> 172.21.15.92:0/564805860 conn(0x7fc68007b850 msgr2=0x7fc68007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 -- 172.21.15.92:0/564805860 shutdown_connections 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 -- 172.21.15.92:0/564805860 wait complete. 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.326+0000 7fc687a19640 1 Processor -- start 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 -- start start 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc680103b60 0x7fc68019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6801094a0 0x7fc68019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 0x7fc68019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc680113ab0 con 0x7fc680109df0 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc680113930 con 0x7fc680103b60 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc687a19640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc680113c30 con 0x7fc6801094a0 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 0x7fc68019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 0x7fc68019fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58122/0 (socket says 172.21.15.92:58122) 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 -- 172.21.15.92:0/2292089858 learned_addr learned my addr 172.21.15.92:0/2292089858 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc6857b5640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc680103b60 0x7fc68019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc684fb4640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6801094a0 0x7fc68019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 -- 172.21.15.92:0/2292089858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6801094a0 msgr2=0x7fc68019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6801094a0 0x7fc68019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 -- 172.21.15.92:0/2292089858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc680103b60 msgr2=0x7fc68019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc680103b60 0x7fc68019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.327+0000 7fc685fb6640 1 -- 172.21.15.92:0/2292089858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6801a0430 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.328+0000 7fc685fb6640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 0x7fc68019fdc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc67800c970 tx=0x7fc67800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.328+0000 7fc67e7fc640 1 -- 172.21.15.92:0/2292089858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678007bb0 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.328+0000 7fc67e7fc640 1 -- 172.21.15.92:0/2292089858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc678007d50 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.328+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6801a0720 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.328+0000 7fc67e7fc640 1 -- 172.21.15.92:0/2292089858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678018770 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.328+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6801a0be0 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc68df8d740 1 Processor -- start 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc67e7fc640 1 -- 172.21.15.92:0/2292089858 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc67800f450 con 0x7fc680109df0 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc68df8d740 1 -- start start 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c276a1b0 0x55b9c2b3a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b3ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.331+0000 7fc68df8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2ab0ea0 con 0x55b9c2772390 2024-09-21T02:12:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc68df8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2aa02d0 con 0x55b9c276a1b0 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc68df8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2aa40e0 con 0x55b9c2793070 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b3ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b3ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58126/0 (socket says 172.21.15.92:58126) 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 -- 172.21.15.92:0/675254304 learned_addr learned my addr 172.21.15.92:0/675254304 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc6857b5640 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c276a1b0 0x55b9c2b3a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc67e7fc640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6680b6540 0x7fc6680b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc685fb6640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6680b6540 0x7fc6680b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 -- 172.21.15.92:0/675254304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 msgr2=0x55b9c2b3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b3a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 -- 172.21.15.92:0/675254304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c276a1b0 msgr2=0x55b9c2b3a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c276a1b0 0x55b9c2b3a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 -- 172.21.15.92:0/675254304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c27fe650 con 0x55b9c2772390 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc6857b5640 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c276a1b0 0x55b9c2b3a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:12:41.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc67e7fc640 1 -- 172.21.15.92:0/2292089858 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc6780e7300 con 0x7fc680109df0 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.332+0000 7fc684fb4640 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b3ad40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc6740096c0 tx=0x7fc67402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17d4212e7089c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.333+0000 7fc6677fe640 1 -- 172.21.15.92:0/675254304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674035040 con 0x55b9c2772390 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.333+0000 7fc685fb6640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6680b6540 0x7fc6680b8a50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc6780e5150 tx=0x7fc6780ddf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.333+0000 7fc6677fe640 1 -- 172.21.15.92:0/675254304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6740035f0 con 0x55b9c2772390 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.333+0000 7fc6677fe640 1 -- 172.21.15.92:0/675254304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6740044e0 con 0x55b9c2772390 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 -- 172.21.15.92:0/675254304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 msgr2=0x55b9c2b3ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b3ad40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc6740096c0 tx=0x7fc67402a410 comp rx=0 tx=0).stop 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 -- 172.21.15.92:0/675254304 shutdown_connections 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b3ad40 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c276a1b0 0x55b9c2b3a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 --2- 172.21.15.92:0/675254304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 -- 172.21.15.92:0/675254304 >> 172.21.15.92:0/675254304 conn(0x55b9c2787460 msgr2=0x55b9c2b381b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 -- 172.21.15.92:0/675254304 shutdown_connections 2024-09-21T02:12:41.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 -- 172.21.15.92:0/675254304 wait complete. 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 -- start start 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.334+0000 7fc68df8d740 1 Processor -- start 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2772390 0x55b9c2b45ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b51d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2901700 con 0x55b9c276a1b0 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2a3b160 con 0x55b9c2772390 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2b48a40 con 0x55b9c2793070 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58128/0 (socket says 172.21.15.92:58128) 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 -- 172.21.15.92:0/2393657708 learned_addr learned my addr 172.21.15.92:0/2393657708 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:41.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc6857b5640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2772390 0x55b9c2b45ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b51d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 -- 172.21.15.92:0/2393657708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 msgr2=0x55b9c2b51d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 10 client.0 _ll_register_callbacks cb 0x55b9c2a549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b51d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 -- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2772390 msgr2=0x55b9c2b45ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2772390 0x55b9c2b45ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc684fb4640 1 -- 172.21.15.92:0/2393657708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c2a68c90 con 0x55b9c276a1b0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.335+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6740035f0 con 0x55b9c276a1b0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc684fb4640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45960 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc6740092a0 tx=0x7fc674009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4397d5393c011968 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc664ff9640 10 client.17754 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc664ff9640 1 -- 172.21.15.92:0/2393657708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67405a050 con 0x55b9c276a1b0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc664ff9640 1 -- 172.21.15.92:0/2393657708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc674004270 con 0x55b9c276a1b0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc664ff9640 1 -- 172.21.15.92:0/2393657708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc674035070 con 0x55b9c276a1b0 2024-09-21T02:12:41.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 10 client.17754 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b9c2b5ce40 con 0x55b9c276a1b0 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc664ff9640 1 -- 172.21.15.92:0/2393657708 <== mon.0 v2:172.21.15.92:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc67405d020 con 0x55b9c276a1b0 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 10 client.17754 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 10 client.17754 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 20 client.17754 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c27519f0 0x55b9c2b601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 4 client.17754 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.336+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c2ab2df0 con 0x55b9c27519f0 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.337+0000 7fc6857b5640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c27519f0 0x55b9c2b601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.337+0000 7fc664ff9640 10 client.17754 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:12:41.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.337+0000 7fc6857b5640 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c27519f0 0x55b9c2b601a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=beaad14d867e3b51 server_cookie=e5d8973646001582 in_seq=0 out_seq=0 2024-09-21T02:12:41.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc664ff9640 1 -- 172.21.15.92:0/2393657708 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9c2ab2df0 con 0x55b9c27519f0 2024-09-21T02:12:41.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc664ff9640 10 client.17754 handle_command_reply: tid=0 2024-09-21T02:12:41.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 client.17754 shutdown 2024-09-21T02:12:41.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 msgr2=0x55b9c2b45960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45960 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc6740092a0 tx=0x7fc674009230 comp rx=0 tx=0).stop 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 shutdown_connections 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c2793070 0x55b9c2b51d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2772390 0x55b9c2b45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c27519f0 0x55b9c2b601a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 --2- 172.21.15.92:0/2393657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45960 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 >> 172.21.15.92:0/2393657708 conn(0x55b9c2787460 msgr2=0x55b9c2b3d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc664ff9640 0 client.17754 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 shutdown_connections 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 1 -- 172.21.15.92:0/2393657708 wait complete. 2024-09-21T02:12:41.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.338+0000 7fc68df8d740 20 client.17754 trim_cache size 0 max 16384 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 Processor -- start 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 -- start start 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b51ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b42b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801ad170 con 0x55b9c2772390 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801aecb0 con 0x55b9c2793070 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc68df8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801ad840 con 0x55b9c276a1b0 2024-09-21T02:12:41.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b51ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b51ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58138/0 (socket says 172.21.15.92:58138) 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 -- 172.21.15.92:0/261803043 learned_addr learned my addr 172.21.15.92:0/261803043 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc684fb4640 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b42b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc684fb4640 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 -- 172.21.15.92:0/261803043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 msgr2=0x55b9c2b45e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 -- 172.21.15.92:0/261803043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 msgr2=0x55b9c2b42b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b42b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.352+0000 7fc6857b5640 1 -- 172.21.15.92:0/261803043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c2b46800 con 0x55b9c2772390 2024-09-21T02:12:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.353+0000 7fc6857b5640 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b51ca0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x55b9c2ab0ed0 tx=0x7fc670012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55eefb9822105f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.353+0000 7fc666ffd640 1 -- 172.21.15.92:0/261803043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670012da0 con 0x55b9c2772390 2024-09-21T02:12:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.353+0000 7fc666ffd640 1 -- 172.21.15.92:0/261803043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc670011d40 con 0x55b9c2772390 2024-09-21T02:12:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.353+0000 7fc666ffd640 1 -- 172.21.15.92:0/261803043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67001ec90 con 0x55b9c2772390 2024-09-21T02:12:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 -- 172.21.15.92:0/261803043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 msgr2=0x55b9c2b51ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b51ca0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x55b9c2ab0ed0 tx=0x7fc670012bb0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 -- 172.21.15.92:0/261803043 shutdown_connections 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b42b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b51ca0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 --2- 172.21.15.92:0/261803043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b45e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 -- 172.21.15.92:0/261803043 >> 172.21.15.92:0/261803043 conn(0x55b9c2787460 msgr2=0x55b9c2b45710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 -- 172.21.15.92:0/261803043 shutdown_connections 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 -- 172.21.15.92:0/261803043 wait complete. 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 -- start start 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.354+0000 7fc68df8d740 1 Processor -- start 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b61820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b61d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801ae5e0 con 0x55b9c2772390 2024-09-21T02:12:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801adf10 con 0x55b9c2793070 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801ab180 con 0x55b9c276a1b0 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc68df8d740 10 client.0 _ll_register_callbacks cb 0x7fc670009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b61820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b61820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58150/0 (socket says 172.21.15.92:58150) 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 -- 172.21.15.92:0/1937137239 learned_addr learned my addr 172.21.15.92:0/1937137239 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc684fb4640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b61d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc684fb4640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 -- 172.21.15.92:0/1937137239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 msgr2=0x55b9c2b612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 -- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 msgr2=0x55b9c2b61d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b61d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 -- 172.21.15.92:0/1937137239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c2bc9a50 con 0x55b9c2772390 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc684fb4640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b61d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc6857b5640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b61820 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc670005fa0 tx=0x7fc670011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910d6e934d79d178 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.355+0000 7fc664ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc664ff9640 1 -- 172.21.15.92:0/1937137239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67001e420 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc664ff9640 1 -- 172.21.15.92:0/1937137239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc670020040 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc670011d40 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc664ff9640 1 -- 172.21.15.92:0/1937137239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc67001e5c0 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 10 client.17760 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9c2bcac20 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc664ff9640 1 -- 172.21.15.92:0/1937137239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc670011d40 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc664ff9640 1 -- 172.21.15.92:0/1937137239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc67000f040 con 0x55b9c2772390 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 10 client.17760 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 10 client.17760 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 20 client.17760 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c2726da0 0x55b9c2bcd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 4 client.17760 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:12:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.356+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9c2aae8a0 con 0x55b9c2726da0 2024-09-21T02:12:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc684fb4640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c2726da0 0x55b9c2bcd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc684fb4640 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c2726da0 0x55b9c2bcd2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f374148ac6149b7 server_cookie=c6f02b77e7ae553d in_seq=0 out_seq=0 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc664ff9640 10 client.17760 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc664ff9640 1 -- 172.21.15.92:0/1937137239 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9c2aae8a0 con 0x55b9c2726da0 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc664ff9640 10 client.17760 handle_command_reply: tid=0 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc68df8d740 1 client.17760 shutdown 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 msgr2=0x55b9c2b61820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc68df8d740 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b61820 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc670005fa0 tx=0x7fc670011890 comp rx=0 tx=0).stop 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 shutdown_connections 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc68df8d740 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9c2793070 0x55b9c2b61d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.357+0000 7fc68df8d740 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55b9c2726da0 0x55b9c2bcd2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc68df8d740 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9c2772390 0x55b9c2b61820 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc664ff9640 0 client.17760 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:12:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc68df8d740 1 --2- 172.21.15.92:0/1937137239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9c276a1b0 0x55b9c2b612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 >> 172.21.15.92:0/1937137239 conn(0x55b9c2787460 msgr2=0x55b9c2b46e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 shutdown_connections 2024-09-21T02:12:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc68df8d740 1 -- 172.21.15.92:0/1937137239 wait complete. 2024-09-21T02:12:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc68df8d740 20 client.17760 trim_cache size 0 max 16384 2024-09-21T02:12:41.363 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6680b6540 msgr2=0x7fc6680b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc687a19640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6680b6540 0x7fc6680b8a50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc6780e5150 tx=0x7fc6780ddf10 comp rx=0 tx=0).stop 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 msgr2=0x7fc68019fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc687a19640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 0x7fc68019fdc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc67800c970 tx=0x7fc67800ce30 comp rx=0 tx=0).stop 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.358+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 shutdown_connections 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc6680b6540 0x7fc6680b8a50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc680109df0 0x7fc68019fdc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6801094a0 0x7fc68019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 --2- 172.21.15.92:0/2292089858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc680103b60 0x7fc68019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:41.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 >> 172.21.15.92:0/2292089858 conn(0x7fc68007b850 msgr2=0x7fc680105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:41.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 shutdown_connections 2024-09-21T02:12:41.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:41.359+0000 7fc687a19640 1 -- 172.21.15.92:0/2292089858 wait complete. 2024-09-21T02:12:41.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:12:42.237 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 -- 172.21.15.92:0/2412954686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c84109e00 msgr2=0x7f8c84110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/2412954686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c84109e00 0x7f8c84110630 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009ca0 tx=0x7f8c7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 -- 172.21.15.92:0/2412954686 shutdown_connections 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/2412954686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c84109e00 0x7f8c84110630 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/2412954686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c841094b0 0x7f8c841098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/2412954686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84101f70 0x7f8c84102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 -- 172.21.15.92:0/2412954686 >> 172.21.15.92:0/2412954686 conn(0x7f8c84077c60 msgr2=0x7f8c840780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 -- 172.21.15.92:0/2412954686 shutdown_connections 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.734+0000 7f8c8ae09640 1 -- 172.21.15.92:0/2412954686 wait complete. 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 Processor -- start 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 -- start start 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c84101f70 0x7f8c8419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 0x7f8c8419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84109e00 0x7f8c8419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84113ac0 con 0x7f8c841094b0 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84113940 con 0x7f8c84101f70 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c8ae09640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84113c40 con 0x7f8c84109e00 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 0x7f8c8419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 0x7f8c8419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58180/0 (socket says 172.21.15.92:58180) 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 -- 172.21.15.92:0/3581590370 learned_addr learned my addr 172.21.15.92:0/3581590370 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c84101f70 0x7f8c8419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84109e00 0x7f8c8419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 -- 172.21.15.92:0/3581590370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84109e00 msgr2=0x7f8c8419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84109e00 0x7f8c8419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 -- 172.21.15.92:0/3581590370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c84101f70 msgr2=0x7f8c8419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c84101f70 0x7f8c8419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c83fff640 1 -- 172.21.15.92:0/3581590370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c841a0400 con 0x7f8c841094b0 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c84101f70 0x7f8c8419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.735+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84109e00 0x7f8c8419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.736+0000 7f8c83fff640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 0x7f8c8419aa30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800d740 tx=0x7f8c7800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.736+0000 7f8c81ffb640 1 -- 172.21.15.92:0/3581590370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78017070 con 0x7f8c841094b0 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.736+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841a06f0 con 0x7f8c841094b0 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.736+0000 7f8c81ffb640 1 -- 172.21.15.92:0/3581590370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7800de50 con 0x7f8c841094b0 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.736+0000 7f8c81ffb640 1 -- 172.21.15.92:0/3581590370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800f620 con 0x7f8c841094b0 2024-09-21T02:12:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.736+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a6e50 con 0x7f8c841094b0 2024-09-21T02:12:42.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 Processor -- start 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 -- start start 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c81ffb640 1 -- 172.21.15.92:0/3581590370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8c7801a070 con 0x7f8c841094b0 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f60b918ea0 con 0x55f60b5da390 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f60b9082d0 con 0x55f60b5fb070 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c9137d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f60b90c0e0 con 0x55f60b5d21b0 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58196/0 (socket says 172.21.15.92:58196) 2024-09-21T02:12:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c83fff640 1 -- 172.21.15.92:0/1648295192 learned_addr learned my addr 172.21.15.92:0/1648295192 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.739+0000 7f8c893a6640 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c83fff640 1 -- 172.21.15.92:0/1648295192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 msgr2=0x55f60b9a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c83fff640 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c83fff640 1 -- 172.21.15.92:0/1648295192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 msgr2=0x55f60b9a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c83fff640 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c83fff640 1 -- 172.21.15.92:0/1648295192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60b666650 con 0x55f60b5da390 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c81ffb640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c6c0b6510 0x7f8c6c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c83fff640 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9a3120 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8c780eafc0 tx=0x7f8c780ef8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750a6454a659fe39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c6c0b6510 0x7f8c6c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c6affd640 1 -- 172.21.15.92:0/1648295192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c780a9500 con 0x55f60b5da390 2024-09-21T02:12:42.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c6affd640 1 -- 172.21.15.92:0/1648295192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c780ac9c0 con 0x55f60b5da390 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c6affd640 1 -- 172.21.15.92:0/1648295192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78078650 con 0x55f60b5da390 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.740+0000 7f8c81ffb640 1 -- 172.21.15.92:0/3581590370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8c780e6d40 con 0x7f8c841094b0 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 -- 172.21.15.92:0/1648295192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 msgr2=0x55f60b9a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9a3120 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8c780eafc0 tx=0x7f8c780ef8d0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c6c0b6510 0x7f8c6c0b8a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02f500 tx=0x7f8c7c02f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 -- 172.21.15.92:0/1648295192 shutdown_connections 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9a3120 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 --2- 172.21.15.92:0/1648295192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 -- 172.21.15.92:0/1648295192 >> 172.21.15.92:0/1648295192 conn(0x55f60b5ef460 msgr2=0x55f60b9a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 -- 172.21.15.92:0/1648295192 shutdown_connections 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.741+0000 7f8c9137d740 1 -- 172.21.15.92:0/1648295192 wait complete. 2024-09-21T02:12:42.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 -- start start 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 Processor -- start 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5d21b0 0x55f60b9add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9ae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5fb070 0x55f60b9be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f60b769700 con 0x55f60b5da390 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f60b8a3160 con 0x55f60b5d21b0 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f60b9b5290 con 0x55f60b5fb070 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.742+0000 7f8c9137d740 10 client.0 _ll_register_callbacks cb 0x55f60b8bc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58210/0 (socket says 172.21.15.92:58210) 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 -- 172.21.15.92:0/621920556 learned_addr learned my addr 172.21.15.92:0/621920556 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c893a6640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5fb070 0x55f60b9be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5d21b0 0x55f60b9add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 -- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5fb070 msgr2=0x55f60b9be5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5fb070 0x55f60b9be5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 -- 172.21.15.92:0/621920556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5d21b0 msgr2=0x55f60b9add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5d21b0 0x55f60b9add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 -- 172.21.15.92:0/621920556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60b8d0c90 con 0x55f60b5da390 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5d21b0 0x55f60b9add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c893a6640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5fb070 0x55f60b9be5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c83fff640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9ae260 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8c780acbb0 tx=0x7f8c78010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13b046e0e56e87e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c63fff640 1 -- 172.21.15.92:0/621920556 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78021030 con 0x55f60b5da390 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c63fff640 1 -- 172.21.15.92:0/621920556 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8c78009c30 con 0x55f60b5da390 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c780ac9c0 con 0x55f60b5da390 2024-09-21T02:12:42.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c63fff640 1 -- 172.21.15.92:0/621920556 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c780788c0 con 0x55f60b5da390 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c9137d740 10 client.17772 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.743+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f60b9c0f90 con 0x55f60b5da390 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c63fff640 1 -- 172.21.15.92:0/621920556 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c780ac9c0 con 0x55f60b5da390 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c63fff640 1 -- 172.21.15.92:0/621920556 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c780e4ce0 con 0x55f60b5da390 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c9137d740 10 client.17772 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c9137d740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c9137d740 20 client.17772 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c9137d740 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b5b99f0 0x55f60b9c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c9137d740 4 client.17772 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f60b91adf0 con 0x55f60b5b99f0 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.744+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b5b99f0 0x55f60b9c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.745+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b5b99f0 0x55f60b9c42f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d91725a99c9c2a server_cookie=dde763a28ce2aa3a in_seq=0 out_seq=0 2024-09-21T02:12:42.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.745+0000 7f8c63fff640 10 client.17772 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:42.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.745+0000 7f8c63fff640 1 -- 172.21.15.92:0/621920556 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f60b91adf0 con 0x55f60b5b99f0 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.745+0000 7f8c63fff640 10 client.17772 handle_command_reply: tid=0 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.745+0000 7f8c9137d740 1 client.17772 shutdown 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 msgr2=0x55f60b9ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9ae260 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8c780acbb0 tx=0x7f8c78010b70 comp rx=0 tx=0).stop 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 shutdown_connections 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b5b99f0 0x55f60b9c42f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5fb070 0x55f60b9be5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5da390 0x55f60b9ae260 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 --2- 172.21.15.92:0/621920556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5d21b0 0x55f60b9add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 >> 172.21.15.92:0/621920556 conn(0x55f60b5ef460 msgr2=0x55f60b9a5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c63fff640 0 client.17772 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 shutdown_connections 2024-09-21T02:12:42.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 1 -- 172.21.15.92:0/621920556 wait complete. 2024-09-21T02:12:42.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.746+0000 7f8c9137d740 20 client.17772 trim_cache size 0 max 16384 2024-09-21T02:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 Processor -- start 2024-09-21T02:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 -- start start 2024-09-21T02:12:42.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 0x55f60b9acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5da390 0x55f60b9af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9be520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ad250 con 0x55f60b5d21b0 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aed90 con 0x55f60b5fb070 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c9137d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ad920 con 0x55f60b5da390 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 0x55f60b9acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 0x55f60b9acc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58218/0 (socket says 172.21.15.92:58218) 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 -- 172.21.15.92:0/3832046413 learned_addr learned my addr 172.21.15.92:0/3832046413 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 -- 172.21.15.92:0/3832046413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5da390 msgr2=0x55f60b9af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5da390 0x55f60b9af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 -- 172.21.15.92:0/3832046413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 msgr2=0x55f60b9be520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9be520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.759+0000 7f8c88ba5640 1 -- 172.21.15.92:0/3832046413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60b5b9c20 con 0x55f60b5d21b0 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.760+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 0x55f60b9acc10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005fd0 tx=0x7f8c74005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aac45511d468423 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.760+0000 7f8c6a7fc640 1 -- 172.21.15.92:0/3832046413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74005230 con 0x55f60b5d21b0 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.760+0000 7f8c6a7fc640 1 -- 172.21.15.92:0/3832046413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c740053d0 con 0x55f60b5d21b0 2024-09-21T02:12:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.760+0000 7f8c6a7fc640 1 -- 172.21.15.92:0/3832046413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740118a0 con 0x55f60b5d21b0 2024-09-21T02:12:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 -- 172.21.15.92:0/3832046413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 msgr2=0x55f60b9acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 0x55f60b9acc10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005fd0 tx=0x7f8c74005850 comp rx=0 tx=0).stop 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 -- 172.21.15.92:0/3832046413 shutdown_connections 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5fb070 0x55f60b9be520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5da390 0x55f60b9af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3832046413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5d21b0 0x55f60b9acc10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 -- 172.21.15.92:0/3832046413 >> 172.21.15.92:0/3832046413 conn(0x55f60b5ef460 msgr2=0x55f60b9af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 -- 172.21.15.92:0/3832046413 shutdown_connections 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 -- 172.21.15.92:0/3832046413 wait complete. 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 -- start start 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 Processor -- start 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.761+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5da390 0x55f60b9c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c9137d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 0x55f60b9c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c9137d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ae6c0 con 0x55f60b5fb070 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c9137d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841adff0 con 0x55f60b5da390 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c9137d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ab230 con 0x55f60b5d21b0 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c9137d740 10 client.0 _ll_register_callbacks cb 0x7f8c780eae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 0x55f60b9c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 0x55f60b9c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58220/0 (socket says 172.21.15.92:58220) 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 -- 172.21.15.92:0/3208174146 learned_addr learned my addr 172.21.15.92:0/3208174146 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c83fff640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5da390 0x55f60b9c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 -- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 msgr2=0x55f60b9c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 -- 172.21.15.92:0/3208174146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5da390 msgr2=0x55f60b9c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5da390 0x55f60b9c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 -- 172.21.15.92:0/3208174146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60b9c5ca0 con 0x55f60b5fb070 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c893a6640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 0x55f60b9c5510 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c033040 tx=0x7f8c7c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea802274daaf977f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c63fff640 1 -- 172.21.15.92:0/3208174146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c004250 con 0x55f60b5fb070 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c63fff640 1 -- 172.21.15.92:0/3208174146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8c7c0043f0 con 0x55f60b5fb070 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.762+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c740053d0 con 0x55f60b5fb070 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c63fff640 1 -- 172.21.15.92:0/3208174146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c0045f0 con 0x55f60b5fb070 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 10 client.17778 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:42.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f60ba2e9f0 con 0x55f60b5fb070 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c63fff640 1 -- 172.21.15.92:0/3208174146 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c740053d0 con 0x55f60b5fb070 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c63fff640 1 -- 172.21.15.92:0/3208174146 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c7c043040 con 0x55f60b5fb070 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 10 client.17778 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 20 client.17778 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b58eda0 0x55f60ba31210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 4 client.17778 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f60b9168a0 con 0x55f60b58eda0 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.763+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b58eda0 0x55f60ba31210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c63fff640 10 client.17778 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c88ba5640 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b58eda0 0x55f60ba31210 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3473f2897ea13159 server_cookie=c11ed0f217d79289 in_seq=0 out_seq=0 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c63fff640 1 -- 172.21.15.92:0/3208174146 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f60b9168a0 con 0x55f60b58eda0 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c63fff640 10 client.17778 handle_command_reply: tid=0 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 client.17778 shutdown 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 msgr2=0x55f60b9c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 0x55f60b9c5510 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c033040 tx=0x7f8c7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 shutdown_connections 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55f60b58eda0 0x55f60ba31210 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f60b5fb070 0x55f60b9c5510 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f60b5da390 0x55f60b9c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 --2- 172.21.15.92:0/3208174146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f60b5d21b0 0x55f60b9c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c63fff640 0 client.17778 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 >> 172.21.15.92:0/3208174146 conn(0x55f60b5ef460 msgr2=0x55f60b9aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 shutdown_connections 2024-09-21T02:12:42.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 1 -- 172.21.15.92:0/3208174146 wait complete. 2024-09-21T02:12:42.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.764+0000 7f8c9137d740 20 client.17778 trim_cache size 0 max 16384 2024-09-21T02:12:42.772 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c6c0b6510 msgr2=0x7f8c6c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c6c0b6510 0x7f8c6c0b8a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02f500 tx=0x7f8c7c02f310 comp rx=0 tx=0).stop 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 msgr2=0x7f8c8419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 0x7f8c8419aa30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800d740 tx=0x7f8c7800dc00 comp rx=0 tx=0).stop 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 shutdown_connections 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8c6c0b6510 0x7f8c6c0b8a20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c84109e00 0x7f8c8419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c841094b0 0x7f8c8419aa30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 --2- 172.21.15.92:0/3581590370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c84101f70 0x7f8c8419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 >> 172.21.15.92:0/3581590370 conn(0x7f8c84077c60 msgr2=0x7f8c84105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 shutdown_connections 2024-09-21T02:12:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:42.765+0000 7f8c8ae09640 1 -- 172.21.15.92:0/3581590370 wait complete. 2024-09-21T02:12:43.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:42 smithi092 ceph-mon[31190]: pgmap v592: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:12:43.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:42 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:12:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:42 smithi136 ceph-mon[34754]: pgmap v592: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:12:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:42 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:12:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:42 smithi099 ceph-mon[34404]: pgmap v592: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:12:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:42 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:12:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:12:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:12:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:12:44.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:12:44.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:12:44.305 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:12:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:12:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:12:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:12:44.493 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:12:44.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:12:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:45.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:44 smithi092 ceph-mon[31190]: pgmap v593: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:12:45.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 -- 172.21.15.92:0/3664113760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac01083b0 msgr2=0x7f6ac01087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/3664113760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac01083b0 0x7f6ac01087c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009900 tx=0x7f6ab402a410 comp rx=0 tx=0).stop 2024-09-21T02:12:45.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 -- 172.21.15.92:0/3664113760 shutdown_connections 2024-09-21T02:12:45.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/3664113760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ac0108d00 0x7f6ac010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/3664113760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac01083b0 0x7f6ac01087c0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/3664113760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0102a70 0x7f6ac0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 -- 172.21.15.92:0/3664113760 >> 172.21.15.92:0/3664113760 conn(0x7f6ac00796e0 msgr2=0x7f6ac0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 -- 172.21.15.92:0/3664113760 shutdown_connections 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.231+0000 7f6ac6ede640 1 -- 172.21.15.92:0/3664113760 wait complete. 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 Processor -- start 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 -- start start 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 0x7f6ac01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ac01083b0 0x7f6ac0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0108d00 0x7f6ac019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01129c0 con 0x7f6ac0102a70 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0112840 con 0x7f6ac01083b0 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac6ede640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0112b40 con 0x7f6ac0108d00 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 0x7f6ac01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 0x7f6ac01994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43712/0 (socket says 172.21.15.92:43712) 2024-09-21T02:12:45.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/1969755078 learned_addr learned my addr 172.21.15.92:0/1969755078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6abffff640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ac01083b0 0x7f6ac0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0108d00 0x7f6ac019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/1969755078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0108d00 msgr2=0x7f6ac019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0108d00 0x7f6ac019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/1969755078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ac01083b0 msgr2=0x7f6ac0199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ac01083b0 0x7f6ac0199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/1969755078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac019f420 con 0x7f6ac0102a70 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0108d00 0x7f6ac019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.232+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 0x7f6ac01994b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000d770 tx=0x7f6ab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.233+0000 7f6abdffb640 1 -- 172.21.15.92:0/1969755078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0017070 con 0x7f6ac0102a70 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.233+0000 7f6abdffb640 1 -- 172.21.15.92:0/1969755078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab00054d0 con 0x7f6ac0102a70 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.233+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac019f710 con 0x7f6ac0102a70 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.233+0000 7f6abdffb640 1 -- 172.21.15.92:0/1969755078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0004c30 con 0x7f6ac0102a70 2024-09-21T02:12:45.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.233+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac01a5e70 con 0x7f6ac0102a70 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6abdffb640 1 -- 172.21.15.92:0/1969755078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f6ab0004e10 con 0x7f6ac0102a70 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 Processor -- start 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 -- start start 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab7519350 0x55aab78c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55aab7836e60 con 0x55aab74f0490 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55aab7826290 con 0x55aab7519350 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6acd452740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aab782a0a0 con 0x55aab74f8670 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43726/0 (socket says 172.21.15.92:43726) 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/3010568381 learned_addr learned my addr 172.21.15.92:0/3010568381 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:45.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac547b640 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab7519350 0x55aab78c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6abffff640 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/3010568381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 msgr2=0x55aab78c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78c0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/3010568381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab7519350 msgr2=0x55aab78c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab7519350 0x55aab78c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.236+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/3010568381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab7584930 con 0x55aab74f0490 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6abdffb640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6aa80b6510 0x7f6aa80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c0970 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00eaf20 tx=0x7f6ab00ef570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518930b8c5981ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6aa80b6510 0x7f6aa80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6aa6ffd640 1 -- 172.21.15.92:0/3010568381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00a95d0 con 0x55aab74f0490 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6abdffb640 1 -- 172.21.15.92:0/1969755078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6ab00e6e10 con 0x7f6ac0102a70 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6aa6ffd640 1 -- 172.21.15.92:0/3010568381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab00aca90 con 0x55aab74f0490 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.237+0000 7f6aa6ffd640 1 -- 172.21.15.92:0/3010568381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0022510 con 0x55aab74f0490 2024-09-21T02:12:45.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 -- 172.21.15.92:0/3010568381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 msgr2=0x55aab78c0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c0970 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00eaf20 tx=0x7f6ab00ef570 comp rx=0 tx=0).stop 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6aa80b6510 0x7f6aa80b8a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005fd0 tx=0x7f6ab8008370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 -- 172.21.15.92:0/3010568381 shutdown_connections 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c0970 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 --2- 172.21.15.92:0/3010568381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab7519350 0x55aab78c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 -- 172.21.15.92:0/3010568381 >> 172.21.15.92:0/3010568381 conn(0x55aab750d740 msgr2=0x55aab78be320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 -- 172.21.15.92:0/3010568381 shutdown_connections 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 -- 172.21.15.92:0/3010568381 wait complete. 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 -- start start 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.238+0000 7f6acd452740 1 Processor -- start 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f0490 0x55aab78d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 0x55aab78d6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55aab7687890 con 0x55aab7519350 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55aab77c0d40 con 0x55aab74f0490 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aab78d0d70 con 0x55aab74f8670 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6acd452740 10 client.0 _ll_register_callbacks cb 0x55aab7824c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 0x55aab78d6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 0x55aab78d6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43730/0 (socket says 172.21.15.92:43730) 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 -- 172.21.15.92:0/1290993474 learned_addr learned my addr 172.21.15.92:0/1290993474 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f0490 0x55aab78d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6abffff640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 -- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 msgr2=0x55aab78d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 -- 172.21.15.92:0/1290993474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f0490 msgr2=0x55aab78d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f0490 0x55aab78d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.239+0000 7f6ac547b640 1 -- 172.21.15.92:0/1290993474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab78d67a0 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6ac547b640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 0x55aab78d6130 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8010e00 tx=0x7f6ab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c7986024ad8fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6a9ffff640 1 -- 172.21.15.92:0/1290993474 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab801d620 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6a9ffff640 1 -- 172.21.15.92:0/1290993474 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6ab801d7c0 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab00aca90 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6a9ffff640 1 -- 172.21.15.92:0/1290993474 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ab8010e30 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 10 client.17790 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aab78df1f0 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6a9ffff640 1 -- 172.21.15.92:0/1290993474 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab00aca90 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6a9ffff640 1 -- 172.21.15.92:0/1290993474 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ab801bab0 con 0x55aab7519350 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 10 client.17790 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 20 client.17790 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab74d7cd0 0x55aab78e2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 4 client.17790 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.240+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aab7838db0 con 0x55aab74d7cd0 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.241+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab74d7cd0 0x55aab78e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.241+0000 7f6a9ffff640 10 client.17790 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.241+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab74d7cd0 0x55aab78e2720 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e9b08738e4deae server_cookie=3a0250ef7416b288 in_seq=0 out_seq=0 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6a9ffff640 1 -- 172.21.15.92:0/1290993474 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aab7838db0 con 0x55aab74d7cd0 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6a9ffff640 10 client.17790 handle_command_reply: tid=0 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 client.17790 shutdown 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 msgr2=0x55aab78d6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 0x55aab78d6130 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8010e00 tx=0x7f6ab8004a40 comp rx=0 tx=0).stop 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 shutdown_connections 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab74d7cd0 0x55aab78e2720 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab7519350 0x55aab78d6130 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f8670 0x55aab78d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6a9ffff640 0 client.17790 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 --2- 172.21.15.92:0/1290993474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f0490 0x55aab78d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 >> 172.21.15.92:0/1290993474 conn(0x55aab750d740 msgr2=0x55aab78c1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 shutdown_connections 2024-09-21T02:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 1 -- 172.21.15.92:0/1290993474 wait complete. 2024-09-21T02:12:45.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.242+0000 7f6acd452740 20 client.17790 trim_cache size 0 max 16384 2024-09-21T02:12:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 Processor -- start 2024-09-21T02:12:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 -- start start 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74d7cd0 0x55aab78c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab78c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01ad3d0 con 0x55aab74f0490 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01aef10 con 0x55aab74f8670 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6acd452740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01adaa0 con 0x55aab74d7cd0 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.260+0000 7f6abffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43746/0 (socket says 172.21.15.92:43746) 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 -- 172.21.15.92:0/253080801 learned_addr learned my addr 172.21.15.92:0/253080801 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:45.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6ac547b640 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab78c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74d7cd0 0x55aab78c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 -- 172.21.15.92:0/253080801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74d7cd0 msgr2=0x55aab78c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74d7cd0 0x55aab78c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 -- 172.21.15.92:0/253080801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 msgr2=0x55aab78c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab78c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 -- 172.21.15.92:0/253080801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aab78b8f70 con 0x55aab74f0490 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74d7cd0 0x55aab78c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6abffff640 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c6f40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009900 tx=0x7f6ab4032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f97d5cb148a047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6aa67fc640 1 -- 172.21.15.92:0/253080801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4035040 con 0x55aab74f0490 2024-09-21T02:12:45.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6aa67fc640 1 -- 172.21.15.92:0/253080801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab40316a0 con 0x55aab74f0490 2024-09-21T02:12:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.261+0000 7f6aa67fc640 1 -- 172.21.15.92:0/253080801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4004750 con 0x55aab74f0490 2024-09-21T02:12:45.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 -- 172.21.15.92:0/253080801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 msgr2=0x55aab78c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c6f40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009900 tx=0x7f6ab4032e60 comp rx=0 tx=0).stop 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 -- 172.21.15.92:0/253080801 shutdown_connections 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab78c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74f0490 0x55aab78c6f40 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 --2- 172.21.15.92:0/253080801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74d7cd0 0x55aab78c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 -- 172.21.15.92:0/253080801 >> 172.21.15.92:0/253080801 conn(0x55aab750d740 msgr2=0x55aab78c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 -- 172.21.15.92:0/253080801 shutdown_connections 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.262+0000 7f6acd452740 1 -- 172.21.15.92:0/253080801 wait complete. 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 -- start start 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 Processor -- start 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 0x55aab78d90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f0490 0x55aab78d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab79461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01ae840 con 0x55aab74d7cd0 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01ae170 con 0x55aab74f8670 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac01ab3b0 con 0x55aab74f0490 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6acd452740 10 client.0 _ll_register_callbacks cb 0x7f6ab8008ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6ac4c7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 0x55aab78d90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6ac4c7a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 0x55aab78d90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43754/0 (socket says 172.21.15.92:43754) 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/865340302 learned_addr learned my addr 172.21.15.92:0/865340302 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.263+0000 7f6ac547b640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab79461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6abffff640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f0490 0x55aab78d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f0490 msgr2=0x55aab78d96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f0490 0x55aab78d96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/865340302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 msgr2=0x55aab79461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab79461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac4c7a640 1 -- 172.21.15.92:0/865340302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aab7946840 con 0x55aab74d7cd0 2024-09-21T02:12:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac547b640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab79461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6ac4c7a640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 0x55aab78d90d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009610 tx=0x7f6ab00e7110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95829f690f573441 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6a9ffff640 1 -- 172.21.15.92:0/865340302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab001ce60 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6a9ffff640 1 -- 172.21.15.92:0/865340302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6ab00ea6c0 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab40316a0 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6a9ffff640 1 -- 172.21.15.92:0/865340302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ab0020050 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6acd452740 10 client.17796 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aab794dba0 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.264+0000 7f6a9ffff640 1 -- 172.21.15.92:0/865340302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab40316a0 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6a9ffff640 1 -- 172.21.15.92:0/865340302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ab00a91f0 con 0x55aab74d7cd0 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6acd452740 10 client.17796 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6acd452740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6acd452740 20 client.17796 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:45.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6acd452740 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab7519350 0x55aab794ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6acd452740 4 client.17796 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:12:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aab7834860 con 0x55aab7519350 2024-09-21T02:12:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.265+0000 7f6abffff640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab7519350 0x55aab794ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.266+0000 7f6a9ffff640 10 client.17796 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:45.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.266+0000 7f6abffff640 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab7519350 0x55aab794ff40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387c2cae5ec2f32e server_cookie=b218ac311b419d09 in_seq=0 out_seq=0 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6a9ffff640 1 -- 172.21.15.92:0/865340302 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aab7834860 con 0x55aab7519350 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6a9ffff640 10 client.17796 handle_command_reply: tid=0 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 client.17796 shutdown 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 msgr2=0x55aab78d90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 0x55aab78d90d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009610 tx=0x7f6ab00e7110 comp rx=0 tx=0).stop 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 shutdown_connections 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55aab7519350 0x55aab794ff40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aab74f8670 0x55aab79461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aab74f0490 0x55aab78d96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 --2- 172.21.15.92:0/865340302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55aab74d7cd0 0x55aab78d90d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 >> 172.21.15.92:0/865340302 conn(0x55aab750d740 msgr2=0x55aab78c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.267+0000 7f6a9ffff640 0 client.17796 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 shutdown_connections 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6acd452740 1 -- 172.21.15.92:0/865340302 wait complete. 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6acd452740 20 client.17796 trim_cache size 0 max 16384 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "4091c28f-03b5-4162-8d4f-dbcbba169f12", 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:12:45.272 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6aa80b6510 msgr2=0x7f6aa80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6aa80b6510 0x7f6aa80b8a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005fd0 tx=0x7f6ab8008370 comp rx=0 tx=0).stop 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 msgr2=0x7f6ac01994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 0x7f6ac01994b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000d770 tx=0x7f6ab000dc30 comp rx=0 tx=0).stop 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 shutdown_connections 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6aa80b6510 0x7f6aa80b8a20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ac0108d00 0x7f6ac019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ac01083b0 0x7f6ac0199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 --2- 172.21.15.92:0/1969755078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ac0102a70 0x7f6ac01994b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:45.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.268+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 >> 172.21.15.92:0/1969755078 conn(0x7f6ac00796e0 msgr2=0x7f6ac01044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:45.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.269+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 shutdown_connections 2024-09-21T02:12:45.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:45.269+0000 7f6ac6ede640 1 -- 172.21.15.92:0/1969755078 wait complete. 2024-09-21T02:12:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:44 smithi136 ceph-mon[34754]: pgmap v593: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:12:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:44 smithi099 ceph-mon[34404]: pgmap v593: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:12:46.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:12:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/3871851531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 msgr2=0x7f2d0810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/3871851531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d0810f520 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009ca0 tx=0x7f2d0002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/3871851531 shutdown_connections 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/3871851531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d0810f520 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/3871851531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d081083a0 0x7f2d081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/3871851531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d08102a60 0x7f2d08102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/3871851531 >> 172.21.15.92:0/3871851531 conn(0x7f2d08079730 msgr2=0x7f2d08079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/3871851531 shutdown_connections 2024-09-21T02:12:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.776+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/3871851531 wait complete. 2024-09-21T02:12:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 Processor -- start 2024-09-21T02:12:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 -- start start 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08102a60 0x7f2d0819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d081083a0 0x7f2d0819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d081a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d08112960 con 0x7f2d08108cf0 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081127e0 con 0x7f2d081083a0 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d0c9d8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d08112ae0 con 0x7f2d08102a60 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d081a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d081a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43774/0 (socket says 172.21.15.92:43774) 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.777+0000 7f2d06d9d640 1 -- 172.21.15.92:0/4253109112 learned_addr learned my addr 172.21.15.92:0/4253109112 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d081083a0 0x7f2d0819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d0659c640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08102a60 0x7f2d0819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d06d9d640 1 -- 172.21.15.92:0/4253109112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08102a60 msgr2=0x7f2d0819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d06d9d640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08102a60 0x7f2d0819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d06d9d640 1 -- 172.21.15.92:0/4253109112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d081083a0 msgr2=0x7f2d0819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d06d9d640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d081083a0 0x7f2d0819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d06d9d640 1 -- 172.21.15.92:0/4253109112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d081a2550 con 0x7f2d08108cf0 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d0659c640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08102a60 0x7f2d0819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d06d9d640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d081a1ee0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005cc0 tx=0x7f2d0002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2cfffff640 1 -- 172.21.15.92:0/4253109112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00003bf0 con 0x7f2d08108cf0 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2cfffff640 1 -- 172.21.15.92:0/4253109112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00003d90 con 0x7f2d08108cf0 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d081a27e0 con 0x7f2d08108cf0 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2cfffff640 1 -- 172.21.15.92:0/4253109112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0002fb20 con 0x7f2d08108cf0 2024-09-21T02:12:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.778+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d081a2d20 con 0x7f2d08108cf0 2024-09-21T02:12:46.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.781+0000 7f2cfffff640 1 -- 172.21.15.92:0/4253109112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2d0002fcc0 con 0x7f2d08108cf0 2024-09-21T02:12:46.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.781+0000 7f2d12f4c740 1 Processor -- start 2024-09-21T02:12:46.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.781+0000 7f2d12f4c740 1 -- start start 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.781+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f3db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b75390 0x558ae5f3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558ae5eb3e50 con 0x558ae5b6d1b0 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ae5ea3280 con 0x558ae5b96070 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ae5ea7090 con 0x558ae5b75390 2024-09-21T02:12:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f3db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f3db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43776/0 (socket says 172.21.15.92:43776) 2024-09-21T02:12:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 -- 172.21.15.92:0/923975722 learned_addr learned my addr 172.21.15.92:0/923975722 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2cfffff640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2cf40b6540 0x7f2cf40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b75390 0x558ae5f3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d06d9d640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2cf40b6540 0x7f2cf40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 -- 172.21.15.92:0/923975722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b75390 msgr2=0x558ae5f3e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b75390 0x558ae5f3e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 -- 172.21.15.92:0/923975722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 msgr2=0x558ae5f3d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f3d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 -- 172.21.15.92:0/923975722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae5c01650 con 0x558ae5b6d1b0 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b75390 0x558ae5f3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2cfffff640 1 -- 172.21.15.92:0/4253109112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2d001086b0 con 0x7f2d08108cf0 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2d0659c640 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f3db90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800bcc0 tx=0x7f2cf800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8478ae268c025e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.782+0000 7f2cfcd99640 1 -- 172.21.15.92:0/923975722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8005020 con 0x558ae5b6d1b0 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2cfcd99640 1 -- 172.21.15.92:0/923975722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf80047a0 con 0x558ae5b6d1b0 2024-09-21T02:12:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2cfcd99640 1 -- 172.21.15.92:0/923975722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8011660 con 0x558ae5b6d1b0 2024-09-21T02:12:46.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d12f4c740 1 -- 172.21.15.92:0/923975722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 msgr2=0x558ae5f3db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d06d9d640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2cf40b6540 0x7f2cf40b8a50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ce520 tx=0x7f2d000ff2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:46.789 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f3db90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800bcc0 tx=0x7f2cf800f730 comp rx=0 tx=0).stop 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d12f4c740 1 -- 172.21.15.92:0/923975722 shutdown_connections 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b75390 0x558ae5f3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f3db90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.783+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/923975722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- 172.21.15.92:0/923975722 >> 172.21.15.92:0/923975722 conn(0x558ae5b8a460 msgr2=0x558ae5f3b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- 172.21.15.92:0/923975722 shutdown_connections 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- 172.21.15.92:0/923975722 wait complete. 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- start start 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 Processor -- start 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f48ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 0x558ae5f49720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558ae5d04700 con 0x558ae5b96070 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ae5e3e110 con 0x558ae5b75390 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ae5f4be30 con 0x558ae5b6d1b0 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.784+0000 7f2d12f4c740 10 client.0 _ll_register_callbacks cb 0x558ae5e57990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 0x558ae5f49720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 0x558ae5f49720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43790/0 (socket says 172.21.15.92:43790) 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1312086325 learned_addr learned my addr 172.21.15.92:0/1312086325 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f48ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 msgr2=0x558ae5f48ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f48ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1312086325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 msgr2=0x558ae5f491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1312086325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae5e6bc40 con 0x558ae5b96070 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:12:46.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 0x558ae5f49720 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x558ae5f2d760 tx=0x7f2cf0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23072c60cbc8dc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cedffb640 1 -- 172.21.15.92:0/1312086325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf001a040 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cedffb640 1 -- 172.21.15.92:0/1312086325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2cf00047a0 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf80047a0 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2cedffb640 1 -- 172.21.15.92:0/1312086325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cf00049a0 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2d12f4c740 10 client.17808 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.785+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ae5f57a80 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2cedffb640 1 -- 172.21.15.92:0/1312086325 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf80047a0 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2cedffb640 1 -- 172.21.15.92:0/1312086325 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2cf002b420 con 0x558ae5b96070 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d12f4c740 10 client.17808 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d12f4c740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d12f4c740 20 client.17808 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b549f0 0x558ae5f5afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d12f4c740 4 client.17808 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ae5eb5da0 con 0x558ae5b549f0 2024-09-21T02:12:46.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.786+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b549f0 0x558ae5f5afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.787+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b549f0 0x558ae5f5afe0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436e73a3831b6387 server_cookie=8b12a5d6be430db3 in_seq=0 out_seq=0 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.787+0000 7f2cedffb640 10 client.17808 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.787+0000 7f2cedffb640 1 -- 172.21.15.92:0/1312086325 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ae5eb5da0 con 0x558ae5b549f0 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.787+0000 7f2cedffb640 10 client.17808 handle_command_reply: tid=0 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.787+0000 7f2d12f4c740 1 client.17808 shutdown 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 msgr2=0x558ae5f49720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 0x558ae5f49720 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x558ae5f2d760 tx=0x7f2cf0013040 comp rx=0 tx=0).stop 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 shutdown_connections 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b549f0 0x558ae5f5afe0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b96070 0x558ae5f49720 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1312086325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f48ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 >> 172.21.15.92:0/1312086325 conn(0x558ae5b8a460 msgr2=0x558ae5f40830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2cedffb640 0 client.17808 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 shutdown_connections 2024-09-21T02:12:46.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1312086325 wait complete. 2024-09-21T02:12:46.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.788+0000 7f2d12f4c740 20 client.17808 trim_cache size 0 max 16384 2024-09-21T02:12:46.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 Processor -- start 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 -- start start 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f4a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f45e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b96070 0x558ae5f463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081b1700 con 0x558ae5b6d1b0 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081b3240 con 0x558ae5b75390 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081b1dd0 con 0x558ae5b96070 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2cfdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f4a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2cfdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f4a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43792/0 (socket says 172.21.15.92:43792) 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.805+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1799830609 learned_addr learned my addr 172.21.15.92:0/1799830609 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1799830609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b96070 msgr2=0x558ae5f463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b96070 0x558ae5f463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1799830609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 msgr2=0x558ae5f45e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f45e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2cfdd9b640 1 -- 172.21.15.92:0/1799830609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae5f49f60 con 0x558ae5b6d1b0 2024-09-21T02:12:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f4a8d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0004700 tx=0x7f2cf000ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55a25110d093120 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2ceffff640 1 -- 172.21.15.92:0/1799830609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0030070 con 0x558ae5b6d1b0 2024-09-21T02:12:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2ceffff640 1 -- 172.21.15.92:0/1799830609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf000fe20 con 0x558ae5b6d1b0 2024-09-21T02:12:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.806+0000 7f2ceffff640 1 -- 172.21.15.92:0/1799830609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0021b90 con 0x558ae5b6d1b0 2024-09-21T02:12:46.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1799830609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 msgr2=0x558ae5f4a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f4a8d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0004700 tx=0x7f2cf000ad40 comp rx=0 tx=0).stop 2024-09-21T02:12:46.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1799830609 shutdown_connections 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b96070 0x558ae5f463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b75390 0x558ae5f45e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1799830609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b6d1b0 0x558ae5f4a8d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1799830609 >> 172.21.15.92:0/1799830609 conn(0x558ae5b8a460 msgr2=0x558ae5f4a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1799830609 shutdown_connections 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1799830609 wait complete. 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- start start 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 Processor -- start 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f51990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 0x558ae5f4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f4d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081b2b70 con 0x558ae5b75390 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081b24a0 con 0x558ae5b96070 2024-09-21T02:12:46.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.807+0000 7f2d12f4c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081af6e0 con 0x558ae5b6d1b0 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d12f4c740 10 client.0 _ll_register_callbacks cb 0x7f2cf000ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 0x558ae5f4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 0x558ae5f4ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43802/0 (socket says 172.21.15.92:43802) 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 -- 172.21.15.92:0/1452334478 learned_addr learned my addr 172.21.15.92:0/1452334478 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f51990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 -- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 msgr2=0x558ae5f51990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f51990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f4d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 -- 172.21.15.92:0/1452334478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 msgr2=0x558ae5f4d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f4d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 -- 172.21.15.92:0/1452334478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae5f4dca0 con 0x558ae5b75390 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f4d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d0659c640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 0x558ae5f4ce90 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800bcc0 tx=0x7f2cf800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6a0e0031d5bf33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:46.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cedffb640 1 -- 172.21.15.92:0/1452334478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf802a030 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cedffb640 1 -- 172.21.15.92:0/1452334478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2cf8004250 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf000fe20 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2cedffb640 1 -- 172.21.15.92:0/1452334478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cf8035070 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d12f4c740 10 client.17814 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.808+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ae5fc5ae0 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2cedffb640 1 -- 172.21.15.92:0/1452334478 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf000fe20 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2cedffb640 1 -- 172.21.15.92:0/1452334478 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2cf8003bd0 con 0x558ae5b75390 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2d12f4c740 10 client.17814 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2d12f4c740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2d12f4c740 20 client.17814 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b29da0 0x558ae5fc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2d12f4c740 4 client.17814 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:12:46.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ae5eb1850 con 0x558ae5b29da0 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.809+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b29da0 0x558ae5fc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2cfdd9b640 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b29da0 0x558ae5fc8020 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443448b4f852efe1 server_cookie=64d8fb52af558d09 in_seq=0 out_seq=0 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2cedffb640 10 client.17814 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2cedffb640 1 -- 172.21.15.92:0/1452334478 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x558ae5eb1850 con 0x558ae5b29da0 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2cedffb640 10 client.17814 handle_command_reply: tid=0 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 client.17814 shutdown 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 msgr2=0x558ae5f4ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 0x558ae5f4ce90 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800bcc0 tx=0x7f2cf800b420 comp rx=0 tx=0).stop 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 shutdown_connections 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x558ae5b29da0 0x558ae5fc8020 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ae5b96070 0x558ae5f4d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2cedffb640 0 client.17814 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:46.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ae5b75390 0x558ae5f4ce90 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 --2- 172.21.15.92:0/1452334478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ae5b6d1b0 0x558ae5f51990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 >> 172.21.15.92:0/1452334478 conn(0x558ae5b8a460 msgr2=0x558ae5f4a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 shutdown_connections 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 1 -- 172.21.15.92:0/1452334478 wait complete. 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.810+0000 7f2d12f4c740 20 client.17814 trim_cache size 0 max 16384 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "4091c28f-03b5-4162-8d4f-dbcbba169f12", 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:12:46.817 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:12:46.818 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.820 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.821 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.822 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:12:46.823 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2cf40b6540 msgr2=0x7f2cf40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2cf40b6540 0x7f2cf40b8a50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ce520 tx=0x7f2d000ff2c0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 msgr2=0x7f2d081a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d081a1ee0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005cc0 tx=0x7f2d0002f380 comp rx=0 tx=0).stop 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 shutdown_connections 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2cf40b6540 0x7f2cf40b8a50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08108cf0 0x7f2d081a1ee0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d081083a0 0x7f2d0819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 --2- 172.21.15.92:0/4253109112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08102a60 0x7f2d0819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 >> 172.21.15.92:0/4253109112 conn(0x7f2d08079730 msgr2=0x7f2d0806af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 shutdown_connections 2024-09-21T02:12:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:12:46.811+0000 7f2d0c9d8640 1 -- 172.21.15.92:0/4253109112 wait complete. 2024-09-21T02:12:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:46 smithi092 ceph-mon[31190]: pgmap v594: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:12:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:46 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:12:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:46 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:46 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:12:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:46 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:46 smithi136 ceph-mon[34754]: pgmap v594: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:12:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:46 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:12:47.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:46 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:47.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:46 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:12:47.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:46 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:47.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:46 smithi099 ceph-mon[34404]: pgmap v594: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:12:47.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:46 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:12:47.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:46 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:47.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:46 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:12:47.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:46 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:12:47.602 INFO:tasks.cephfs.filesystem:scrub status for tag:4091c28f-03b5-4162-8d4f-dbcbba169f12 - {'path': '/', 'tag': '4091c28f-03b5-4162-8d4f-dbcbba169f12', 'options': 'recursive,force'} 2024-09-21T02:12:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.450+0000 7f19bf512640 1 -- 172.21.15.92:0/56148063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 msgr2=0x7f19b8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.450+0000 7f19bf512640 1 --2- 172.21.15.92:0/56148063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8102ef0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009900 tx=0x7f19b402a410 comp rx=0 tx=0).stop 2024-09-21T02:12:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 -- 172.21.15.92:0/56148063 shutdown_connections 2024-09-21T02:12:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 --2- 172.21.15.92:0/56148063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19b81040d0 0x7f19b810ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 --2- 172.21.15.92:0/56148063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 0x7f19b8103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 --2- 172.21.15.92:0/56148063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8102ef0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 -- 172.21.15.92:0/56148063 >> 172.21.15.92:0/56148063 conn(0x7f19b80fd6d0 msgr2=0x7f19b80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 -- 172.21.15.92:0/56148063 shutdown_connections 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 -- 172.21.15.92:0/56148063 wait complete. 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 Processor -- start 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.451+0000 7f19bf512640 1 -- start start 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 0x7f19b8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19b81040d0 0x7f19b819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b810e200 con 0x7f19b8102ae0 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19b810e080 con 0x7f19b81034c0 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19b810e380 con 0x7f19b81040d0 2024-09-21T02:12:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43818/0 (socket says 172.21.15.92:43818) 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 -- 172.21.15.92:0/459456301 learned_addr learned my addr 172.21.15.92:0/459456301 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19b81040d0 0x7f19b819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bcaad640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 0x7f19b8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 -- 172.21.15.92:0/459456301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19b81040d0 msgr2=0x7f19b819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19b81040d0 0x7f19b819a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 -- 172.21.15.92:0/459456301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 msgr2=0x7f19b8199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 0x7f19b8199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 -- 172.21.15.92:0/459456301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b819a860 con 0x7f19b8102ae0 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bcaad640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 0x7f19b8199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8198e20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006010 tx=0x7f19b4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19ae7fc640 1 -- 172.21.15.92:0/459456301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b403f030 con 0x7f19b8102ae0 2024-09-21T02:12:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b819aaf0 con 0x7f19b8102ae0 2024-09-21T02:12:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19ae7fc640 1 -- 172.21.15.92:0/459456301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b4003aa0 con 0x7f19b8102ae0 2024-09-21T02:12:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19ae7fc640 1 -- 172.21.15.92:0/459456301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4035070 con 0x7f19b8102ae0 2024-09-21T02:12:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.452+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b819ae00 con 0x7f19b8102ae0 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 Processor -- start 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 -- start start 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19ae7fc640 1 -- 172.21.15.92:0/459456301 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f19b4042030 con 0x7f19b8102ae0 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 0x5565659ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55656591a6f0 con 0x5565655df4d0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556565909b20 con 0x5565655d72f0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19c6478740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55656590d930 con 0x5565656001b0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19bcaad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 0x5565659ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.455+0000 7f19bcaad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 0x5565659ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43822/0 (socket says 172.21.15.92:43822) 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 -- 172.21.15.92:0/3070007592 learned_addr learned my addr 172.21.15.92:0/3070007592 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 -- 172.21.15.92:0/3070007592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 msgr2=0x5565659aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 -- 172.21.15.92:0/3070007592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 msgr2=0x5565659aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 -- 172.21.15.92:0/3070007592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565658cf530 con 0x5565655df4d0 2024-09-21T02:12:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19ae7fc640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19a40b6510 0x7f19a40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bcaad640 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 0x5565659ab220 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f19a800bb70 tx=0x7f19a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c86c7d4f7398a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19a40b6510 0x7f19a40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19a37fe640 1 -- 172.21.15.92:0/3070007592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8017620 con 0x5565655df4d0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19ae7fc640 1 -- 172.21.15.92:0/459456301 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f19b41079b0 con 0x7f19b8102ae0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19a37fe640 1 -- 172.21.15.92:0/3070007592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8017c70 con 0x5565655df4d0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.456+0000 7f19a37fe640 1 -- 172.21.15.92:0/3070007592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a800fc00 con 0x5565655df4d0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19a40b6510 0x7f19a40b8a20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005fd0 tx=0x7f19b0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 -- 172.21.15.92:0/3070007592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 msgr2=0x5565659ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 0x5565659ab220 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f19a800bb70 tx=0x7f19a800d710 comp rx=0 tx=0).stop 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 -- 172.21.15.92:0/3070007592 shutdown_connections 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655df4d0 0x5565659ab220 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 --2- 172.21.15.92:0/3070007592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 -- 172.21.15.92:0/3070007592 >> 172.21.15.92:0/3070007592 conn(0x5565655f4300 msgr2=0x5565659a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 -- 172.21.15.92:0/3070007592 shutdown_connections 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.457+0000 7f19c6478740 1 -- 172.21.15.92:0/3070007592 wait complete. 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 -- start start 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 Processor -- start 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655df4d0 0x5565659b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556565841ab0 con 0x5565656001b0 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565658af730 con 0x5565655d72f0 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5565659abd40 con 0x5565655df4d0 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19c6478740 10 client.0 _ll_register_callbacks cb 0x5565659036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19bdaaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19bdaaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43834/0 (socket says 172.21.15.92:43834) 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19bdaaf640 1 -- 172.21.15.92:0/2540789841 learned_addr learned my addr 172.21.15.92:0/2540789841 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19bcaad640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655df4d0 0x5565659b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.458+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19bdaaf640 1 -- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655df4d0 msgr2=0x5565659b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655df4d0 0x5565659b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19bdaaf640 1 -- 172.21.15.92:0/2540789841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 msgr2=0x5565659b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19bdaaf640 1 -- 172.21.15.92:0/2540789841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656566b790 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659b4a50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f19b001f6a0 tx=0x7f19b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47ed8bf15983123 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19a0ff9640 1 -- 172.21.15.92:0/2540789841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b002c060 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19a0ff9640 1 -- 172.21.15.92:0/2540789841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f19b0017030 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a8017c70 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19a0ff9640 1 -- 172.21.15.92:0/2540789841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19b0011810 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19c6478740 10 client.17826 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565656e4620 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19a0ff9640 1 -- 172.21.15.92:0/2540789841 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a8017c70 con 0x5565656001b0 2024-09-21T02:12:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19a0ff9640 1 -- 172.21.15.92:0/2540789841 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19b0027b40 con 0x5565656001b0 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19c6478740 10 client.17826 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19c6478740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.459+0000 7f19c6478740 20 client.17826 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.460+0000 7f19c6478740 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655bed80 0x5565659c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.460+0000 7f19c6478740 4 client.17826 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.460+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55656591c640 con 0x5565655bed80 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.460+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655bed80 0x5565659c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.460+0000 7f19a0ff9640 10 client.17826 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.460+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655bed80 0x5565659c1070 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b3764ea96ab0f9 server_cookie=5ac099f04a73f4b5 in_seq=0 out_seq=0 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19a0ff9640 1 -- 172.21.15.92:0/2540789841 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55656591c640 con 0x5565655bed80 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19a0ff9640 10 client.17826 handle_command_reply: tid=0 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 client.17826 shutdown 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 msgr2=0x5565659b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659b4a50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f19b001f6a0 tx=0x7f19b0004ab0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 shutdown_connections 2024-09-21T02:12:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655bed80 0x5565659c1070 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659b4a50 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655df4d0 0x5565659b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19a0ff9640 0 client.17826 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 --2- 172.21.15.92:0/2540789841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655d72f0 0x5565659b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 >> 172.21.15.92:0/2540789841 conn(0x5565655f4300 msgr2=0x5565659a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 shutdown_connections 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 1 -- 172.21.15.92:0/2540789841 wait complete. 2024-09-21T02:12:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.461+0000 7f19c6478740 20 client.17826 trim_cache size 0 max 16384 2024-09-21T02:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 Processor -- start 2024-09-21T02:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 -- start start 2024-09-21T02:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 0x5565659b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81a8e00 con 0x5565655d72f0 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81aa940 con 0x5565655df4d0 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19c6478740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81a94d0 con 0x5565656001b0 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 0x5565659b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 0x5565659b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43844/0 (socket says 172.21.15.92:43844) 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 -- 172.21.15.92:0/4139192277 learned_addr learned my addr 172.21.15.92:0/4139192277 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bcaad640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 -- 172.21.15.92:0/4139192277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 msgr2=0x5565659b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 -- 172.21.15.92:0/4139192277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 msgr2=0x5565659b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 -- 172.21.15.92:0/4139192277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565659b19b0 con 0x5565655d72f0 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bcaad640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 0x5565659b2a40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f19b40ca2e0 tx=0x7f19b4105360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10418af53d0ddc25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.476+0000 7f19a2ffd640 1 -- 172.21.15.92:0/4139192277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4107e60 con 0x5565655d72f0 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19a2ffd640 1 -- 172.21.15.92:0/4139192277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b40cee50 con 0x5565655d72f0 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19a2ffd640 1 -- 172.21.15.92:0/4139192277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4049040 con 0x5565655d72f0 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- 172.21.15.92:0/4139192277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 msgr2=0x5565659b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 0x5565659b2a40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f19b40ca2e0 tx=0x7f19b4105360 comp rx=0 tx=0).stop 2024-09-21T02:12:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- 172.21.15.92:0/4139192277 shutdown_connections 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565656001b0 0x5565659b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- 172.21.15.92:0/4139192277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565655d72f0 0x5565659b2a40 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- 172.21.15.92:0/4139192277 >> 172.21.15.92:0/4139192277 conn(0x5565655f4300 msgr2=0x5565659a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- 172.21.15.92:0/4139192277 shutdown_connections 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- 172.21.15.92:0/4139192277 wait complete. 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- start start 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 Processor -- start 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655d72f0 0x5565659bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81aa270 con 0x5565656001b0 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81a9ba0 con 0x5565655df4d0 2024-09-21T02:12:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81a6de0 con 0x5565655d72f0 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19c6478740 10 client.0 _ll_register_callbacks cb 0x7f19b0009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19bdaaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19bdaaf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659ba9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43850/0 (socket says 172.21.15.92:43850) 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19bdaaf640 1 -- 172.21.15.92:0/1670999290 learned_addr learned my addr 172.21.15.92:0/1670999290 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655d72f0 0x5565659bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.477+0000 7f19bcaad640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bdaaf640 1 -- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655d72f0 msgr2=0x5565659bb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655d72f0 0x5565659bb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bdaaf640 1 -- 172.21.15.92:0/1670999290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 msgr2=0x5565659ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bdaaf640 1 -- 172.21.15.92:0/1670999290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565659bb100 con 0x5565656001b0 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655d72f0 0x5565659bb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bcaad640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19bdaaf640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659ba9a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f19b001ff30 tx=0x7f19b0011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd485c6d3e3d26a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19a0ff9640 1 -- 172.21.15.92:0/1670999290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0032b50 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19a0ff9640 1 -- 172.21.15.92:0/1670999290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f19b0032cf0 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b40cee50 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19a0ff9640 1 -- 172.21.15.92:0/1670999290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19b002c020 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 10 client.17832 fetch_fsmap learned FSMap version 29 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556565a2ba30 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19a0ff9640 1 -- 172.21.15.92:0/1670999290 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b40cee50 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19a0ff9640 1 -- 172.21.15.92:0/1670999290 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19b001c510 con 0x5565656001b0 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 10 client.17832 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:12:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 20 client.17832 populate_metadata read hostname 'smithi092' 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655941c0 0x556565a2e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 4 client.17832 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.478+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565659a2440 con 0x5565655941c0 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.479+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655941c0 0x556565a2e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.479+0000 7f19bd2ae640 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655941c0 0x556565a2e400 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b2b7f3bbce1f16 server_cookie=959e983fe7c32784 in_seq=0 out_seq=0 2024-09-21T02:12:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.479+0000 7f19a0ff9640 10 client.17832 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19a0ff9640 1 -- 172.21.15.92:0/1670999290 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565659a2440 con 0x5565655941c0 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19a0ff9640 10 client.17832 handle_command_reply: tid=0 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 client.17832 shutdown 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 msgr2=0x5565659ba9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659ba9a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f19b001ff30 tx=0x7f19b0011be0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 shutdown_connections 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5565655941c0 0x556565a2e400 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565656001b0 0x5565659ba9a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565655df4d0 0x5565659ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 --2- 172.21.15.92:0/1670999290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565655d72f0 0x5565659bb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19a0ff9640 0 client.17832 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 >> 172.21.15.92:0/1670999290 conn(0x5565655f4300 msgr2=0x5565659a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 shutdown_connections 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 1 -- 172.21.15.92:0/1670999290 wait complete. 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.493+0000 7f19c6478740 20 client.17832 trim_cache size 0 max 16384 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:12:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:12:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:12:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19a40b6510 msgr2=0x7f19a40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19a40b6510 0x7f19a40b8a20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005fd0 tx=0x7f19b0009500 comp rx=0 tx=0).stop 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 msgr2=0x7f19b8198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8198e20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006010 tx=0x7f19b4031a50 comp rx=0 tx=0).stop 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 shutdown_connections 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f19a40b6510 0x7f19a40b8a20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19b81040d0 0x7f19b819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19b81034c0 0x7f19b8199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 --2- 172.21.15.92:0/459456301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19b8102ae0 0x7f19b8198e20 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:12:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 >> 172.21.15.92:0/459456301 conn(0x7f19b80fd6d0 msgr2=0x7f19b8106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:12:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 shutdown_connections 2024-09-21T02:12:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:12:48.494+0000 7f19bf512640 1 -- 172.21.15.92:0/459456301 wait complete. 2024-09-21T02:12:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:48 smithi136 ceph-mon[34754]: pgmap v595: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:48 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:12:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:48 smithi092 ceph-mon[31190]: pgmap v595: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:48 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:12:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:48 smithi099 ceph-mon[34404]: pgmap v595: 97 pgs: 97 active+clean; 1.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:12:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:48 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:375/600: open fd = 3 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:376/600: open fd = 3 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:377/600: open fd = 3 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:378/600: open fd = 3 2024-09-21T02:12:51.204 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:379/600: open fd = 3 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:380/600: open fd = 3 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:381/600: open fd = 3 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:382/600: open fd = 3 2024-09-21T02:12:51.205 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:383/600: open fd = 3 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:384/600: open fd = 3 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:385/600: open fd = 3 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.206 INFO:tasks.workunit.client.0.smithi092.stdout:386/600: open fd = 3 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:387/600: open fd = 3 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:388/600: open fd = 3 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:389/600: open fd = 3 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.207 INFO:tasks.workunit.client.0.smithi092.stdout:390/600: open fd = 3 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:391/600: open fd = 3 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:392/600: open fd = 3 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:393/600: open fd = 3 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:394/600: open fd = 3 2024-09-21T02:12:51.208 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:395/600: open fd = 3 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:396/600: open fd = 3 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:397/600: open fd = 3 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:398/600: open fd = 3 2024-09-21T02:12:51.209 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:399/600: open fd = 3 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:400/600: open fd = 3 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:401/600: open fd = 3 2024-09-21T02:12:51.210 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:402/600: open fd = 3 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:403/600: open fd = 3 2024-09-21T02:12:51.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:404/600: open fd = 3 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:405/600: open fd = 3 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:406/600: open fd = 3 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.212 INFO:tasks.workunit.client.0.smithi092.stdout:407/600: open fd = 3 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:408/600: open fd = 3 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:409/600: open fd = 3 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:410/600: open fd = 3 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.213 INFO:tasks.workunit.client.0.smithi092.stdout:411/600: open fd = 3 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:412/600: open fd = 3 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:413/600: open fd = 3 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:414/600: open fd = 3 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.214 INFO:tasks.workunit.client.0.smithi092.stdout:415/600: open fd = 3 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:416/600: open fd = 3 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:417/600: open fd = 3 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:418/600: open fd = 3 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:419/600: open fd = 3 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:420/600: open fd = 3 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:421/600: open fd = 3 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:422/600: open fd = 3 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:423/600: open fd = 3 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:424/600: open fd = 3 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:425/600: open fd = 3 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:426/600: open fd = 3 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:427/600: open fd = 3 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:428/600: open fd = 3 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:429/600: open fd = 3 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.218 INFO:tasks.workunit.client.0.smithi092.stdout:430/600: open fd = 3 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:431/600: open fd = 3 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:432/600: open fd = 3 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:433/600: open fd = 3 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:434/600: open fd = 3 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:435/600: open fd = 3 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:436/600: open fd = 3 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:12:51.220 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:12:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:50 smithi136 ceph-mon[34754]: pgmap v596: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:12:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:50 smithi092 ceph-mon[31190]: pgmap v596: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:12:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:50 smithi099 ceph-mon[34404]: pgmap v596: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:12:53.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:52 smithi136 ceph-mon[34754]: pgmap v597: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:12:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:52 smithi092 ceph-mon[31190]: pgmap v597: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:12:53.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:52 smithi099 ceph-mon[34404]: pgmap v597: 97 pgs: 97 active+clean; 2.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-21T02:12:55.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:54 smithi136 ceph-mon[34754]: pgmap v598: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:12:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:54 smithi092 ceph-mon[31190]: pgmap v598: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:12:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:54 smithi099 ceph-mon[34404]: pgmap v598: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:12:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:56 smithi136 ceph-mon[34754]: pgmap v599: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:12:57.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:56 smithi092 ceph-mon[31190]: pgmap v599: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:12:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:56 smithi099 ceph-mon[34404]: pgmap v599: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:12:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:12:58 smithi136 ceph-mon[34754]: pgmap v600: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:59.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:12:58 smithi092 ceph-mon[31190]: pgmap v600: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:12:59.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:12:58 smithi099 ceph-mon[34404]: pgmap v600: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:13:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:00 smithi136 ceph-mon[34754]: pgmap v601: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:13:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:00 smithi092 ceph-mon[31190]: pgmap v601: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:13:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:00 smithi099 ceph-mon[34404]: pgmap v601: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:13:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 -- 172.21.15.92:0/1727626636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1201025c0 msgr2=0x7fa120102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 --2- 172.21.15.92:0/1727626636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1201025c0 0x7fa120102a30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa110009900 tx=0x7fa11002a410 comp rx=0 tx=0).stop 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 -- 172.21.15.92:0/1727626636 shutdown_connections 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 --2- 172.21.15.92:0/1727626636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa120103fe0 0x7fa12010a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 --2- 172.21.15.92:0/1727626636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa120103640 0x7fa120103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 --2- 172.21.15.92:0/1727626636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1201025c0 0x7fa120102a30 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 -- 172.21.15.92:0/1727626636 >> 172.21.15.92:0/1727626636 conn(0x7fa1200fdf00 msgr2=0x7fa1201003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 -- 172.21.15.92:0/1727626636 shutdown_connections 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.602+0000 7fa125be4640 1 -- 172.21.15.92:0/1727626636 wait complete. 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 Processor -- start 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 -- start start 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1201025c0 0x7fa120199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 0x7fa12019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa120103fe0 0x7fa12019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa12010e200 con 0x7fa120103640 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa12010e080 con 0x7fa120103fe0 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa125be4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa12010e380 con 0x7fa1201025c0 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 0x7fa12019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 0x7fa12019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43932/0 (socket says 172.21.15.92:43932) 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 -- 172.21.15.92:0/3894948827 learned_addr learned my addr 172.21.15.92:0/3894948827 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1201025c0 0x7fa120199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa11ffff640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa120103fe0 0x7fa12019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 -- 172.21.15.92:0/3894948827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1201025c0 msgr2=0x7fa120199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1201025c0 0x7fa120199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 -- 172.21.15.92:0/3894948827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa120103fe0 msgr2=0x7fa12019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa120103fe0 0x7fa12019b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 -- 172.21.15.92:0/3894948827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12019ba00 con 0x7fa120103640 2024-09-21T02:13:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.603+0000 7fa117fff640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 0x7fa12019a630 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa10800d950 tx=0x7fa10800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.604+0000 7fa11d7fa640 1 -- 172.21.15.92:0/3894948827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108017070 con 0x7fa120103640 2024-09-21T02:13:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.604+0000 7fa11d7fa640 1 -- 172.21.15.92:0/3894948827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa108009c30 con 0x7fa120103640 2024-09-21T02:13:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.604+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12019bcf0 con 0x7fa120103640 2024-09-21T02:13:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.604+0000 7fa11d7fa640 1 -- 172.21.15.92:0/3894948827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10800f620 con 0x7fa120103640 2024-09-21T02:13:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.604+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1201a23e0 con 0x7fa120103640 2024-09-21T02:13:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 Processor -- start 2024-09-21T02:13:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 -- start start 2024-09-21T02:13:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 0x5631cc1cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbdfb2f0 0x5631cc1cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe034d0 0x5631cc1cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5631cc13e6f0 con 0x5631cbe241b0 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5631cc12db20 con 0x5631cbe034d0 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa12cc31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631cc131930 con 0x5631cbdfb2f0 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11d7fa640 1 -- 172.21.15.92:0/3894948827 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa108010460 con 0x7fa120103640 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 0x5631cc1cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 0x5631cc1cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43934/0 (socket says 172.21.15.92:43934) 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 -- 172.21.15.92:0/3979176849 learned_addr learned my addr 172.21.15.92:0/3979176849 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbdfb2f0 0x5631cc1cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 -- 172.21.15.92:0/3979176849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbdfb2f0 msgr2=0x5631cc1cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbdfb2f0 0x5631cc1cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 -- 172.21.15.92:0/3979176849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe034d0 msgr2=0x5631cc1cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.607+0000 7fa11ffff640 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe034d0 0x5631cc1cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa11ffff640 1 -- 172.21.15.92:0/3979176849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cc0f3530 con 0x5631cbe241b0 2024-09-21T02:13:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbdfb2f0 0x5631cc1cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa11ffff640 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 0x5631cc1cbe50 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa118009df0 tx=0x7fa11800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0583a0ebf3cdef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa115ffb640 1 -- 172.21.15.92:0/3979176849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118003970 con 0x5631cbe241b0 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa115ffb640 1 -- 172.21.15.92:0/3979176849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa118004030 con 0x5631cbe241b0 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa115ffb640 1 -- 172.21.15.92:0/3979176849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118011720 con 0x5631cbe241b0 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa11d7fa640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa10c0b6510 0x7fa10c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa10c0b6510 0x7fa10c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.608+0000 7fa11d7fa640 1 -- 172.21.15.92:0/3894948827 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa1080e6f60 con 0x7fa120103640 2024-09-21T02:13:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 -- 172.21.15.92:0/3979176849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 msgr2=0x5631cc1cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa10c0b6510 0x7fa10c0b8a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa110031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 0x5631cc1cbe50 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa118009df0 tx=0x7fa11800f730 comp rx=0 tx=0).stop 2024-09-21T02:13:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 -- 172.21.15.92:0/3979176849 shutdown_connections 2024-09-21T02:13:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe034d0 0x5631cc1cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbdfb2f0 0x5631cc1cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3979176849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe241b0 0x5631cc1cbe50 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 -- 172.21.15.92:0/3979176849 >> 172.21.15.92:0/3979176849 conn(0x5631cbe18300 msgr2=0x5631cc1c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 -- 172.21.15.92:0/3979176849 shutdown_connections 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 -- 172.21.15.92:0/3979176849 wait complete. 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.609+0000 7fa12cc31740 1 -- start start 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 Processor -- start 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5631cc065ab0 con 0x5631cbe034d0 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5631cc0d3730 con 0x5631cbdfb2f0 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631cc1cdb30 con 0x5631cbe241b0 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa12cc31740 10 client.0 _ll_register_callbacks cb 0x5631cc1276b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43938/0 (socket says 172.21.15.92:43938) 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 -- 172.21.15.92:0/3850107718 learned_addr learned my addr 172.21.15.92:0/3850107718 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa11ffff640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 -- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 msgr2=0x5631cc1d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 -- 172.21.15.92:0/3850107718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 msgr2=0x5631cc1d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 -- 172.21.15.92:0/3850107718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cbe8f790 con 0x5631cbe034d0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa117fff640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d5d00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa1080acdd0 tx=0x7fa1080acfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c472e6d395584da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.610+0000 7fa0feffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa0feffd640 1 -- 172.21.15.92:0/3850107718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108020070 con 0x5631cbe034d0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa0feffd640 1 -- 172.21.15.92:0/3850107718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa1080e90d0 con 0x5631cbe034d0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa118004030 con 0x5631cbe034d0 2024-09-21T02:13:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa0feffd640 1 -- 172.21.15.92:0/3850107718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1080acbe0 con 0x5631cbe034d0 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 10 client.17844 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631cbf08620 con 0x5631cbe034d0 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa0feffd640 1 -- 172.21.15.92:0/3850107718 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa118004030 con 0x5631cbe034d0 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa0feffd640 1 -- 172.21.15.92:0/3850107718 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa1080abda0 con 0x5631cbe034d0 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 10 client.17844 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 10 client.17844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 20 client.17844 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbde2d80 0x5631cc1e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 4 client.17844 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.611+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631cc140640 con 0x5631cbde2d80 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.612+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbde2d80 0x5631cc1e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.612+0000 7fa0feffd640 10 client.17844 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.612+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbde2d80 0x5631cc1e2bf0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1592d85c65026f server_cookie=bb0a902df8d80458 in_seq=0 out_seq=0 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa0feffd640 1 -- 172.21.15.92:0/3850107718 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631cc140640 con 0x5631cbde2d80 2024-09-21T02:13:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa0feffd640 10 client.17844 handle_command_reply: tid=0 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 client.17844 shutdown 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 msgr2=0x5631cc1d5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d5d00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa1080acdd0 tx=0x7fa1080acfd0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 shutdown_connections 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbde2d80 0x5631cc1e2bf0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d5d00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa0feffd640 0 client.17844 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 --2- 172.21.15.92:0/3850107718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 >> 172.21.15.92:0/3850107718 conn(0x5631cbe18300 msgr2=0x5631cc1c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 shutdown_connections 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 1 -- 172.21.15.92:0/3850107718 wait complete. 2024-09-21T02:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.613+0000 7fa12cc31740 20 client.17844 trim_cache size 0 max 16384 2024-09-21T02:13:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.627+0000 7fa12cc31740 1 Processor -- start 2024-09-21T02:13:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.627+0000 7fa12cc31740 1 -- start start 2024-09-21T02:13:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe034d0 0x5631cc1d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe241b0 0x5631cc1cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa12cc31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201ab090 con 0x5631cbdfb2f0 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa12cc31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201acbd0 con 0x5631cbe241b0 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa12cc31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201ab760 con 0x5631cbe034d0 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43952/0 (socket says 172.21.15.92:43952) 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 -- 172.21.15.92:0/978687494 learned_addr learned my addr 172.21.15.92:0/978687494 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 -- 172.21.15.92:0/978687494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe034d0 msgr2=0x5631cc1d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe034d0 0x5631cc1d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 -- 172.21.15.92:0/978687494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe241b0 msgr2=0x5631cc1cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe241b0 0x5631cc1cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 -- 172.21.15.92:0/978687494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cc1c6fd0 con 0x5631cbdfb2f0 2024-09-21T02:13:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d3b50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa11002a930 tx=0x7fa11003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4141803963cb6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa1157fa640 1 -- 172.21.15.92:0/978687494 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100037f0 con 0x5631cbdfb2f0 2024-09-21T02:13:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa1157fa640 1 -- 172.21.15.92:0/978687494 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa110057030 con 0x5631cbdfb2f0 2024-09-21T02:13:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.628+0000 7fa1157fa640 1 -- 172.21.15.92:0/978687494 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11003db40 con 0x5631cbdfb2f0 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 -- 172.21.15.92:0/978687494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 msgr2=0x5631cc1d3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d3b50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa11002a930 tx=0x7fa11003f000 comp rx=0 tx=0).stop 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 -- 172.21.15.92:0/978687494 shutdown_connections 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbe241b0 0x5631cc1cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe034d0 0x5631cc1d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 --2- 172.21.15.92:0/978687494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbdfb2f0 0x5631cc1d3b50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 -- 172.21.15.92:0/978687494 >> 172.21.15.92:0/978687494 conn(0x5631cbe18300 msgr2=0x5631cc1d5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 -- 172.21.15.92:0/978687494 shutdown_connections 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 -- 172.21.15.92:0/978687494 wait complete. 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.629+0000 7fa12cc31740 1 -- start start 2024-09-21T02:13:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 Processor -- start 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201ac500 con 0x5631cbe034d0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201abe30 con 0x5631cbdfb2f0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201a9070 con 0x5631cbe241b0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa12cc31740 10 client.0 _ll_register_callbacks cb 0x7fa1080a9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43960/0 (socket says 172.21.15.92:43960) 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 -- 172.21.15.92:0/2449305104 learned_addr learned my addr 172.21.15.92:0/2449305104 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa11ffff640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 -- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 msgr2=0x5631cc1d9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 -- 172.21.15.92:0/2449305104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 msgr2=0x5631cc1dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1dd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.630+0000 7fa117fff640 1 -- 172.21.15.92:0/2449305104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cc1da360 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa117fff640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d91f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa1080aa340 tx=0x7fa108010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b425df4af438b2a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa0feffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa0feffd640 1 -- 172.21.15.92:0/2449305104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108020030 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa0feffd640 1 -- 172.21.15.92:0/2449305104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa108010750 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa110057030 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa0feffd640 1 -- 172.21.15.92:0/2449305104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1080b6b40 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 10 client.17850 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631cc24d7a0 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa0feffd640 1 -- 172.21.15.92:0/2449305104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa110057030 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa0feffd640 1 -- 172.21.15.92:0/2449305104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa10805c1a0 con 0x5631cbe034d0 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 10 client.17850 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 10 client.17850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 20 client.17850 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbdb81c0 0x5631cc250210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 4 client.17850 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.631+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631cc13c0f0 con 0x5631cbdb81c0 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.632+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbdb81c0 0x5631cc250210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.632+0000 7fa0feffd640 10 client.17850 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.632+0000 7fa11f7fe640 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbdb81c0 0x5631cc250210 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c8008a2e19d037 server_cookie=da93a310bb234d4 in_seq=0 out_seq=0 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa0feffd640 1 -- 172.21.15.92:0/2449305104 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631cc13c0f0 con 0x5631cbdb81c0 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa0feffd640 10 client.17850 handle_command_reply: tid=0 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 client.17850 shutdown 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 msgr2=0x5631cc1d91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d91f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa1080aa340 tx=0x7fa108010cb0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 shutdown_connections 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5631cbdb81c0 0x5631cc250210 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631cbe241b0 0x5631cc1d9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa0feffd640 0 client.17850 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631cbe034d0 0x5631cc1d91f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 --2- 172.21.15.92:0/2449305104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631cbdfb2f0 0x5631cc1dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 >> 172.21.15.92:0/2449305104 conn(0x5631cbe18300 msgr2=0x5631cc1d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 shutdown_connections 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 1 -- 172.21.15.92:0/2449305104 wait complete. 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.648+0000 7fa12cc31740 20 client.17850 trim_cache size 0 max 16384 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:13:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:13:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:13:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa10c0b6510 msgr2=0x7fa10c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa10c0b6510 0x7fa10c0b8a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa110031400 comp rx=0 tx=0).stop 2024-09-21T02:13:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 msgr2=0x7fa12019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 0x7fa12019a630 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa10800d950 tx=0x7fa10800de10 comp rx=0 tx=0).stop 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 shutdown_connections 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa10c0b6510 0x7fa10c0b8a20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa120103fe0 0x7fa12019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120103640 0x7fa12019a630 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 --2- 172.21.15.92:0/3894948827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1201025c0 0x7fa120199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 >> 172.21.15.92:0/3894948827 conn(0x7fa1200fdf00 msgr2=0x7fa1200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 shutdown_connections 2024-09-21T02:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:01.649+0000 7fa125be4640 1 -- 172.21.15.92:0/3894948827 wait complete. 2024-09-21T02:13:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:02 smithi136 ceph-mon[34754]: pgmap v602: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:03.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:02 smithi092 ceph-mon[31190]: pgmap v602: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:03.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:02 smithi099 ceph-mon[34404]: pgmap v602: 97 pgs: 97 active+clean; 2.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:04 smithi136 ceph-mon[34754]: pgmap v603: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-09-21T02:13:05.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:04 smithi092 ceph-mon[31190]: pgmap v603: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-09-21T02:13:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:04 smithi099 ceph-mon[34404]: pgmap v603: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-09-21T02:13:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:06 smithi136 ceph-mon[34754]: pgmap v604: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:13:07.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:06 smithi092 ceph-mon[31190]: pgmap v604: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:13:07.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:06 smithi099 ceph-mon[34404]: pgmap v604: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:13:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:08 smithi136 ceph-mon[34754]: pgmap v605: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:09.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:08 smithi092 ceph-mon[31190]: pgmap v605: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:08 smithi099 ceph-mon[34404]: pgmap v605: 97 pgs: 97 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:10 smithi136 ceph-mon[34754]: pgmap v606: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:10 smithi092 ceph-mon[31190]: pgmap v606: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:11.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:10 smithi099 ceph-mon[34404]: pgmap v606: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:12 smithi136 ceph-mon[34754]: pgmap v607: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:12 smithi092 ceph-mon[31190]: pgmap v607: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:12 smithi099 ceph-mon[34404]: pgmap v607: 97 pgs: 97 active+clean; 2.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 -- 172.21.15.92:0/881428532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0134104630 msgr2=0x7f0134106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 --2- 172.21.15.92:0/881428532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0134104630 0x7f0134106ab0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0128009900 tx=0x7f012802a410 comp rx=0 tx=0).stop 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 -- 172.21.15.92:0/881428532 shutdown_connections 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 --2- 172.21.15.92:0/881428532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01341071f0 0x7f01341076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 --2- 172.21.15.92:0/881428532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0134104630 0x7f0134106ab0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 --2- 172.21.15.92:0/881428532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013406c4d0 0x7f01341040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 -- 172.21.15.92:0/881428532 >> 172.21.15.92:0/881428532 conn(0x7f01340fdf00 msgr2=0x7f01341003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 -- 172.21.15.92:0/881428532 shutdown_connections 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.762+0000 7f013b4b2640 1 -- 172.21.15.92:0/881428532 wait complete. 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 Processor -- start 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 -- start start 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013406c4d0 0x7f01341a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0134104630 0x7f01341a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 0x7f01341a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0134115900 con 0x7f01341071f0 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0134115780 con 0x7f0134104630 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013b4b2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0134115a80 con 0x7f013406c4d0 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f0139a4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 0x7f01341a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f0139a4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 0x7f01341a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43630/0 (socket says 172.21.15.92:43630) 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f0139a4f640 1 -- 172.21.15.92:0/2804758838 learned_addr learned my addr 172.21.15.92:0/2804758838 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f0138a4d640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0134104630 0x7f01341a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.763+0000 7f013924e640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013406c4d0 0x7f01341a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f0139a4f640 1 -- 172.21.15.92:0/2804758838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013406c4d0 msgr2=0x7f01341a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f0139a4f640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013406c4d0 0x7f01341a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f0139a4f640 1 -- 172.21.15.92:0/2804758838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0134104630 msgr2=0x7f01341a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f0139a4f640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0134104630 0x7f01341a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f0139a4f640 1 -- 172.21.15.92:0/2804758838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01341a2050 con 0x7f01341071f0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f0139a4f640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 0x7f01341a1830 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f012c00cad0 tx=0x7f012c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f01327fc640 1 -- 172.21.15.92:0/2804758838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00ae20 con 0x7f01341071f0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f01327fc640 1 -- 172.21.15.92:0/2804758838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c005020 con 0x7f01341071f0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01341a2340 con 0x7f01341071f0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f01327fc640 1 -- 172.21.15.92:0/2804758838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00f6b0 con 0x7f01341071f0 2024-09-21T02:13:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.764+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341a8a30 con 0x7f01341071f0 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f014241d740 1 Processor -- start 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f01327fc640 1 -- 172.21.15.92:0/2804758838 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f012c0051d0 con 0x7f01341071f0 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f014241d740 1 -- start start 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f014241d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f16fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f014241d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f1704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f014241d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f171010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.767+0000 7f014241d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a80f0e26f0 con 0x55a80edc81b0 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f014241d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a80f0d1b20 con 0x55a80eda74d0 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f014241d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a80f0d5930 con 0x55a80ed9f2f0 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f16fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f16fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43638/0 (socket says 172.21.15.92:43638) 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 -- 172.21.15.92:0/1399556599 learned_addr learned my addr 172.21.15.92:0/1399556599 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f013924e640 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f1704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f171010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f01327fc640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f011c0b6590 0x7f011c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 -- 172.21.15.92:0/1399556599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 msgr2=0x55a80f1704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f1704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 -- 172.21.15.92:0/1399556599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 msgr2=0x55a80f171010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f171010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 -- 172.21.15.92:0/1399556599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80f097530 con 0x55a80edc81b0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0139a4f640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f011c0b6590 0x7f011c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f01327fc640 1 -- 172.21.15.92:0/2804758838 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f012c0e7270 con 0x7f01341071f0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.768+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f16fe50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f01280096c0 tx=0x7f0128000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d502955e6531f3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.769+0000 7f011b7fe640 1 -- 172.21.15.92:0/1399556599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128035070 con 0x55a80edc81b0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.769+0000 7f011b7fe640 1 -- 172.21.15.92:0/1399556599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01280317f0 con 0x55a80edc81b0 2024-09-21T02:13:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.769+0000 7f011b7fe640 1 -- 172.21.15.92:0/1399556599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128003ce0 con 0x55a80edc81b0 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 -- 172.21.15.92:0/1399556599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 msgr2=0x55a80f16fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f0139a4f640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f011c0b6590 0x7f011c0b8aa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f012c0aa650 tx=0x7f012c071520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f16fe50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f01280096c0 tx=0x7f0128000bc0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 -- 172.21.15.92:0/1399556599 shutdown_connections 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f171010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f1704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 --2- 172.21.15.92:0/1399556599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f16fe50 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 -- 172.21.15.92:0/1399556599 >> 172.21.15.92:0/1399556599 conn(0x55a80edbc300 msgr2=0x55a80f16b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 -- 172.21.15.92:0/1399556599 shutdown_connections 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 -- 172.21.15.92:0/1399556599 wait complete. 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 -- start start 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.770+0000 7f014241d740 1 Processor -- start 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 0x55a80f179560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f179c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80edc81b0 0x55a80f17a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a80f009ab0 con 0x55a80ed9f2f0 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a80f077730 con 0x55a80eda74d0 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a80f171ad0 con 0x55a80edc81b0 2024-09-21T02:13:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 10 client.0 _ll_register_callbacks cb 0x55a80f0cb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 0x55a80f179560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 0x55a80f179560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43652/0 (socket says 172.21.15.92:43652) 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 -- 172.21.15.92:0/1253056901 learned_addr learned my addr 172.21.15.92:0/1253056901 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f013924e640 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f179c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 -- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80edc81b0 msgr2=0x55a80f17a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80edc81b0 0x55a80f17a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 -- 172.21.15.92:0/1253056901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 msgr2=0x55a80f179c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f179c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 -- 172.21.15.92:0/1253056901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a80ee33790 con 0x55a80ed9f2f0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0138a4d640 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 0x55a80f179560 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0128005ec0 tx=0x7f012802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6d1bd24aca279b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0118ff9640 1 -- 172.21.15.92:0/1253056901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128035050 con 0x55a80ed9f2f0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f0118ff9640 1 -- 172.21.15.92:0/1253056901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0128004030 con 0x55a80ed9f2f0 2024-09-21T02:13:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.771+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01280317f0 con 0x55a80ed9f2f0 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f0118ff9640 1 -- 172.21.15.92:0/1253056901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0128004230 con 0x55a80ed9f2f0 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 10 client.17862 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a80eeac620 con 0x55a80ed9f2f0 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f0118ff9640 1 -- 172.21.15.92:0/1253056901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01280317f0 con 0x55a80ed9f2f0 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f0118ff9640 1 -- 172.21.15.92:0/1253056901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f012803b460 con 0x55a80ed9f2f0 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 10 client.17862 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 20 client.17862 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed86d80 0x55a80f186870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 4 client.17862 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.772+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a80f0e4640 con 0x55a80ed86d80 2024-09-21T02:13:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.773+0000 7f013924e640 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed86d80 0x55a80f186870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.773+0000 7f0118ff9640 10 client.17862 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.773+0000 7f013924e640 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed86d80 0x55a80f186870 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1613fb431b5a42aa server_cookie=fb409bf6b0804e42 in_seq=0 out_seq=0 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f0118ff9640 1 -- 172.21.15.92:0/1253056901 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a80f0e4640 con 0x55a80ed86d80 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f0118ff9640 10 client.17862 handle_command_reply: tid=0 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 client.17862 shutdown 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 msgr2=0x55a80f179560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 0x55a80f179560 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0128005ec0 tx=0x7f012802ab10 comp rx=0 tx=0).stop 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 shutdown_connections 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed86d80 0x55a80f186870 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80edc81b0 0x55a80f17a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f179c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f0118ff9640 0 client.17862 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 --2- 172.21.15.92:0/1253056901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80ed9f2f0 0x55a80f179560 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 >> 172.21.15.92:0/1253056901 conn(0x55a80edbc300 msgr2=0x55a80f169ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 shutdown_connections 2024-09-21T02:13:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.774+0000 7f014241d740 1 -- 172.21.15.92:0/1253056901 wait complete. 2024-09-21T02:13:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.775+0000 7f014241d740 20 client.17862 trim_cache size 0 max 16384 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 Processor -- start 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 -- start start 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f178760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f177af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f178030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b1540 con 0x55a80edc81b0 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b3080 con 0x55a80eda74d0 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f014241d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b1c10 con 0x55a80ed9f2f0 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.789+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f178030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f178030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43656/0 (socket says 172.21.15.92:43656) 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 -- 172.21.15.92:0/3978775312 learned_addr learned my addr 172.21.15.92:0/3978775312 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f178760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 -- 172.21.15.92:0/3978775312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 msgr2=0x55a80f178760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f178760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 -- 172.21.15.92:0/3978775312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 msgr2=0x55a80f177af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f177af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 -- 172.21.15.92:0/3978775312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80f170500 con 0x55a80edc81b0 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f178030 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0128005f70 tx=0x7f0128046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6b3d17e144575c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f011affd640 1 -- 172.21.15.92:0/3978775312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128035030 con 0x55a80edc81b0 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f011affd640 1 -- 172.21.15.92:0/3978775312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012804f070 con 0x55a80edc81b0 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.790+0000 7f011affd640 1 -- 172.21.15.92:0/3978775312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128004090 con 0x55a80edc81b0 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 -- 172.21.15.92:0/3978775312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 msgr2=0x55a80f178030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f178030 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0128005f70 tx=0x7f0128046040 comp rx=0 tx=0).stop 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 -- 172.21.15.92:0/3978775312 shutdown_connections 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f178030 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f177af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 --2- 172.21.15.92:0/3978775312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f178760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 -- 172.21.15.92:0/3978775312 >> 172.21.15.92:0/3978775312 conn(0x55a80edbc300 msgr2=0x55a80f16fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 -- 172.21.15.92:0/3978775312 shutdown_connections 2024-09-21T02:13:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 -- 172.21.15.92:0/3978775312 wait complete. 2024-09-21T02:13:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 -- start start 2024-09-21T02:13:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.791+0000 7f014241d740 1 Processor -- start 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f180fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f17fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f180380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b29b0 con 0x55a80edc81b0 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01341b22e0 con 0x55a80eda74d0 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01341af520 con 0x55a80ed9f2f0 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f014241d740 10 client.0 _ll_register_callbacks cb 0x7f0128031dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f180380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f180380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43658/0 (socket says 172.21.15.92:43658) 2024-09-21T02:13:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 -- 172.21.15.92:0/3282667132 learned_addr learned my addr 172.21.15.92:0/3282667132 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f180fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f013924e640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f17fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 -- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 msgr2=0x55a80f180fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f180fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 -- 172.21.15.92:0/3282667132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 msgr2=0x55a80f17fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f17fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 -- 172.21.15.92:0/3282667132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a80f17e850 con 0x55a80edc81b0 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f013924e640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f17fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0138a4d640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f180380 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0128005f40 tx=0x7f0128003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9939bffb57f199ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.792+0000 7f0118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f0118ff9640 1 -- 172.21.15.92:0/3282667132 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128004030 con 0x55a80edc81b0 2024-09-21T02:13:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f0118ff9640 1 -- 172.21.15.92:0/3282667132 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f012803b460 con 0x55a80edc81b0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012804f070 con 0x55a80edc81b0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f0118ff9640 1 -- 172.21.15.92:0/3282667132 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f012803b660 con 0x55a80edc81b0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 10 client.17868 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a80f1f11e0 con 0x55a80edc81b0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f0118ff9640 1 -- 172.21.15.92:0/3282667132 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012804f070 con 0x55a80edc81b0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f0118ff9640 1 -- 172.21.15.92:0/3282667132 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f012804a4f0 con 0x55a80edc81b0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 10 client.17868 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 20 client.17868 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed5c1c0 0x55a80f1f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 4 client.17868 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a80f0e00f0 con 0x55a80ed5c1c0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.793+0000 7f013924e640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed5c1c0 0x55a80f1f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.794+0000 7f0118ff9640 10 client.17868 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.794+0000 7f013924e640 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed5c1c0 0x55a80f1f3b60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b081088f0e5268 server_cookie=f9811f81b27673a5 in_seq=0 out_seq=0 2024-09-21T02:13:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f0118ff9640 1 -- 172.21.15.92:0/3282667132 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a80f0e00f0 con 0x55a80ed5c1c0 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f0118ff9640 10 client.17868 handle_command_reply: tid=0 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 client.17868 shutdown 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 msgr2=0x55a80f180380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f180380 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0128005f40 tx=0x7f0128003ab0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 shutdown_connections 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a80ed5c1c0 0x55a80f1f3b60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a80edc81b0 0x55a80f180380 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f0118ff9640 0 client.17868 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a80eda74d0 0x55a80f17fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 --2- 172.21.15.92:0/3282667132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a80ed9f2f0 0x55a80f180fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 >> 172.21.15.92:0/3282667132 conn(0x55a80edbc300 msgr2=0x55a80f16fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 shutdown_connections 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 1 -- 172.21.15.92:0/3282667132 wait complete. 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.809+0000 7f014241d740 20 client.17868 trim_cache size 0 max 16384 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:13:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f011c0b6590 msgr2=0x7f011c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f011c0b6590 0x7f011c0b8aa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f012c0aa650 tx=0x7f012c071520 comp rx=0 tx=0).stop 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 msgr2=0x7f01341a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 0x7f01341a1830 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f012c00cad0 tx=0x7f012c00cf90 comp rx=0 tx=0).stop 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 shutdown_connections 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f011c0b6590 0x7f011c0b8aa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01341071f0 0x7f01341a1830 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0134104630 0x7f01341a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 --2- 172.21.15.92:0/2804758838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f013406c4d0 0x7f01341a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 >> 172.21.15.92:0/2804758838 conn(0x7f01340fdf00 msgr2=0x7f0134105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 shutdown_connections 2024-09-21T02:13:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:14.810+0000 7f013b4b2640 1 -- 172.21.15.92:0/2804758838 wait complete. 2024-09-21T02:13:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:14 smithi136 ceph-mon[34754]: pgmap v608: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:13:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:14 smithi092 ceph-mon[31190]: pgmap v608: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:13:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:14 smithi099 ceph-mon[34404]: pgmap v608: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:13:17.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:16 smithi136 ceph-mon[34754]: pgmap v609: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-21T02:13:17.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:16 smithi092 ceph-mon[31190]: pgmap v609: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-21T02:13:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:16 smithi099 ceph-mon[34404]: pgmap v609: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-21T02:13:17.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:13:17.865 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 -- 172.21.15.92:0/993340433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 msgr2=0x7ff2dc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 --2- 172.21.15.92:0/993340433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1087b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009900 tx=0x7ff2c402a410 comp rx=0 tx=0).stop 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 -- 172.21.15.92:0/993340433 shutdown_connections 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 --2- 172.21.15.92:0/993340433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 --2- 172.21.15.92:0/993340433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1087b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 --2- 172.21.15.92:0/993340433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2dc102a60 0x7ff2dc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 -- 172.21.15.92:0/993340433 >> 172.21.15.92:0/993340433 conn(0x7ff2dc079730 msgr2=0x7ff2dc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 -- 172.21.15.92:0/993340433 shutdown_connections 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 -- 172.21.15.92:0/993340433 wait complete. 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 Processor -- start 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.342+0000 7ff2e295e640 1 -- start start 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e295e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc102a60 0x7ff2dc0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e295e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e295e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e295e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112960 con 0x7ff2dc1083a0 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e295e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1127e0 con 0x7ff2dc102a60 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e295e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112ae0 con 0x7ff2dc108cf0 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc0ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43680/0 (socket says 172.21.15.92:43680) 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 -- 172.21.15.92:0/3038921669 learned_addr learned my addr 172.21.15.92:0/3038921669 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc102a60 0x7ff2dc0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 -- 172.21.15.92:0/3038921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2dc108cf0 msgr2=0x7ff2dc1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 -- 172.21.15.92:0/3038921669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc102a60 msgr2=0x7ff2dc0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc102a60 0x7ff2dc0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 -- 172.21.15.92:0/3038921669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc1a8860 con 0x7ff2dc1083a0 2024-09-21T02:13:18.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc0ffbb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4032fd0 tx=0x7ff2c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.343+0000 7ff2d9ffb640 1 -- 172.21.15.92:0/3038921669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c403f030 con 0x7ff2dc1083a0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.344+0000 7ff2d9ffb640 1 -- 172.21.15.92:0/3038921669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c4004490 con 0x7ff2dc1083a0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.344+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1a8af0 con 0x7ff2dc1083a0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.344+0000 7ff2d9ffb640 1 -- 172.21.15.92:0/3038921669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4035070 con 0x7ff2dc1083a0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.344+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1a8fd0 con 0x7ff2dc1083a0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 Processor -- start 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2d9ffb640 1 -- 172.21.15.92:0/3038921669 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff2c4042030 con 0x7ff2dc1083a0 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 -- start start 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4433db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4433e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ee442b3e50 con 0x55ee43f75390 2024-09-21T02:13:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ee442a3280 con 0x55ee43f96070 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.346+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ee442a7090 con 0x55ee43f6d1b0 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4433e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4433e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43688/0 (socket says 172.21.15.92:43688) 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 -- 172.21.15.92:0/2300571944 learned_addr learned my addr 172.21.15.92:0/2300571944 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4433db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 -- 172.21.15.92:0/2300571944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 msgr2=0x55ee4433db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4433db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 -- 172.21.15.92:0/2300571944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 msgr2=0x55ee4433d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 -- 172.21.15.92:0/2300571944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee44001650 con 0x55ee43f75390 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d9ffb640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff2c80b6510 0x7ff2c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff2c80b6510 0x7ff2c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4433e0d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff2c410dc10 tx=0x7ff2c410dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31e44f5040cb6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d9ffb640 1 -- 172.21.15.92:0/3038921669 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff2c41078b0 con 0x7ff2dc1083a0 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.347+0000 7ff2d27fc640 1 -- 172.21.15.92:0/2300571944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40ca070 con 0x55ee43f75390 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2d27fc640 1 -- 172.21.15.92:0/2300571944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c40c9180 con 0x55ee43f75390 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2d27fc640 1 -- 172.21.15.92:0/2300571944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40cd190 con 0x55ee43f75390 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/2300571944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 msgr2=0x55ee4433e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff2c80b6510 0x7ff2c80b8a20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4008dc0 tx=0x7ff2d4008570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4433e0d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff2c410dc10 tx=0x7ff2c410dd00 comp rx=0 tx=0).stop 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/2300571944 shutdown_connections 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4433e0d0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4433db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/2300571944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.348+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/2300571944 >> 172.21.15.92:0/2300571944 conn(0x55ee43f8a460 msgr2=0x55ee4433b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/2300571944 shutdown_connections 2024-09-21T02:13:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/2300571944 wait complete. 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 -- start start 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 Processor -- start 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f6d1b0 0x55ee44348ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee443491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f96070 0x55ee44354e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ee44104700 con 0x55ee43f75390 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ee4423e110 con 0x55ee43f6d1b0 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.349+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ee4434bbf0 con 0x55ee43f96070 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2e8ed2740 10 client.0 _ll_register_callbacks cb 0x55ee44257990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee443491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee443491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43694/0 (socket says 172.21.15.92:43694) 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 -- 172.21.15.92:0/15653436 learned_addr learned my addr 172.21.15.92:0/15653436 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f96070 0x55ee44354e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f6d1b0 0x55ee44348ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 -- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f96070 msgr2=0x55ee44354e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f96070 0x55ee44354e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 -- 172.21.15.92:0/15653436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f6d1b0 msgr2=0x55ee44348ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f6d1b0 0x55ee44348ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 -- 172.21.15.92:0/15653436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4426bc40 con 0x55ee43f75390 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee443491e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4032af0 tx=0x7ff2c403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76955010af57253 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/15653436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4049050 con 0x55ee43f75390 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/15653436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff2c410d790 con 0x55ee43f75390 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c40c9180 con 0x55ee43f75390 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.350+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/15653436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2c40cd370 con 0x55ee43f75390 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 10 client.17880 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee44357810 con 0x55ee43f75390 2024-09-21T02:13:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/15653436 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c40c9180 con 0x55ee43f75390 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/15653436 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2c40e3bd0 con 0x55ee43f75390 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 10 client.17880 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 20 client.17880 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f549f0 0x55ee4435ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 4 client.17880 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee442b5da0 con 0x55ee43f549f0 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.351+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f549f0 0x55ee4435ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.352+0000 7ff2bb7fe640 10 client.17880 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.352+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f549f0 0x55ee4435ab70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de22f9781bc83aae server_cookie=5606230781b45581 in_seq=0 out_seq=0 2024-09-21T02:13:18.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.352+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/15653436 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee442b5da0 con 0x55ee43f549f0 2024-09-21T02:13:18.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.352+0000 7ff2bb7fe640 10 client.17880 handle_command_reply: tid=0 2024-09-21T02:13:18.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 client.17880 shutdown 2024-09-21T02:13:18.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 msgr2=0x55ee443491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee443491e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4032af0 tx=0x7ff2c403b720 comp rx=0 tx=0).stop 2024-09-21T02:13:18.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 shutdown_connections 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f549f0 0x55ee4435ab70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f96070 0x55ee44354e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee443491e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2bb7fe640 0 client.17880 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/15653436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f6d1b0 0x55ee44348ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 >> 172.21.15.92:0/15653436 conn(0x55ee43f8a460 msgr2=0x55ee443407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 shutdown_connections 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/15653436 wait complete. 2024-09-21T02:13:18.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.353+0000 7ff2e8ed2740 20 client.17880 trim_cache size 0 max 16384 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 Processor -- start 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 -- start start 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 0x55ee4434a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f75390 0x55ee4433ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b16f0 con 0x55ee43f6d1b0 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b3230 con 0x55ee43f96070 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b1dc0 con 0x55ee43f75390 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 0x55ee4434a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 0x55ee4434a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43706/0 (socket says 172.21.15.92:43706) 2024-09-21T02:13:18.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.370+0000 7ff2dbfff640 1 -- 172.21.15.92:0/88108442 learned_addr learned my addr 172.21.15.92:0/88108442 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f75390 0x55ee4433ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2dbfff640 1 -- 172.21.15.92:0/88108442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f75390 msgr2=0x55ee4433ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f75390 0x55ee4433ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2dbfff640 1 -- 172.21.15.92:0/88108442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 msgr2=0x55ee4433b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2dbfff640 1 -- 172.21.15.92:0/88108442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4433f940 con 0x55ee43f6d1b0 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 0x55ee4434a8d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00bcb0 tx=0x7ff2cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975c54ed80287bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2d1ffb640 1 -- 172.21.15.92:0/88108442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc005220 con 0x55ee43f6d1b0 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2d1ffb640 1 -- 172.21.15.92:0/88108442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc004c50 con 0x55ee43f6d1b0 2024-09-21T02:13:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.371+0000 7ff2d1ffb640 1 -- 172.21.15.92:0/88108442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01d820 con 0x55ee43f6d1b0 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/88108442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 msgr2=0x55ee4434a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 0x55ee4434a8d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00bcb0 tx=0x7ff2cc011810 comp rx=0 tx=0).stop 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/88108442 shutdown_connections 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee4433b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f75390 0x55ee4433ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/88108442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f6d1b0 0x55ee4434a8d0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/88108442 >> 172.21.15.92:0/88108442 conn(0x55ee43f8a460 msgr2=0x55ee4434a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/88108442 shutdown_connections 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/88108442 wait complete. 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.372+0000 7ff2e8ed2740 1 -- start start 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 Processor -- start 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4434d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4434d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee443c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b2b60 con 0x55ee43f75390 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b2490 con 0x55ee43f96070 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1af6d0 con 0x55ee43f6d1b0 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e8ed2740 10 client.0 _ll_register_callbacks cb 0x7ff2d4008410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4434d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4434d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43716/0 (socket says 172.21.15.92:43716) 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 -- 172.21.15.92:0/369110489 learned_addr learned my addr 172.21.15.92:0/369110489 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:18.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4434d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee443c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 -- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 msgr2=0x55ee4434d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4434d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 -- 172.21.15.92:0/369110489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 msgr2=0x55ee443c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee443c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2d3fff640 1 -- 172.21.15.92:0/369110489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4434dd60 con 0x55ee43f75390 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.373+0000 7ff2e0efb640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee443c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2d3fff640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4434d550 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4031440 tx=0x7ff2c4031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7f410dcf144f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/369110489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4049030 con 0x55ee43f75390 2024-09-21T02:13:18.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/369110489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff2c403b460 con 0x55ee43f75390 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2cc004c50 con 0x55ee43f75390 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/369110489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2c40e8030 con 0x55ee43f75390 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 10 client.17886 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee4434dd60 con 0x55ee43f75390 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/369110489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc004c50 con 0x55ee43f75390 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/369110489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2c410d3e0 con 0x55ee43f75390 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 10 client.17886 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 20 client.17886 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f29da0 0x55ee443c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 4 client.17886 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.374+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee442b1850 con 0x55ee43f29da0 2024-09-21T02:13:18.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.375+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f29da0 0x55ee443c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.375+0000 7ff2dbfff640 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f29da0 0x55ee443c7e10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eea03108f3a7e0 server_cookie=6788a0425781cb85 in_seq=0 out_seq=0 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.375+0000 7ff2bb7fe640 10 client.17886 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.375+0000 7ff2bb7fe640 1 -- 172.21.15.92:0/369110489 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55ee442b1850 con 0x55ee43f29da0 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.375+0000 7ff2bb7fe640 10 client.17886 handle_command_reply: tid=0 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.375+0000 7ff2e8ed2740 1 client.17886 shutdown 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 msgr2=0x55ee4434d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4434d550 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4031440 tx=0x7ff2c4031a10 comp rx=0 tx=0).stop 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 shutdown_connections 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55ee43f29da0 0x55ee443c7e10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee43f96070 0x55ee443c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee43f75390 0x55ee4434d550 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2bb7fe640 0 client.17886 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 --2- 172.21.15.92:0/369110489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee43f6d1b0 0x55ee4434d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 >> 172.21.15.92:0/369110489 conn(0x55ee43f8a460 msgr2=0x55ee4434aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 shutdown_connections 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 1 -- 172.21.15.92:0/369110489 wait complete. 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e8ed2740 20 client.17886 trim_cache size 0 max 16384 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:13:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:13:18.384 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.385 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.386 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:13:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:13:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:13:18.389 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff2c80b6510 msgr2=0x7ff2c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e295e640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff2c80b6510 0x7ff2c80b8a20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4008dc0 tx=0x7ff2d4008570 comp rx=0 tx=0).stop 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 msgr2=0x7ff2dc0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.376+0000 7ff2e295e640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc0ffbb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4032fd0 tx=0x7ff2c4004270 comp rx=0 tx=0).stop 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 shutdown_connections 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff2c80b6510 0x7ff2c80b8a20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc0ffbb0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 --2- 172.21.15.92:0/3038921669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc102a60 0x7ff2dc0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 >> 172.21.15.92:0/3038921669 conn(0x7ff2dc079730 msgr2=0x7ff2dc101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:18.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 shutdown_connections 2024-09-21T02:13:18.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:18.377+0000 7ff2e295e640 1 -- 172.21.15.92:0/3038921669 wait complete. 2024-09-21T02:13:19.188 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:18 smithi092 ceph-mon[31190]: pgmap v610: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:13:19.216 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:13:19.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:13:19.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:18 smithi136 ceph-mon[34754]: pgmap v610: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:13:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:18 smithi099 ceph-mon[34404]: pgmap v610: 97 pgs: 97 active+clean; 2.3 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:13:19.463 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:19.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 -- 172.21.15.92:0/2832550019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c34108030 msgr2=0x7f5c3410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 --2- 172.21.15.92:0/2832550019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c34108030 0x7f5c3410e860 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c840 tx=0x7f5c2c02ae30 comp rx=0 tx=0).stop 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 -- 172.21.15.92:0/2832550019 shutdown_connections 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 --2- 172.21.15.92:0/2832550019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c34108030 0x7f5c3410e860 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 --2- 172.21.15.92:0/2832550019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c341076e0 0x7f5c34107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 --2- 172.21.15.92:0/2832550019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c34101da0 0x7f5c341021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.938+0000 7f5c3c153640 1 -- 172.21.15.92:0/2832550019 >> 172.21.15.92:0/2832550019 conn(0x7f5c34077800 msgr2=0x7f5c34077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.939+0000 7f5c3c153640 1 -- 172.21.15.92:0/2832550019 shutdown_connections 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.939+0000 7f5c3c153640 1 -- 172.21.15.92:0/2832550019 wait complete. 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.939+0000 7f5c3c153640 1 Processor -- start 2024-09-21T02:13:19.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.939+0000 7f5c3c153640 1 -- start start 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3c153640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c34101da0 0x7f5c3419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3c153640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 0x7f5c3419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3c153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c34108030 0x7f5c3419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3c153640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34111890 con 0x7f5c341076e0 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3c153640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34111710 con 0x7f5c34108030 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3c153640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34111a10 con 0x7f5c34101da0 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 0x7f5c3419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 0x7f5c3419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43722/0 (socket says 172.21.15.92:43722) 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 -- 172.21.15.92:0/968096878 learned_addr learned my addr 172.21.15.92:0/968096878 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c39eef640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c34101da0 0x7f5c3419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3a6f0640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c34108030 0x7f5c3419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 -- 172.21.15.92:0/968096878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c34101da0 msgr2=0x7f5c3419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c34101da0 0x7f5c3419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 -- 172.21.15.92:0/968096878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c34108030 msgr2=0x7f5c3419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c34108030 0x7f5c3419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 -- 172.21.15.92:0/968096878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c341a6550 con 0x7f5c341076e0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c3a6f0640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c34108030 0x7f5c3419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c396ee640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 0x7f5c3419b810 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5c24006fd0 tx=0x7f5c2400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.940+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24015020 con 0x7f5c341076e0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.941+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c24011ce0 con 0x7f5c341076e0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.941+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24010d90 con 0x7f5c341076e0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.941+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c341a6840 con 0x7f5c341076e0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.941+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341a6d80 con 0x7f5c341076e0 2024-09-21T02:13:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.942+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c341021b0 con 0x7f5c341076e0 2024-09-21T02:13:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.943+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5c24018020 con 0x7f5c341076e0 2024-09-21T02:13:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.944+0000 7f5c2affd640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c200b6590 0x7f5c200b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.944+0000 7f5c39eef640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c200b6590 0x7f5c200b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.944+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5c240e6f80 con 0x7f5c341076e0 2024-09-21T02:13:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.945+0000 7f5c39eef640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c200b6590 0x7f5c200b8aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5c341060b0 tx=0x7f5c3000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:19.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:19.947+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c2401f020 con 0x7f5c341076e0 2024-09-21T02:13:20.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.163+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c3402e640 con 0x7f5c341076e0 2024-09-21T02:13:20.164 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.164+0000 7f5c2affd640 1 -- 172.21.15.92:0/968096878 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f5c240ad7f0 con 0x7f5c341076e0 2024-09-21T02:13:20.165 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:13:20.165 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:13:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:13:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c200b6590 msgr2=0x7f5c200b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c200b6590 0x7f5c200b8aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5c341060b0 tx=0x7f5c3000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 msgr2=0x7f5c3419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 0x7f5c3419b810 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5c24006fd0 tx=0x7f5c2400b820 comp rx=0 tx=0).stop 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 shutdown_connections 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5c200b6590 0x7f5c200b8aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c34108030 0x7f5c3419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c341076e0 0x7f5c3419b810 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 --2- 172.21.15.92:0/968096878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c34101da0 0x7f5c3419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 >> 172.21.15.92:0/968096878 conn(0x7f5c34077800 msgr2=0x7f5c34100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 shutdown_connections 2024-09-21T02:13:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:20.166+0000 7f5c3c153640 1 -- 172.21.15.92:0/968096878 wait complete. 2024-09-21T02:13:20.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:13:21.037 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:21.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:20 smithi092 ceph-mon[31190]: pgmap v611: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:21.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:20 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/968096878' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:13:21.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:20 smithi136 ceph-mon[34754]: pgmap v611: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:21.359 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:20 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/968096878' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:13:21.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:20 smithi099 ceph-mon[34404]: pgmap v611: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:21.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:20 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/968096878' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:13:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 -- 172.21.15.92:0/715083840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b8108400 msgr2=0x7f01b8108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 --2- 172.21.15.92:0/715083840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b8108400 0x7f01b8108810 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009900 tx=0x7f01a002a410 comp rx=0 tx=0).stop 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 -- 172.21.15.92:0/715083840 shutdown_connections 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 --2- 172.21.15.92:0/715083840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b8108d50 0x7f01b810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 --2- 172.21.15.92:0/715083840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b8108400 0x7f01b8108810 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 --2- 172.21.15.92:0/715083840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01b81031d0 0x7f01b81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 -- 172.21.15.92:0/715083840 >> 172.21.15.92:0/715083840 conn(0x7f01b8079f90 msgr2=0x7f01b807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 -- 172.21.15.92:0/715083840 shutdown_connections 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.523+0000 7f01bce36640 1 -- 172.21.15.92:0/715083840 wait complete. 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.524+0000 7f01bce36640 1 Processor -- start 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.524+0000 7f01bce36640 1 -- start start 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 0x7f01b819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b8108400 0x7f01b819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01b8108d50 0x7f01b819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8112a10 con 0x7f01b81031d0 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8112890 con 0x7f01b8108400 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8112b90 con 0x7f01b8108d50 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 0x7f01b819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 0x7f01b819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43750/0 (socket says 172.21.15.92:43750) 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 -- 172.21.15.92:0/3739170784 learned_addr learned my addr 172.21.15.92:0/3739170784 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b8108400 0x7f01b819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01b8108d50 0x7f01b819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 -- 172.21.15.92:0/3739170784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01b8108d50 msgr2=0x7f01b819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01b8108d50 0x7f01b819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 -- 172.21.15.92:0/3739170784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b8108400 msgr2=0x7f01b819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b8108400 0x7f01b819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 -- 172.21.15.92:0/3739170784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b81a04e0 con 0x7f01b81031d0 2024-09-21T02:13:21.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01b659c640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 0x7f01b819a630 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f01a800d770 tx=0x7f01a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01af7fe640 1 -- 172.21.15.92:0/3739170784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8017070 con 0x7f01b81031d0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01af7fe640 1 -- 172.21.15.92:0/3739170784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a80054d0 con 0x7f01b81031d0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.525+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b81a07d0 con 0x7f01b81031d0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.526+0000 7f01af7fe640 1 -- 172.21.15.92:0/3739170784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8004c30 con 0x7f01b81031d0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.526+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b81a0c90 con 0x7f01b81031d0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.528+0000 7f01af7fe640 1 -- 172.21.15.92:0/3739170784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f01a8004dd0 con 0x7f01b81031d0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.528+0000 7f01c33aa740 1 Processor -- start 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.528+0000 7f01c33aa740 1 -- start start 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01c33aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c0fb05eea0 con 0x55c0fad20390 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01c33aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c0fb04e2d0 con 0x55c0fad181b0 2024-09-21T02:13:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01c33aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0fb0520e0 con 0x55c0fad41070 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43762/0 (socket says 172.21.15.92:43762) 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 -- 172.21.15.92:0/2821264347 learned_addr learned my addr 172.21.15.92:0/2821264347 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01af7fe640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f01a40b6540 0x7f01a40b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b659c640 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f01a40b6540 0x7f01a40b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 -- 172.21.15.92:0/2821264347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 msgr2=0x55c0fb0e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 -- 172.21.15.92:0/2821264347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 msgr2=0x55c0fb0e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0e8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b5d9b640 1 -- 172.21.15.92:0/2821264347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fadac650 con 0x55c0fad20390 2024-09-21T02:13:21.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.529+0000 7f01af7fe640 1 -- 172.21.15.92:0/3739170784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f01a80e6d40 con 0x7f01b81031d0 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.530+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0e9120 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009900 tx=0x7f01a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ac7e9d7ea8f1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.530+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f01a40b6540 0x7f01a40b8a50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f01b0014610 tx=0x7f01b0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.530+0000 7f01acff9640 1 -- 172.21.15.92:0/2821264347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0040070 con 0x55c0fad20390 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.530+0000 7f01acff9640 1 -- 172.21.15.92:0/2821264347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a0005db0 con 0x55c0fad20390 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.530+0000 7f01acff9640 1 -- 172.21.15.92:0/2821264347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0035040 con 0x55c0fad20390 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 -- 172.21.15.92:0/2821264347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 msgr2=0x55c0fb0e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0e9120 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009900 tx=0x7f01a0000bc0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 -- 172.21.15.92:0/2821264347 shutdown_connections 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0e9120 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2821264347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 -- 172.21.15.92:0/2821264347 >> 172.21.15.92:0/2821264347 conn(0x55c0fad35460 msgr2=0x55c0fb0e6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 -- 172.21.15.92:0/2821264347 shutdown_connections 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 -- 172.21.15.92:0/2821264347 wait complete. 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 -- start start 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.531+0000 7f01c33aa740 1 Processor -- start 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c0faeaf700 con 0x55c0fad20390 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c0fafe9160 con 0x55c0fad181b0 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0fb0f6e20 con 0x55c0fad41070 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01c33aa740 10 client.0 _ll_register_callbacks cb 0x55c0fb0029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43778/0 (socket says 172.21.15.92:43778) 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 -- 172.21.15.92:0/3437340673 learned_addr learned my addr 172.21.15.92:0/3437340673 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b659c640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 -- 172.21.15.92:0/3437340673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 msgr2=0x55c0fb0f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 -- 172.21.15.92:0/3437340673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 msgr2=0x55c0fb0f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 -- 172.21.15.92:0/3437340673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fb016c90 con 0x55c0fad20390 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b659c640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f4200 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005640 tx=0x7f01a0005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be328344d3cdaef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.532+0000 7f019dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f019dffb640 1 -- 172.21.15.92:0/3437340673 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a004b020 con 0x55c0fad20390 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f019dffb640 1 -- 172.21.15.92:0/3437340673 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f01a0004750 con 0x55c0fad20390 2024-09-21T02:13:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a0005db0 con 0x55c0fad20390 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f019dffb640 1 -- 172.21.15.92:0/3437340673 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01a0056050 con 0x55c0fad20390 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 10 client.17904 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0fb102aa0 con 0x55c0fad20390 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f019dffb640 1 -- 172.21.15.92:0/3437340673 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a0005db0 con 0x55c0fad20390 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f019dffb640 1 -- 172.21.15.92:0/3437340673 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f01a003c9a0 con 0x55c0fad20390 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 10 client.17904 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 10 client.17904 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 20 client.17904 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facff9f0 0x55c0fb105e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 4 client.17904 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.533+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0fb060df0 con 0x55c0facff9f0 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.534+0000 7f01b659c640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facff9f0 0x55c0fb105e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.534+0000 7f019dffb640 10 client.17904 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:13:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.534+0000 7f01b659c640 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facff9f0 0x55c0fb105e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0b86df15594ccb6 server_cookie=819f4a957b386038 in_seq=0 out_seq=0 2024-09-21T02:13:21.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.534+0000 7f019dffb640 1 -- 172.21.15.92:0/3437340673 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0fb060df0 con 0x55c0facff9f0 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.534+0000 7f019dffb640 10 client.17904 handle_command_reply: tid=0 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 client.17904 shutdown 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 msgr2=0x55c0fb0f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f4200 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005640 tx=0x7f01a0005670 comp rx=0 tx=0).stop 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 shutdown_connections 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facff9f0 0x55c0fb105e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f4200 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f019dffb640 0 client.17904 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:13:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 --2- 172.21.15.92:0/3437340673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 >> 172.21.15.92:0/3437340673 conn(0x55c0fad35460 msgr2=0x55c0fb0eb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 shutdown_connections 2024-09-21T02:13:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 1 -- 172.21.15.92:0/3437340673 wait complete. 2024-09-21T02:13:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.535+0000 7f01c33aa740 20 client.17904 trim_cache size 0 max 16384 2024-09-21T02:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 Processor -- start 2024-09-21T02:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 -- start start 2024-09-21T02:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81af420 con 0x55c0fad20390 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81b0f60 con 0x55c0fad181b0 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01c33aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81afaf0 con 0x55c0fad41070 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01b5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01b5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43784/0 (socket says 172.21.15.92:43784) 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01b5d9b640 1 -- 172.21.15.92:0/917018673 learned_addr learned my addr 172.21.15.92:0/917018673 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.548+0000 7f01b659c640 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f01b5d9b640 1 -- 172.21.15.92:0/917018673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 msgr2=0x55c0fb0f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f01b5d9b640 1 -- 172.21.15.92:0/917018673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 msgr2=0x55c0fb0f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f01b5d9b640 1 -- 172.21.15.92:0/917018673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fb0f4f80 con 0x55c0fad20390 2024-09-21T02:13:21.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f0eb0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f01a002a450 tx=0x7f01a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0dba7e2c69732e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:21.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f019ffff640 1 -- 172.21.15.92:0/917018673 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a004b070 con 0x55c0fad20390 2024-09-21T02:13:21.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f019ffff640 1 -- 172.21.15.92:0/917018673 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a0005db0 con 0x55c0fad20390 2024-09-21T02:13:21.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.549+0000 7f019ffff640 1 -- 172.21.15.92:0/917018673 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0035070 con 0x55c0fad20390 2024-09-21T02:13:21.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 -- 172.21.15.92:0/917018673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 msgr2=0x55c0fb0f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f0eb0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f01a002a450 tx=0x7f01a0000bc0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 -- 172.21.15.92:0/917018673 shutdown_connections 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad41070 0x55c0fb0f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad20390 0x55c0fb0f0eb0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 --2- 172.21.15.92:0/917018673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb0f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 -- 172.21.15.92:0/917018673 >> 172.21.15.92:0/917018673 conn(0x55c0fad35460 msgr2=0x55c0fb0f3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 -- 172.21.15.92:0/917018673 shutdown_connections 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 -- 172.21.15.92:0/917018673 wait complete. 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 -- start start 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.550+0000 7f01c33aa740 1 Processor -- start 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb1075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad20390 0x55c0fb107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 0x55c0fb16f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81b0890 con 0x55c0fad41070 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81b01c0 con 0x55c0fad181b0 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81ad400 con 0x55c0fad20390 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 10 client.0 _ll_register_callbacks cb 0x7f01a80a96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 0x55c0fb16f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 0x55c0fb16f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43794/0 (socket says 172.21.15.92:43794) 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 -- 172.21.15.92:0/2920183005 learned_addr learned my addr 172.21.15.92:0/2920183005 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b659c640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb1075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad20390 0x55c0fb107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 -- 172.21.15.92:0/2920183005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad20390 msgr2=0x55c0fb107b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad20390 0x55c0fb107b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 -- 172.21.15.92:0/2920183005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 msgr2=0x55c0fb1075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb1075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 -- 172.21.15.92:0/2920183005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fb16faf0 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b5d9b640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad20390 0x55c0fb107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01b6d9d640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 0x55c0fb16f580 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f01b0011f90 tx=0x7f01b00079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e017100a9482b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f019dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f019dffb640 1 -- 172.21.15.92:0/2920183005 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0029480 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f019dffb640 1 -- 172.21.15.92:0/2920183005 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f01b001e410 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.551+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a0005db0 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f019dffb640 1 -- 172.21.15.92:0/2920183005 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01b002f070 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 10 client.17910 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0fb170dd0 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f019dffb640 1 -- 172.21.15.92:0/2920183005 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a0005db0 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f019dffb640 1 -- 172.21.15.92:0/2920183005 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f01b001e8b0 con 0x55c0fad41070 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 10 client.17910 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 10 client.17910 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:13:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 20 client.17910 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facd4da0 0x55c0fb173640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 4 client.17910 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0fb05c8a0 con 0x55c0facd4da0 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01b659c640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facd4da0 0x55c0fb173640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f019dffb640 10 client.17910 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.552+0000 7f01b659c640 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facd4da0 0x55c0fb173640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0a77cc92548a5ad server_cookie=bb75806550addb51 in_seq=0 out_seq=0 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f019dffb640 1 -- 172.21.15.92:0/2920183005 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0fb05c8a0 con 0x55c0facd4da0 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f019dffb640 10 client.17910 handle_command_reply: tid=0 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 client.17910 shutdown 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 msgr2=0x55c0fb16f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 0x55c0fb16f580 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f01b0011f90 tx=0x7f01b00079b0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 shutdown_connections 2024-09-21T02:13:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55c0facd4da0 0x55c0fb173640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c0fad41070 0x55c0fb16f580 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0fad20390 0x55c0fb107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 --2- 172.21.15.92:0/2920183005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0fad181b0 0x55c0fb1075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 >> 172.21.15.92:0/2920183005 conn(0x55c0fad35460 msgr2=0x55c0fb0e96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f019dffb640 0 client.17910 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 shutdown_connections 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 1 -- 172.21.15.92:0/2920183005 wait complete. 2024-09-21T02:13:21.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.553+0000 7f01c33aa740 20 client.17910 trim_cache size 0 max 16384 2024-09-21T02:13:21.564 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:13:21.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f01a40b6540 msgr2=0x7f01a40b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f01a40b6540 0x7f01a40b8a50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f01b0014610 tx=0x7f01b0004ab0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 msgr2=0x7f01b819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:21.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 0x7f01b819a630 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f01a800d770 tx=0x7f01a800dc30 comp rx=0 tx=0).stop 2024-09-21T02:13:21.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 shutdown_connections 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f01a40b6540 0x7f01a40b8a50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01b8108d50 0x7f01b819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b8108400 0x7f01b819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 --2- 172.21.15.92:0/3739170784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01b81031d0 0x7f01b819a630 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 >> 172.21.15.92:0/3739170784 conn(0x7f01b8079f90 msgr2=0x7f01b81013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 shutdown_connections 2024-09-21T02:13:21.566 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:21.554+0000 7f01bce36640 1 -- 172.21.15.92:0/3739170784 wait complete. 2024-09-21T02:13:22.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:13:22.494 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:22.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 -- 172.21.15.92:0/1359184296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80109df0 msgr2=0x7f5a80110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1359184296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80109df0 0x7f5a80110620 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009ca0 tx=0x7f5a7802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 -- 172.21.15.92:0/1359184296 shutdown_connections 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1359184296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80109df0 0x7f5a80110620 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1359184296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a801094a0 0x7f5a801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1359184296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a80103b60 0x7f5a80103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 -- 172.21.15.92:0/1359184296 >> 172.21.15.92:0/1359184296 conn(0x7f5a8007b850 msgr2=0x7f5a8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 -- 172.21.15.92:0/1359184296 shutdown_connections 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.984+0000 7f5a84f01640 1 -- 172.21.15.92:0/1359184296 wait complete. 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 Processor -- start 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 -- start start 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 0x7f5a8019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a801094a0 0x7f5a8019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a80109df0 0x7f5a8019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113ab0 con 0x7f5a80103b60 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113930 con 0x7f5a801094a0 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a84f01640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113c30 con 0x7f5a80109df0 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 0x7f5a8019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 0x7f5a8019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43800/0 (socket says 172.21.15.92:43800) 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 -- 172.21.15.92:0/1314844449 learned_addr learned my addr 172.21.15.92:0/1314844449 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a801094a0 0x7f5a8019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a80109df0 0x7f5a8019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 -- 172.21.15.92:0/1314844449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a80109df0 msgr2=0x7f5a8019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a80109df0 0x7f5a8019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 -- 172.21.15.92:0/1314844449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a801094a0 msgr2=0x7f5a8019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a801094a0 0x7f5a8019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.985+0000 7f5a7e59c640 1 -- 172.21.15.92:0/1314844449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a801a0370 con 0x7f5a80103b60 2024-09-21T02:13:22.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.986+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 0x7f5a8019a400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000d950 tx=0x7f5a7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.986+0000 7f5a6f7fe640 1 -- 172.21.15.92:0/1314844449 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70017070 con 0x7f5a80103b60 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.986+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a801a0660 con 0x7f5a80103b60 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.986+0000 7f5a6f7fe640 1 -- 172.21.15.92:0/1314844449 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70009c30 con 0x7f5a80103b60 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.986+0000 7f5a6f7fe640 1 -- 172.21.15.92:0/1314844449 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000f620 con 0x7f5a80103b60 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.986+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a801a6dc0 con 0x7f5a80103b60 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 Processor -- start 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 -- start start 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a6f7fe640 1 -- 172.21.15.92:0/1314844449 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5a70010460 con 0x7f5a80103b60 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x5633152866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x563315286be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315287120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5633151fcea0 con 0x563314ebe390 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5633151ec2d0 con 0x563314edf070 2024-09-21T02:13:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a8b475740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5633151f00e0 con 0x563314eb61b0 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a7dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315287120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a7dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315287120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43806/0 (socket says 172.21.15.92:43806) 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/2901852915 learned_addr learned my addr 172.21.15.92:0/2901852915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x5633152866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.989+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x563315286be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/2901852915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 msgr2=0x563315286be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x563315286be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/2901852915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 msgr2=0x5633152866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x5633152866a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/2901852915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563314f4a650 con 0x563314ebe390 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a6f7fe640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a680b6510 0x7f5a680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315287120 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400aba0 tx=0x7f5a7400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4c6d675617173d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a680b6510 0x7f5a680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a6f7fe640 1 -- 172.21.15.92:0/1314844449 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5a700e6f60 con 0x7f5a80103b60 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a6cff9640 1 -- 172.21.15.92:0/2901852915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74004220 con 0x563314ebe390 2024-09-21T02:13:22.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a6cff9640 1 -- 172.21.15.92:0/2901852915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a740043c0 con 0x563314ebe390 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.990+0000 7f5a6cff9640 1 -- 172.21.15.92:0/2901852915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740187a0 con 0x563314ebe390 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 -- 172.21.15.92:0/2901852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 msgr2=0x563315287120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a680b6510 0x7f5a680b8a20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005cc0 tx=0x7f5a78005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315287120 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400aba0 tx=0x7f5a7400f790 comp rx=0 tx=0).stop 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 -- 172.21.15.92:0/2901852915 shutdown_connections 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315287120 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x563315286be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2901852915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x5633152866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 -- 172.21.15.92:0/2901852915 >> 172.21.15.92:0/2901852915 conn(0x563314ed3460 msgr2=0x563315284590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 -- 172.21.15.92:0/2901852915 shutdown_connections 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.991+0000 7f5a8b475740 1 -- 172.21.15.92:0/2901852915 wait complete. 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 -- start start 2024-09-21T02:13:22.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 Processor -- start 2024-09-21T02:13:23.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 0x563315296050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314ebe390 0x563315296590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x563315296ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56331504d700 con 0x563314eb61b0 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563315187160 con 0x563314edf070 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5633152991b0 con 0x563314ebe390 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a8b475740 10 client.0 _ll_register_callbacks cb 0x5633151a09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a7e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 0x563315296050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a7e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 0x563315296050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43820/0 (socket says 172.21.15.92:43820) 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a7e59c640 1 -- 172.21.15.92:0/3442359594 learned_addr learned my addr 172.21.15.92:0/3442359594 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x563315296ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.992+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314ebe390 0x563315296590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7e59c640 1 -- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314ebe390 msgr2=0x563315296590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314ebe390 0x563315296590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7e59c640 1 -- 172.21.15.92:0/3442359594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 msgr2=0x563315296ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x563315296ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7e59c640 1 -- 172.21.15.92:0/3442359594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633151b4c90 con 0x563314eb61b0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x563315296ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314ebe390 0x563315296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 0x563315296050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5a700ad060 tx=0x7f5a700ea630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7557c1e5d3983d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a65ffb640 1 -- 172.21.15.92:0/3442359594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70020070 con 0x563314eb61b0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a65ffb640 1 -- 172.21.15.92:0/3442359594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5a700ac530 con 0x563314eb61b0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a740043c0 con 0x563314eb61b0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a65ffb640 1 -- 172.21.15.92:0/3442359594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a700aca20 con 0x563314eb61b0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a8b475740 10 client.17922 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5633152a4e50 con 0x563314eb61b0 2024-09-21T02:13:23.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a65ffb640 1 -- 172.21.15.92:0/3442359594 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a740043c0 con 0x563314eb61b0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a65ffb640 1 -- 172.21.15.92:0/3442359594 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a70098200 con 0x563314eb61b0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a8b475740 10 client.17922 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a8b475740 10 client.17922 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.993+0000 7f5a8b475740 20 client.17922 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.994+0000 7f5a8b475740 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e9d9f0 0x5633152a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.994+0000 7f5a8b475740 4 client.17922 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.994+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633151fedf0 con 0x563314e9d9f0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.994+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e9d9f0 0x5633152a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.994+0000 7f5a65ffb640 10 client.17922 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.994+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e9d9f0 0x5633152a8320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de6aeb3101468a7 server_cookie=18c0c52ccc9f610d in_seq=0 out_seq=0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a65ffb640 1 -- 172.21.15.92:0/3442359594 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633151fedf0 con 0x563314e9d9f0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a65ffb640 10 client.17922 handle_command_reply: tid=0 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 client.17922 shutdown 2024-09-21T02:13:23.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 msgr2=0x563315296050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 0x563315296050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5a700ad060 tx=0x7f5a700ea630 comp rx=0 tx=0).stop 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 shutdown_connections 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e9d9f0 0x5633152a8320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314edf070 0x563315296ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314ebe390 0x563315296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 --2- 172.21.15.92:0/3442359594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314eb61b0 0x563315296050 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 >> 172.21.15.92:0/3442359594 conn(0x563314ed3460 msgr2=0x563315289830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.995+0000 7f5a65ffb640 0 client.17922 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.996+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 shutdown_connections 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.996+0000 7f5a8b475740 1 -- 172.21.15.92:0/3442359594 wait complete. 2024-09-21T02:13:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:22.996+0000 7f5a8b475740 20 client.17922 trim_cache size 0 max 16384 2024-09-21T02:13:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 Processor -- start 2024-09-21T02:13:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 -- start start 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314eb61b0 0x563315297570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315283bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314edf070 0x563315284100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801ad2a0 con 0x563314ebe390 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801aede0 con 0x563314eb61b0 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a8b475740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801ad970 con 0x563314edf070 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315283bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315283bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43828/0 (socket says 172.21.15.92:43828) 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/1817407934 learned_addr learned my addr 172.21.15.92:0/1817407934 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314eb61b0 0x563315297570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314edf070 0x563315284100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/1817407934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314edf070 msgr2=0x563315284100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314edf070 0x563315284100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/1817407934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314eb61b0 msgr2=0x563315297570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314eb61b0 0x563315297570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.009+0000 7f5a7dd9b640 1 -- 172.21.15.92:0/1817407934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563315298420 con 0x563314ebe390 2024-09-21T02:13:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.010+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315283bc0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5a74004140 tx=0x7f5a7400a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d8c0174719647 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.010+0000 7f5a67fff640 1 -- 172.21.15.92:0/1817407934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74031070 con 0x563314ebe390 2024-09-21T02:13:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.010+0000 7f5a67fff640 1 -- 172.21.15.92:0/1817407934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a74004270 con 0x563314ebe390 2024-09-21T02:13:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.010+0000 7f5a67fff640 1 -- 172.21.15.92:0/1817407934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740035f0 con 0x563314ebe390 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- 172.21.15.92:0/1817407934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 msgr2=0x563315283bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315283bc0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5a74004140 tx=0x7f5a7400a8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- 172.21.15.92:0/1817407934 shutdown_connections 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314edf070 0x563315284100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314ebe390 0x563315283bc0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- 172.21.15.92:0/1817407934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314eb61b0 0x563315297570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- 172.21.15.92:0/1817407934 >> 172.21.15.92:0/1817407934 conn(0x563314ed3460 msgr2=0x563315288090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- 172.21.15.92:0/1817407934 shutdown_connections 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- 172.21.15.92:0/1817407934 wait complete. 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- start start 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 Processor -- start 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x56331529ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314ebe390 0x56331529a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 0x56331529a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801ae710 con 0x563314edf070 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801ae040 con 0x563314ebe390 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.011+0000 7f5a8b475740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801ab280 con 0x563314eb61b0 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a8b475740 10 client.0 _ll_register_callbacks cb 0x7f5a7000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 0x56331529a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 0x56331529a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43842/0 (socket says 172.21.15.92:43842) 2024-09-21T02:13:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 -- 172.21.15.92:0/2616109980 learned_addr learned my addr 172.21.15.92:0/2616109980 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7dd9b640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314ebe390 0x56331529a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x56331529ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 -- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 msgr2=0x56331529ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x56331529ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 -- 172.21.15.92:0/2616109980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314ebe390 msgr2=0x56331529a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314ebe390 0x56331529a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 -- 172.21.15.92:0/2616109980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331529af30 con 0x563314edf070 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a7ed9d640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 0x56331529a7d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5a78035000 tx=0x7f5a7802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb2d8f43aba51e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a65ffb640 1 -- 172.21.15.92:0/2616109980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78004040 con 0x563314edf070 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a65ffb640 1 -- 172.21.15.92:0/2616109980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5a78042030 con 0x563314edf070 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.012+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a74004270 con 0x563314edf070 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a65ffb640 1 -- 172.21.15.92:0/2616109980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a780041e0 con 0x563314edf070 2024-09-21T02:13:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 10 client.17928 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563315312cb0 con 0x563314edf070 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a65ffb640 1 -- 172.21.15.92:0/2616109980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a74004270 con 0x563314edf070 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a65ffb640 1 -- 172.21.15.92:0/2616109980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a7802fd50 con 0x563314edf070 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 10 client.17928 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 10 client.17928 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 20 client.17928 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e72da0 0x5633153154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 4 client.17928 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633151fa8a0 con 0x563314e72da0 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.013+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e72da0 0x5633153154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a65ffb640 10 client.17928 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a7e59c640 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e72da0 0x5633153154d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e090eae4121a4a64 server_cookie=d61c4e73e05775a5 in_seq=0 out_seq=0 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a65ffb640 1 -- 172.21.15.92:0/2616109980 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633151fa8a0 con 0x563314e72da0 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a65ffb640 10 client.17928 handle_command_reply: tid=0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 client.17928 shutdown 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 msgr2=0x56331529a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 0x56331529a7d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5a78035000 tx=0x7f5a7802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 shutdown_connections 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563314edf070 0x56331529a7d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x563314e72da0 0x5633153154d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563314ebe390 0x56331529a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a65ffb640 0 client.17928 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 --2- 172.21.15.92:0/2616109980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563314eb61b0 0x56331529ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 >> 172.21.15.92:0/2616109980 conn(0x563314ed3460 msgr2=0x5633152876b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 shutdown_connections 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 1 -- 172.21.15.92:0/2616109980 wait complete. 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.014+0000 7f5a8b475740 20 client.17928 trim_cache size 0 max 16384 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a680b6510 msgr2=0x7f5a680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a680b6510 0x7f5a680b8a20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005cc0 tx=0x7f5a78005c30 comp rx=0 tx=0).stop 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 msgr2=0x7f5a8019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 0x7f5a8019a400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000d950 tx=0x7f5a7000de10 comp rx=0 tx=0).stop 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 shutdown_connections 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5a680b6510 0x7f5a680b8a20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a80109df0 0x7f5a8019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a801094a0 0x7f5a8019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 --2- 172.21.15.92:0/1314844449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a80103b60 0x7f5a8019a400 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 >> 172.21.15.92:0/1314844449 conn(0x7f5a8007b850 msgr2=0x7f5a80107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 shutdown_connections 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:23.015+0000 7f5a84f01640 1 -- 172.21.15.92:0/1314844449 wait complete. 2024-09-21T02:13:23.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:22 smithi136 ceph-mon[34754]: pgmap v612: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:23.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:22 smithi092 ceph-mon[31190]: pgmap v612: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:22 smithi099 ceph-mon[34404]: pgmap v612: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:13:23.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:13:24.025 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 -- 172.21.15.92:0/2310130515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 msgr2=0x7faff810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 --2- 172.21.15.92:0/2310130515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff810f550 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7faff0009ca0 tx=0x7faff002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 -- 172.21.15.92:0/2310130515 shutdown_connections 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 --2- 172.21.15.92:0/2310130515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff810f550 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 --2- 172.21.15.92:0/2310130515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff81083d0 0x7faff81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 --2- 172.21.15.92:0/2310130515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff8102ae0 0x7faff8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 -- 172.21.15.92:0/2310130515 >> 172.21.15.92:0/2310130515 conn(0x7faff8079760 msgr2=0x7faff8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 -- 172.21.15.92:0/2310130515 shutdown_connections 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.517+0000 7faffdb2f640 1 -- 172.21.15.92:0/2310130515 wait complete. 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 Processor -- start 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 -- start start 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8102ae0 0x7faff819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff81083d0 0x7faff819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faff81129f0 con 0x7faff8108d20 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faff8112870 con 0x7faff8102ae0 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faffdb2f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faff8112b70 con 0x7faff81083d0 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faff7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faff7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37742/0 (socket says 172.21.15.92:37742) 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faff7fff640 1 -- 172.21.15.92:0/1262596690 learned_addr learned my addr 172.21.15.92:0/1262596690 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7faff77fe640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8102ae0 0x7faff819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.518+0000 7fafeeffd640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff81083d0 0x7faff819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff7fff640 1 -- 172.21.15.92:0/1262596690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff81083d0 msgr2=0x7faff819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff7fff640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff81083d0 0x7faff819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff7fff640 1 -- 172.21.15.92:0/1262596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8102ae0 msgr2=0x7faff819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff7fff640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8102ae0 0x7faff819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff7fff640 1 -- 172.21.15.92:0/1262596690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff81a0480 con 0x7faff8108d20 2024-09-21T02:13:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff77fe640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8102ae0 0x7faff819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7fafeeffd640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff81083d0 0x7faff819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff7fff640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff819fd50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7faff0005cc0 tx=0x7faff002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff57fa640 1 -- 172.21.15.92:0/1262596690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0003cf0 con 0x7faff8108d20 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff57fa640 1 -- 172.21.15.92:0/1262596690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff00044f0 con 0x7faff8108d20 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff81a0710 con 0x7faff8108d20 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faff57fa640 1 -- 172.21.15.92:0/1262596690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff002fd10 con 0x7faff8108d20 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.519+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a6ed0 con 0x7faff8108d20 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.521+0000 7fb0040a3740 1 Processor -- start 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7faff57fa640 1 -- 172.21.15.92:0/1262596690 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7faff00305c0 con 0x7faff8108d20 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 -- start start 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 0x5600a79246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a7924be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a7925120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5600a789aea0 con 0x5600a757d070 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600a788a2d0 con 0x5600a755c390 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fb0040a3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600a788e0e0 con 0x5600a75541b0 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fafeeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 0x5600a79246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fafeeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 0x5600a79246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37754/0 (socket says 172.21.15.92:37754) 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fafeeffd640 1 -- 172.21.15.92:0/159684698 learned_addr learned my addr 172.21.15.92:0/159684698 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7faff77fe640 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a7924be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.522+0000 7fafeeffd640 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a7925120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafeeffd640 1 -- 172.21.15.92:0/159684698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 msgr2=0x5600a7924be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafeeffd640 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a7924be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafeeffd640 1 -- 172.21.15.92:0/159684698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 msgr2=0x5600a7925120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafeeffd640 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a7925120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafeeffd640 1 -- 172.21.15.92:0/159684698 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a75e8650 con 0x5600a757d070 2024-09-21T02:13:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7faff57fa640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fafe40b6510 0x7fafe40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7faff77fe640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fafe40b6510 0x7fafe40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafeeffd640 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 0x5600a79246a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fafe0015620 tx=0x7fafe000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3289aafe8e4c6034 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7faff57fa640 1 -- 172.21.15.92:0/1262596690 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7faff0108a30 con 0x7faff8108d20 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafedffb640 1 -- 172.21.15.92:0/159684698 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe001d070 con 0x5600a757d070 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafedffb640 1 -- 172.21.15.92:0/159684698 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe0018b20 con 0x5600a757d070 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.523+0000 7fafedffb640 1 -- 172.21.15.92:0/159684698 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000f040 con 0x5600a757d070 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 -- 172.21.15.92:0/159684698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 msgr2=0x5600a79246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7faff77fe640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fafe40b6510 0x7fafe40b8a20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5600a7923320 tx=0x7fafe800c4f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 0x5600a79246a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fafe0015620 tx=0x7fafe000a3f0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 -- 172.21.15.92:0/159684698 shutdown_connections 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a7925120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a7924be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 --2- 172.21.15.92:0/159684698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a757d070 0x5600a79246a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 -- 172.21.15.92:0/159684698 >> 172.21.15.92:0/159684698 conn(0x5600a7571460 msgr2=0x5600a7922590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 -- 172.21.15.92:0/159684698 shutdown_connections 2024-09-21T02:13:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 -- 172.21.15.92:0/159684698 wait complete. 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.524+0000 7fb0040a3740 1 -- start start 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 Processor -- start 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a79302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a757d070 0x5600a793c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5600a76eb700 con 0x5600a75541b0 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600a7825160 con 0x5600a755c390 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600a7932e20 con 0x5600a757d070 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fb0040a3740 10 client.0 _ll_register_callbacks cb 0x5600a783e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37768/0 (socket says 172.21.15.92:37768) 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 -- 172.21.15.92:0/222118682 learned_addr learned my addr 172.21.15.92:0/222118682 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff7fff640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a757d070 0x5600a793c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fafeeffd640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a79302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 -- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a757d070 msgr2=0x5600a793c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a757d070 0x5600a793c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 -- 172.21.15.92:0/222118682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 msgr2=0x5600a79302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a79302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff77fe640 1 -- 172.21.15.92:0/222118682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a7852c90 con 0x5600a75541b0 2024-09-21T02:13:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7fafeeffd640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a79302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.525+0000 7faff7fff640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a757d070 0x5600a793c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7faff77fe640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792fd70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fafe801cdd0 tx=0x7fafe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe0ea92070fe5d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fafd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fafd6ffd640 1 -- 172.21.15.92:0/222118682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8029060 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe0018b20 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fafd6ffd640 1 -- 172.21.15.92:0/222118682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fafe800ed40 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fafd6ffd640 1 -- 172.21.15.92:0/222118682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fafe801a460 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fb0040a3740 10 client.17940 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.526+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5600a793eb80 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fafd6ffd640 1 -- 172.21.15.92:0/222118682 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe0018b20 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fafd6ffd640 1 -- 172.21.15.92:0/222118682 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fafe8018040 con 0x5600a75541b0 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fb0040a3740 10 client.17940 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fb0040a3740 10 client.17940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fb0040a3740 20 client.17940 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fb0040a3740 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a753b9f0 0x5600a7942050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fb0040a3740 4 client.17940 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600a789cdf0 con 0x5600a753b9f0 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.527+0000 7fafeeffd640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a753b9f0 0x5600a7942050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.528+0000 7fafd6ffd640 10 client.17940 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.528+0000 7fafeeffd640 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a753b9f0 0x5600a7942050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86f9be3a508b00c2 server_cookie=a6586fa9d6b6bb50 in_seq=0 out_seq=0 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fafd6ffd640 1 -- 172.21.15.92:0/222118682 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600a789cdf0 con 0x5600a753b9f0 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fafd6ffd640 10 client.17940 handle_command_reply: tid=0 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 client.17940 shutdown 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 msgr2=0x5600a792fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792fd70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fafe801cdd0 tx=0x7fafe80046c0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 shutdown_connections 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a753b9f0 0x5600a7942050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a757d070 0x5600a793c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a755c390 0x5600a79302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 --2- 172.21.15.92:0/222118682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792fd70 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 >> 172.21.15.92:0/222118682 conn(0x5600a7571460 msgr2=0x5600a7927880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fafd6ffd640 0 client.17940 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:13:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 shutdown_connections 2024-09-21T02:13:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 1 -- 172.21.15.92:0/222118682 wait complete. 2024-09-21T02:13:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.529+0000 7fb0040a3740 20 client.17940 trim_cache size 0 max 16384 2024-09-21T02:13:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 Processor -- start 2024-09-21T02:13:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 -- start start 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a755c390 0x5600a7925de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a793c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faff81af610 con 0x5600a75541b0 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faff81b1150 con 0x5600a757d070 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.542+0000 7fb0040a3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faff81afce0 con 0x5600a755c390 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37776/0 (socket says 172.21.15.92:37776) 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 -- 172.21.15.92:0/3173332794 learned_addr learned my addr 172.21.15.92:0/3173332794 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7fafeeffd640 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a755c390 0x5600a7925de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 -- 172.21.15.92:0/3173332794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a755c390 msgr2=0x5600a7925de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a755c390 0x5600a7925de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 -- 172.21.15.92:0/3173332794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 msgr2=0x5600a793c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a793c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 -- 172.21.15.92:0/3173332794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a7927500 con 0x5600a75541b0 2024-09-21T02:13:24.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7faff77fe640 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792ec60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fafe8016040 tx=0x7fafe802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee2feb7f5f92de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7fafed7fa640 1 -- 172.21.15.92:0/3173332794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe800ecc0 con 0x5600a75541b0 2024-09-21T02:13:24.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7fafed7fa640 1 -- 172.21.15.92:0/3173332794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe80054e0 con 0x5600a75541b0 2024-09-21T02:13:24.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.543+0000 7fafed7fa640 1 -- 172.21.15.92:0/3173332794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8004c30 con 0x5600a75541b0 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 -- 172.21.15.92:0/3173332794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 msgr2=0x5600a792ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792ec60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fafe8016040 tx=0x7fafe802fc00 comp rx=0 tx=0).stop 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 -- 172.21.15.92:0/3173332794 shutdown_connections 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a793c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a755c390 0x5600a7925de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 --2- 172.21.15.92:0/3173332794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a75541b0 0x5600a792ec60 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 -- 172.21.15.92:0/3173332794 >> 172.21.15.92:0/3173332794 conn(0x5600a7571460 msgr2=0x5600a79256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 -- 172.21.15.92:0/3173332794 shutdown_connections 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.544+0000 7fb0040a3740 1 -- 172.21.15.92:0/3173332794 wait complete. 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 -- start start 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 Processor -- start 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a79340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 0x5600a79345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a7934b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faff81b0a80 con 0x5600a755c390 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faff81b03b0 con 0x5600a757d070 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faff81ad5f0 con 0x5600a75541b0 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fb0040a3740 10 client.0 _ll_register_callbacks cb 0x7fafe800c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fafeeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 0x5600a79345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fafeeffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 0x5600a79345f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37780/0 (socket says 172.21.15.92:37780) 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7fafeeffd640 1 -- 172.21.15.92:0/4098161537 learned_addr learned my addr 172.21.15.92:0/4098161537 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7faff77fe640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a79340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.545+0000 7faff7fff640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a7934b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafeeffd640 1 -- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 msgr2=0x5600a79340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafeeffd640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a79340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafeeffd640 1 -- 172.21.15.92:0/4098161537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 msgr2=0x5600a7934b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafeeffd640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a7934b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafeeffd640 1 -- 172.21.15.92:0/4098161537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a79aba40 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafeeffd640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 0x5600a79345f0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fafe0015620 tx=0x7fafe0015f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1bdb0f8127f81e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafd6ffd640 1 -- 172.21.15.92:0/4098161537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe001d040 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafd6ffd640 1 -- 172.21.15.92:0/4098161537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fafe0013700 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe80054e0 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafd6ffd640 1 -- 172.21.15.92:0/4098161537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fafe0039070 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fb0040a3740 10 client.17946 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5600a79accc0 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.546+0000 7fafd6ffd640 1 -- 172.21.15.92:0/4098161537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe80054e0 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fafd6ffd640 1 -- 172.21.15.92:0/4098161537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fafe0015040 con 0x5600a755c390 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fb0040a3740 10 client.17946 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fb0040a3740 10 client.17946 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fb0040a3740 20 client.17946 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fb0040a3740 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a7510da0 0x5600a79af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fb0040a3740 4 client.17946 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:13:24.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600a78988a0 con 0x5600a7510da0 2024-09-21T02:13:24.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7faff77fe640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a7510da0 0x5600a79af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7fafd6ffd640 10 client.17946 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.547+0000 7faff77fe640 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a7510da0 0x5600a79af090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9b119e92d777a32 server_cookie=ceb6c2de468f9458 in_seq=0 out_seq=0 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fafd6ffd640 1 -- 172.21.15.92:0/4098161537 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600a78988a0 con 0x5600a7510da0 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fafd6ffd640 10 client.17946 handle_command_reply: tid=0 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 client.17946 shutdown 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 msgr2=0x5600a79345f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 0x5600a79345f0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fafe0015620 tx=0x7fafe0015f70 comp rx=0 tx=0).stop 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 shutdown_connections 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5600a7510da0 0x5600a79af090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a757d070 0x5600a7934b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5600a755c390 0x5600a79345f0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fafd6ffd640 0 client.17946 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 --2- 172.21.15.92:0/4098161537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600a75541b0 0x5600a79340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 >> 172.21.15.92:0/4098161537 conn(0x5600a7571460 msgr2=0x5600a7931200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 shutdown_connections 2024-09-21T02:13:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 1 -- 172.21.15.92:0/4098161537 wait complete. 2024-09-21T02:13:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.548+0000 7fb0040a3740 20 client.17946 trim_cache size 0 max 16384 2024-09-21T02:13:24.554 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:13:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fafe40b6510 msgr2=0x7fafe40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fafe40b6510 0x7fafe40b8a20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5600a7923320 tx=0x7fafe800c4f0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 msgr2=0x7faff819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff819fd50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7faff0005cc0 tx=0x7faff002f470 comp rx=0 tx=0).stop 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 shutdown_connections 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fafe40b6510 0x7fafe40b8a20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faff8108d20 0x7faff819fd50 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faff81083d0 0x7faff819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 --2- 172.21.15.92:0/1262596690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faff8102ae0 0x7faff819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 >> 172.21.15.92:0/1262596690 conn(0x7faff8079760 msgr2=0x7faff806bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 shutdown_connections 2024-09-21T02:13:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:24.549+0000 7faffdb2f640 1 -- 172.21.15.92:0/1262596690 wait complete. 2024-09-21T02:13:25.158 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:24 smithi092 ceph-mon[31190]: pgmap v613: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:13:25.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:13:25.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:24 smithi136 ceph-mon[34754]: pgmap v613: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:13:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:24 smithi099 ceph-mon[34404]: pgmap v613: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:13:25.432 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:25.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.925+0000 7f6dd1a79640 1 -- 172.21.15.92:0/1562707168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1020d0 msgr2=0x7f6dcc1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.925+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/1562707168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1020d0 0x7f6dcc1024e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009900 tx=0x7f6dbc02a410 comp rx=0 tx=0).stop 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 -- 172.21.15.92:0/1562707168 shutdown_connections 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/1562707168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dcc106c10 0x7f6dcc110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/1562707168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc1066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/1562707168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1020d0 0x7f6dcc1024e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 -- 172.21.15.92:0/1562707168 >> 172.21.15.92:0/1562707168 conn(0x7f6dcc079e30 msgr2=0x7f6dcc07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 -- 172.21.15.92:0/1562707168 shutdown_connections 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 -- 172.21.15.92:0/1562707168 wait complete. 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 Processor -- start 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.926+0000 7f6dd1a79640 1 -- start start 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dcc1020d0 0x7f6dcc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc106c10 0x7f6dcc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc113ab0 con 0x7f6dcc1062c0 2024-09-21T02:13:25.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc113930 con 0x7f6dcc106c10 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc113c30 con 0x7f6dcc1020d0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37788/0 (socket says 172.21.15.92:37788) 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3472641469 learned_addr learned my addr 172.21.15.92:0/3472641469 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc106c10 0x7f6dcc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dcc1020d0 0x7f6dcc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3472641469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dcc1020d0 msgr2=0x7f6dcc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dcc1020d0 0x7f6dcc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3472641469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc106c10 msgr2=0x7f6dcc19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc106c10 0x7f6dcc19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3472641469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc1a0470 con 0x7f6dcc1062c0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc106c10 0x7f6dcc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc19aaa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000d770 tx=0x7f6dc000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dc8ff9640 1 -- 172.21.15.92:0/3472641469 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0017070 con 0x7f6dcc1062c0 2024-09-21T02:13:25.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc1a0760 con 0x7f6dcc1062c0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dc8ff9640 1 -- 172.21.15.92:0/3472641469 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc0009c30 con 0x7f6dcc1062c0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.927+0000 7f6dc8ff9640 1 -- 172.21.15.92:0/3472641469 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000f620 con 0x7f6dcc1062c0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.928+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc1a6ec0 con 0x7f6dcc1062c0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.930+0000 7f6dc8ff9640 1 -- 172.21.15.92:0/3472641469 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f6dc001a070 con 0x7f6dcc1062c0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.930+0000 7f6dd7fed740 1 Processor -- start 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.930+0000 7f6dd7fed740 1 -- start start 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5238f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fbf1b0 0x55ec5238fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fc7390 0x55ec52390120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec52305ea0 con 0x55ec51fe8070 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ec522f52d0 con 0x55ec51fc7390 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec522f90e0 con 0x55ec51fbf1b0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5238f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5238f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37796/0 (socket says 172.21.15.92:37796) 2024-09-21T02:13:25.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 -- 172.21.15.92:0/4041551853 learned_addr learned my addr 172.21.15.92:0/4041551853 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fbf1b0 0x55ec5238fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fc7390 0x55ec52390120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 -- 172.21.15.92:0/4041551853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fbf1b0 msgr2=0x55ec5238fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fbf1b0 0x55ec5238fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 -- 172.21.15.92:0/4041551853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fc7390 msgr2=0x55ec52390120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fc7390 0x55ec52390120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 -- 172.21.15.92:0/4041551853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec52053650 con 0x55ec51fe8070 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dc8ff9640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6db40b6510 0x7f6db40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5238f6a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009f30 tx=0x7f6dc4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadfb45efdfa93ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6db40b6510 0x7f6db40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.931+0000 7f6dc8ff9640 1 -- 172.21.15.92:0/3472641469 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6dc00e6ef0 con 0x7f6dcc1062c0 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.932+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6db40b6510 0x7f6db40b8a20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc031440 tx=0x7f6dbc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.933+0000 7f6db1ffb640 1 -- 172.21.15.92:0/4041551853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4016070 con 0x55ec51fe8070 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.933+0000 7f6db1ffb640 1 -- 172.21.15.92:0/4041551853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc401bb10 con 0x55ec51fe8070 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.933+0000 7f6db1ffb640 1 -- 172.21.15.92:0/4041551853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4012bb0 con 0x55ec51fe8070 2024-09-21T02:13:25.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 -- 172.21.15.92:0/4041551853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 msgr2=0x55ec5238f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5238f6a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009f30 tx=0x7f6dc4010800 comp rx=0 tx=0).stop 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 -- 172.21.15.92:0/4041551853 shutdown_connections 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fc7390 0x55ec52390120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fbf1b0 0x55ec5238fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/4041551853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5238f6a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 -- 172.21.15.92:0/4041551853 >> 172.21.15.92:0/4041551853 conn(0x55ec51fdc460 msgr2=0x55ec5238d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 -- 172.21.15.92:0/4041551853 shutdown_connections 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 -- 172.21.15.92:0/4041551853 wait complete. 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.934+0000 7f6dd7fed740 1 -- start start 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 Processor -- start 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 0x55ec5239f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fe8070 0x55ec523ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec52156700 con 0x55ec51fbf1b0 2024-09-21T02:13:25.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ec52290160 con 0x55ec51fe8070 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec523a21e0 con 0x55ec51fc7390 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dd7fed740 10 client.0 _ll_register_callbacks cb 0x55ec522a99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 0x55ec5239f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 0x55ec5239f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37802/0 (socket says 172.21.15.92:37802) 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 -- 172.21.15.92:0/3837945411 learned_addr learned my addr 172.21.15.92:0/3837945411 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fe8070 0x55ec523ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 -- 172.21.15.92:0/3837945411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 msgr2=0x55ec5239f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 -- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fe8070 msgr2=0x55ec523ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fe8070 0x55ec523ab510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.935+0000 7f6dcb7fe640 1 -- 172.21.15.92:0/3837945411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec522bdc90 con 0x55ec51fbf1b0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 0x55ec5239f1c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc005ab0 tx=0x7f6dbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd873a837c0c0ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:25.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6daaffd640 1 -- 172.21.15.92:0/3837945411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc04c030 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6daaffd640 1 -- 172.21.15.92:0/3837945411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6dbc0035f0 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc401bb10 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6daaffd640 1 -- 172.21.15.92:0/3837945411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dbc057070 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 10 client.17958 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec523adf60 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6daaffd640 1 -- 172.21.15.92:0/3837945411 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc401bb10 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6daaffd640 1 -- 172.21.15.92:0/3837945411 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6dbc048880 con 0x55ec51fbf1b0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 10 client.17958 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 10 client.17958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 20 client.17958 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.936+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51fa69f0 0x55ec523b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.937+0000 7f6dd7fed740 4 client.17958 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.937+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec52307df0 con 0x55ec51fa69f0 2024-09-21T02:13:25.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.937+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51fa69f0 0x55ec523b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.937+0000 7f6daaffd640 10 client.17958 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.937+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51fa69f0 0x55ec523b14e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d4ed58e9013b11e server_cookie=65f64cb802760605 in_seq=0 out_seq=0 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6daaffd640 1 -- 172.21.15.92:0/3837945411 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec52307df0 con 0x55ec51fa69f0 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6daaffd640 10 client.17958 handle_command_reply: tid=0 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 client.17958 shutdown 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 msgr2=0x55ec5239f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 0x55ec5239f1c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc005ab0 tx=0x7f6dbc009260 comp rx=0 tx=0).stop 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 shutdown_connections 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fe8070 0x55ec523ab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51fa69f0 0x55ec523b14e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3837945411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fbf1b0 0x55ec5239f1c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 >> 172.21.15.92:0/3837945411 conn(0x55ec51fdc460 msgr2=0x55ec52391090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.938+0000 7f6daaffd640 0 client.17958 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.939+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 shutdown_connections 2024-09-21T02:13:25.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.939+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3837945411 wait complete. 2024-09-21T02:13:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.939+0000 7f6dd7fed740 20 client.17958 trim_cache size 0 max 16384 2024-09-21T02:13:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 Processor -- start 2024-09-21T02:13:25.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 -- start start 2024-09-21T02:13:25.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec52390de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5239c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ad360 con 0x55ec51fe8070 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1aeea0 con 0x55ec51fbf1b0 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ada30 con 0x55ec51fc7390 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5239c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5239c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37812/0 (socket says 172.21.15.92:37812) 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 -- 172.21.15.92:0/770690359 learned_addr learned my addr 172.21.15.92:0/770690359 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec52390de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 -- 172.21.15.92:0/770690359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 msgr2=0x55ec5239c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 -- 172.21.15.92:0/770690359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 msgr2=0x55ec52390de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec52390de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 -- 172.21.15.92:0/770690359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec52394e70 con 0x55ec51fe8070 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.952+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5239c8f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4012860 tx=0x7f6dc4009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637b18513f0828b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6db17fa640 1 -- 172.21.15.92:0/770690359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4016050 con 0x55ec51fe8070 2024-09-21T02:13:25.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6db17fa640 1 -- 172.21.15.92:0/770690359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc402a070 con 0x55ec51fe8070 2024-09-21T02:13:25.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6db17fa640 1 -- 172.21.15.92:0/770690359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4012e60 con 0x55ec51fe8070 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 -- 172.21.15.92:0/770690359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 msgr2=0x55ec5239c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5239c8f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4012860 tx=0x7f6dc4009890 comp rx=0 tx=0).stop 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 -- 172.21.15.92:0/770690359 shutdown_connections 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fe8070 0x55ec5239c8f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fc7390 0x55ec5239c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/770690359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec52390de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.953+0000 7f6dd7fed740 1 -- 172.21.15.92:0/770690359 >> 172.21.15.92:0/770690359 conn(0x55ec51fdc460 msgr2=0x55ec523906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 -- 172.21.15.92:0/770690359 shutdown_connections 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 -- 172.21.15.92:0/770690359 wait complete. 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 -- start start 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 Processor -- start 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec523a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 0x55ec523a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fe8070 0x55ec52412ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ae7d0 con 0x55ec51fc7390 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ae100 con 0x55ec51fbf1b0 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ab340 con 0x55ec51fe8070 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.954+0000 7f6dd7fed740 10 client.0 _ll_register_callbacks cb 0x7f6dbc0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 0x55ec523a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 0x55ec523a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37828/0 (socket says 172.21.15.92:37828) 2024-09-21T02:13:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3775243744 learned_addr learned my addr 172.21.15.92:0/3775243744 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec523a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcbfff640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fe8070 0x55ec52412ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3775243744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fe8070 msgr2=0x55ec52412ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fe8070 0x55ec52412ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 msgr2=0x55ec523a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec523a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 -- 172.21.15.92:0/3775243744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec52413540 con 0x55ec51fc7390 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dcaffd640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 0x55ec523a5530 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00e49d0 tx=0x7f6dc000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4db54366384056 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6daaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6daaffd640 1 -- 172.21.15.92:0/3775243744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0021040 con 0x55ec51fc7390 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6daaffd640 1 -- 172.21.15.92:0/3775243744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6dc00ac4c0 con 0x55ec51fc7390 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc402a070 con 0x55ec51fc7390 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6daaffd640 1 -- 172.21.15.92:0/3775243744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dc00ac7d0 con 0x55ec51fc7390 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dd7fed740 10 client.17964 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.955+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec5241ba40 con 0x55ec51fc7390 2024-09-21T02:13:25.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6daaffd640 1 -- 172.21.15.92:0/3775243744 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc402a070 con 0x55ec51fc7390 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6daaffd640 1 -- 172.21.15.92:0/3775243744 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6dc00981b0 con 0x55ec51fc7390 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dd7fed740 10 client.17964 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dd7fed740 10 client.17964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dd7fed740 20 client.17964 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51f7bda0 0x55ec5241e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dd7fed740 4 client.17964 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec523038a0 con 0x55ec51f7bda0 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51f7bda0 0x55ec5241e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.956+0000 7f6daaffd640 10 client.17964 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dcb7fe640 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51f7bda0 0x55ec5241e180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c378d82f5793f0d server_cookie=bfb0e16eec7fc8e7 in_seq=0 out_seq=0 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6daaffd640 1 -- 172.21.15.92:0/3775243744 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec523038a0 con 0x55ec51f7bda0 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6daaffd640 10 client.17964 handle_command_reply: tid=0 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 client.17964 shutdown 2024-09-21T02:13:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 msgr2=0x55ec523a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 0x55ec523a5530 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00e49d0 tx=0x7f6dc000de80 comp rx=0 tx=0).stop 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 shutdown_connections 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec51fe8070 0x55ec52412ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x55ec51f7bda0 0x55ec5241e180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec51fc7390 0x55ec523a5530 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 --2- 172.21.15.92:0/3775243744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec51fbf1b0 0x55ec523a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 >> 172.21.15.92:0/3775243744 conn(0x55ec51fdc460 msgr2=0x55ec523a05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6daaffd640 0 client.17964 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 shutdown_connections 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 1 -- 172.21.15.92:0/3775243744 wait complete. 2024-09-21T02:13:25.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.957+0000 7f6dd7fed740 20 client.17964 trim_cache size 0 max 16384 2024-09-21T02:13:25.964 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:13:25.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6db40b6510 msgr2=0x7f6db40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6db40b6510 0x7f6db40b8a20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc031440 tx=0x7f6dbc005a20 comp rx=0 tx=0).stop 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 msgr2=0x7f6dcc19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc19aaa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000d770 tx=0x7f6dc000dc30 comp rx=0 tx=0).stop 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 shutdown_connections 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6db40b6510 0x7f6db40b8a20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dcc106c10 0x7f6dcc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dcc1062c0 0x7f6dcc19aaa0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 --2- 172.21.15.92:0/3472641469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dcc1020d0 0x7f6dcc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 >> 172.21.15.92:0/3472641469 conn(0x7f6dcc079e30 msgr2=0x7f6dcc1080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 shutdown_connections 2024-09-21T02:13:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:25.958+0000 7f6dd1a79640 1 -- 172.21.15.92:0/3472641469 wait complete. 2024-09-21T02:13:26.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:13:27.024 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:27.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:26 smithi092 ceph-mon[31190]: pgmap v614: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:13:27.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:26 smithi136 ceph-mon[34754]: pgmap v614: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:13:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:26 smithi099 ceph-mon[34404]: pgmap v614: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:13:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 -- 172.21.15.92:0/4277850636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c8109e40 msgr2=0x7f05c8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 --2- 172.21.15.92:0/4277850636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c8109e40 0x7f05c8110670 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009900 tx=0x7f05b802a410 comp rx=0 tx=0).stop 2024-09-21T02:13:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 -- 172.21.15.92:0/4277850636 shutdown_connections 2024-09-21T02:13:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 --2- 172.21.15.92:0/4277850636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c8109e40 0x7f05c8110670 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 --2- 172.21.15.92:0/4277850636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c81094f0 0x7f05c8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 --2- 172.21.15.92:0/4277850636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8103bb0 0x7f05c8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 -- 172.21.15.92:0/4277850636 >> 172.21.15.92:0/4277850636 conn(0x7f05c80ff600 msgr2=0x7f05c8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 -- 172.21.15.92:0/4277850636 shutdown_connections 2024-09-21T02:13:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.535+0000 7f05cff7d640 1 -- 172.21.15.92:0/4277850636 wait complete. 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.536+0000 7f05cff7d640 1 Processor -- start 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.536+0000 7f05cff7d640 1 -- start start 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.536+0000 7f05cff7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8103bb0 0x7f05c819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.536+0000 7f05cff7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 0x7f05c819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cff7d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c8109e40 0x7f05c819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cff7d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8113b00 con 0x7f05c81094f0 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cff7d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8113980 con 0x7f05c8103bb0 2024-09-21T02:13:27.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cff7d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8113c80 con 0x7f05c8109e40 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 0x7f05c819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 0x7f05c819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37854/0 (socket says 172.21.15.92:37854) 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 -- 172.21.15.92:0/1825773088 learned_addr learned my addr 172.21.15.92:0/1825773088 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cdd19640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8103bb0 0x7f05c819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05ce51a640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c8109e40 0x7f05c819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 -- 172.21.15.92:0/1825773088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c8109e40 msgr2=0x7f05c819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c8109e40 0x7f05c819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 -- 172.21.15.92:0/1825773088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8103bb0 msgr2=0x7f05c819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8103bb0 0x7f05c819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 -- 172.21.15.92:0/1825773088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c81a03e0 con 0x7f05c81094f0 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cd518640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 0x7f05c819aa10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00d750 tx=0x7f05bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05c6ffd640 1 -- 172.21.15.92:0/1825773088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc017070 con 0x7f05c81094f0 2024-09-21T02:13:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05c6ffd640 1 -- 172.21.15.92:0/1825773088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05bc0054d0 con 0x7f05c81094f0 2024-09-21T02:13:27.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81a06d0 con 0x7f05c81094f0 2024-09-21T02:13:27.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05c6ffd640 1 -- 172.21.15.92:0/1825773088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004c30 con 0x7f05c81094f0 2024-09-21T02:13:27.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.537+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81a6e30 con 0x7f05c81094f0 2024-09-21T02:13:27.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.540+0000 7f05c6ffd640 1 -- 172.21.15.92:0/1825773088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f05bc004dd0 con 0x7f05c81094f0 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.540+0000 7f05d64f1740 1 Processor -- start 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.540+0000 7f05d64f1740 1 -- start start 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 0x55b2441956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b244195be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b244196120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05d64f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b24410bea0 con 0x55b243dee070 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05d64f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b2440fb2d0 con 0x55b243dcd390 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05d64f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2440ff0e0 con 0x55b243dc51b0 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 0x55b2441956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 0x55b2441956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37856/0 (socket says 172.21.15.92:37856) 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05cd518640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b244196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 -- 172.21.15.92:0/4154567321 learned_addr learned my addr 172.21.15.92:0/4154567321 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05cdd19640 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b244195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05c6ffd640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f05b00b6590 0x7f05b00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 -- 172.21.15.92:0/4154567321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 msgr2=0x55b244195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b244195be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 -- 172.21.15.92:0/4154567321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 msgr2=0x55b244196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b244196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 -- 172.21.15.92:0/4154567321 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b243e59650 con 0x55b243dee070 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05ce51a640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f05b00b6590 0x7f05b00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.541+0000 7f05c6ffd640 1 -- 172.21.15.92:0/1825773088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f05bc0e6dc0 con 0x7f05c81094f0 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.542+0000 7f05ce51a640 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 0x55b2441956a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f05b8005940 tx=0x7f05b8032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336f34c2c625c16d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.542+0000 7f05affff640 1 -- 172.21.15.92:0/4154567321 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8035070 con 0x55b243dee070 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.542+0000 7f05affff640 1 -- 172.21.15.92:0/4154567321 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b80054f0 con 0x55b243dee070 2024-09-21T02:13:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.542+0000 7f05affff640 1 -- 172.21.15.92:0/4154567321 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8004930 con 0x55b243dee070 2024-09-21T02:13:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 -- 172.21.15.92:0/4154567321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 msgr2=0x55b2441956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 0x55b2441956a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f05b8005940 tx=0x7f05b8032d40 comp rx=0 tx=0).stop 2024-09-21T02:13:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05ce51a640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f05b00b6590 0x7f05b00b8aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f05b8005f70 tx=0x7f05b8045000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 -- 172.21.15.92:0/4154567321 shutdown_connections 2024-09-21T02:13:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b244196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b244195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 --2- 172.21.15.92:0/4154567321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dee070 0x55b2441956a0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 -- 172.21.15.92:0/4154567321 >> 172.21.15.92:0/4154567321 conn(0x55b243de2460 msgr2=0x55b244193590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 -- 172.21.15.92:0/4154567321 shutdown_connections 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 -- 172.21.15.92:0/4154567321 wait complete. 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 -- start start 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.543+0000 7f05d64f1740 1 Processor -- start 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b2441a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dee070 0x55b2441ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b243f5c700 con 0x55b243dcd390 2024-09-21T02:13:27.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b244096160 con 0x55b243dee070 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2441a3e80 con 0x55b243dc51b0 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05d64f1740 10 client.0 _ll_register_callbacks cb 0x55b2440af9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b2441a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b2441a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37868/0 (socket says 172.21.15.92:37868) 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 -- 172.21.15.92:0/3932216944 learned_addr learned my addr 172.21.15.92:0/3932216944 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cdd19640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05ce51a640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dee070 0x55b2441ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 -- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 msgr2=0x55b2441a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 -- 172.21.15.92:0/3932216944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dee070 msgr2=0x55b2441ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dee070 0x55b2441ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 -- 172.21.15.92:0/3932216944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2440c3c90 con 0x55b243dcd390 2024-09-21T02:13:27.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05cd518640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b2441a12b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0ace60 tx=0x7f05bc0ea3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e023ab2b0aea337a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.544+0000 7f05ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3932216944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc020050 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3932216944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f05bc0940e0 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b80054f0 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3932216944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05bc0ac9e0 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 10 client.17976 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b2441afc10 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3932216944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b80054f0 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3932216944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f05bc0980c0 con 0x55b243dcd390 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 10 client.17976 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 20 client.17976 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dac9f0 0x55b2441b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 4 client.17976 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:27.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.545+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b24410ddf0 con 0x55b243dac9f0 2024-09-21T02:13:27.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.546+0000 7f05cdd19640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dac9f0 0x55b2441b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.546+0000 7f05cdd19640 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dac9f0 0x55b2441b30f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5da9aa16e60afb server_cookie=d437a622d21d594a in_seq=0 out_seq=0 2024-09-21T02:13:27.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.546+0000 7f05ad7fa640 10 client.17976 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3932216944 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b24410ddf0 con 0x55b243dac9f0 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05ad7fa640 10 client.17976 handle_command_reply: tid=0 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 client.17976 shutdown 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 msgr2=0x55b2441a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b2441a12b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0ace60 tx=0x7f05bc0ea3f0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 shutdown_connections 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dac9f0 0x55b2441b30f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dee070 0x55b2441ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b2441a12b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05ad7fa640 0 client.17976 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3932216944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 >> 172.21.15.92:0/3932216944 conn(0x55b243de2460 msgr2=0x55b244198880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 shutdown_connections 2024-09-21T02:13:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 1 -- 172.21.15.92:0/3932216944 wait complete. 2024-09-21T02:13:27.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.547+0000 7f05d64f1740 20 client.17976 trim_cache size 0 max 16384 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.560+0000 7f05d64f1740 1 Processor -- start 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.560+0000 7f05d64f1740 1 -- start start 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.560+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dac9f0 0x55b244198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.560+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dc51b0 0x55b2441ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.560+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b244192eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.560+0000 7f05d64f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81af410 con 0x55b243dcd390 2024-09-21T02:13:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05d64f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81b0f50 con 0x55b243dc51b0 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05d64f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81afae0 con 0x55b243dac9f0 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b244192eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b244192eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37878/0 (socket says 172.21.15.92:37878) 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 -- 172.21.15.92:0/1861967862 learned_addr learned my addr 172.21.15.92:0/1861967862 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05cdd19640 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dac9f0 0x55b244198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05cd518640 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dc51b0 0x55b2441ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 -- 172.21.15.92:0/1861967862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dac9f0 msgr2=0x55b244198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dac9f0 0x55b244198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 -- 172.21.15.92:0/1861967862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dc51b0 msgr2=0x55b2441ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dc51b0 0x55b2441ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 -- 172.21.15.92:0/1861967862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b244197bc0 con 0x55b243dcd390 2024-09-21T02:13:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05cdd19640 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dac9f0 0x55b244198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05ce51a640 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b244192eb0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009a20 tx=0x7f05b8032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c73ad656b5890d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05af7fe640 1 -- 172.21.15.92:0/1861967862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8035040 con 0x55b243dcd390 2024-09-21T02:13:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05af7fe640 1 -- 172.21.15.92:0/1861967862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b8004850 con 0x55b243dcd390 2024-09-21T02:13:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.561+0000 7f05af7fe640 1 -- 172.21.15.92:0/1861967862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b80326e0 con 0x55b243dcd390 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 -- 172.21.15.92:0/1861967862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 msgr2=0x55b244192eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b244192eb0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009a20 tx=0x7f05b8032f30 comp rx=0 tx=0).stop 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 -- 172.21.15.92:0/1861967862 shutdown_connections 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dcd390 0x55b244192eb0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dc51b0 0x55b2441ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 --2- 172.21.15.92:0/1861967862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dac9f0 0x55b244198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 -- 172.21.15.92:0/1861967862 >> 172.21.15.92:0/1861967862 conn(0x55b243de2460 msgr2=0x55b2441987a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 -- 172.21.15.92:0/1861967862 shutdown_connections 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.562+0000 7f05d64f1740 1 -- 172.21.15.92:0/1861967862 wait complete. 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 -- start start 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 Processor -- start 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 0x55b2441a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b24421caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81b0880 con 0x55b243dac9f0 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81b01b0 con 0x55b243dcd390 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81ad3f0 con 0x55b243dc51b0 2024-09-21T02:13:27.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05d64f1740 10 client.0 _ll_register_callbacks cb 0x7f05bc00de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05cdd19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 0x55b2441a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05cdd19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 0x55b2441a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37882/0 (socket says 172.21.15.92:37882) 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.563+0000 7f05cdd19640 1 -- 172.21.15.92:0/3805548314 learned_addr learned my addr 172.21.15.92:0/3805548314 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cd518640 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cdd19640 1 -- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 msgr2=0x55b2441a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cdd19640 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cdd19640 1 -- 172.21.15.92:0/3805548314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 msgr2=0x55b24421caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cdd19640 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b24421caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cdd19640 1 -- 172.21.15.92:0/3805548314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2441a6200 con 0x55b243dac9f0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05cdd19640 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 0x55b2441a5420 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x55b244194320 tx=0x7f05c0012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cad43f493dc734d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3805548314 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0029060 con 0x55b243dac9f0 2024-09-21T02:13:27.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3805548314 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f05c001ead0 con 0x55b243dac9f0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b8004850 con 0x55b243dac9f0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3805548314 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05c0012e10 con 0x55b243dac9f0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05d64f1740 10 client.17982 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b24421e0c0 con 0x55b243dac9f0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.564+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3805548314 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b8004850 con 0x55b243dac9f0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3805548314 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f05c000f040 con 0x55b243dac9f0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05d64f1740 10 client.17982 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05d64f1740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05d64f1740 20 client.17982 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dee070 0x55b2442208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05d64f1740 4 client.17982 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2441098a0 con 0x55b243dee070 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.565+0000 7f05cd518640 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dee070 0x55b2442208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05ad7fa640 10 client.17982 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05cd518640 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dee070 0x55b2442208b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e48942984cdc9dd server_cookie=adc0faa3fa0d1ed3 in_seq=0 out_seq=0 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05ad7fa640 1 -- 172.21.15.92:0/3805548314 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2441098a0 con 0x55b243dee070 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05ad7fa640 10 client.17982 handle_command_reply: tid=0 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 client.17982 shutdown 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 msgr2=0x55b2441a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 0x55b2441a5420 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x55b244194320 tx=0x7f05c0012b60 comp rx=0 tx=0).stop 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 shutdown_connections 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b243dee070 0x55b2442208b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b243dcd390 0x55b24421caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05ad7fa640 0 client.17982 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b243dc51b0 0x55b2441a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 --2- 172.21.15.92:0/3805548314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b243dac9f0 0x55b2441a5420 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 >> 172.21.15.92:0/3805548314 conn(0x55b243de2460 msgr2=0x55b244192e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 shutdown_connections 2024-09-21T02:13:27.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 1 -- 172.21.15.92:0/3805548314 wait complete. 2024-09-21T02:13:27.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.566+0000 7f05d64f1740 20 client.17982 trim_cache size 0 max 16384 2024-09-21T02:13:27.573 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:13:27.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f05b00b6590 msgr2=0x7f05b00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f05b00b6590 0x7f05b00b8aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f05b8005f70 tx=0x7f05b8045000 comp rx=0 tx=0).stop 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 msgr2=0x7f05c819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 0x7f05c819aa10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00d750 tx=0x7f05bc00dc10 comp rx=0 tx=0).stop 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 shutdown_connections 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f05b00b6590 0x7f05b00b8aa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c8109e40 0x7f05c819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c81094f0 0x7f05c819aa10 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 --2- 172.21.15.92:0/1825773088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8103bb0 0x7f05c819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 >> 172.21.15.92:0/1825773088 conn(0x7f05c80ff600 msgr2=0x7f05c8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 shutdown_connections 2024-09-21T02:13:27.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:27.567+0000 7f05cff7d640 1 -- 172.21.15.92:0/1825773088 wait complete. 2024-09-21T02:13:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.908+0000 7f1f90f20640 1 -- 172.21.15.92:0/429986577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c1025d0 msgr2=0x7f1f8c102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.908+0000 7f1f90f20640 1 --2- 172.21.15.92:0/429986577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c1025d0 0x7f1f8c102a40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009900 tx=0x7f1f7c02a410 comp rx=0 tx=0).stop 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 -- 172.21.15.92:0/429986577 shutdown_connections 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 --2- 172.21.15.92:0/429986577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c103ff0 0x7f1f8c10a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 --2- 172.21.15.92:0/429986577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103650 0x7f1f8c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 --2- 172.21.15.92:0/429986577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c1025d0 0x7f1f8c102a40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 -- 172.21.15.92:0/429986577 >> 172.21.15.92:0/429986577 conn(0x7f1f8c0fdf30 msgr2=0x7f1f8c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 -- 172.21.15.92:0/429986577 shutdown_connections 2024-09-21T02:13:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 -- 172.21.15.92:0/429986577 wait complete. 2024-09-21T02:13:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 Processor -- start 2024-09-21T02:13:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.909+0000 7f1f90f20640 1 -- start start 2024-09-21T02:13:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f90f20640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c1025d0 0x7f1f8c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f90f20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c103650 0x7f1f8c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f90f20640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103ff0 0x7f1f8c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f90f20640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10e210 con 0x7f1f8c103650 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f90f20640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10e090 con 0x7f1f8c103ff0 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f90f20640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10e390 con 0x7f1f8c1025d0 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f8ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103ff0 0x7f1f8c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f8ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103ff0 0x7f1f8c1a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:49326/0 (socket says 172.21.15.92:49326) 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f8ad9d640 1 -- 172.21.15.92:0/639991234 learned_addr learned my addr 172.21.15.92:0/639991234 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c103650 0x7f1f8c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c1025d0 0x7f1f8c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 -- 172.21.15.92:0/639991234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c1025d0 msgr2=0x7f1f8c06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c1025d0 0x7f1f8c06c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 -- 172.21.15.92:0/639991234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103ff0 msgr2=0x7f1f8c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103ff0 0x7f1f8c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 -- 172.21.15.92:0/639991234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c1a19d0 con 0x7f1f8c103650 2024-09-21T02:13:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.910+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c103650 0x7f1f8c19ec80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000d6d0 tx=0x7f1f8000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.911+0000 7f1f7b7fe640 1 -- 172.21.15.92:0/639991234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80017070 con 0x7f1f8c103650 2024-09-21T02:13:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.911+0000 7f1f7b7fe640 1 -- 172.21.15.92:0/639991234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f80009820 con 0x7f1f8c103650 2024-09-21T02:13:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.911+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1a1cc0 con 0x7f1f8c103650 2024-09-21T02:13:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.911+0000 7f1f7b7fe640 1 -- 172.21.15.92:0/639991234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80004c40 con 0x7f1f8c103650 2024-09-21T02:13:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.911+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1a21c0 con 0x7f1f8c103650 2024-09-21T02:13:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.913+0000 7f1f97e8d740 1 Processor -- start 2024-09-21T02:13:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.913+0000 7f1f97e8d740 1 -- start start 2024-09-21T02:13:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bec903d6f0 con 0x55bec8cfa2f0 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bec902cb20 con 0x55bec8d231b0 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bec9030930 con 0x55bec8d024d0 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37920/0 (socket says 172.21.15.92:37920) 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 -- 172.21.15.92:0/2444298558 learned_addr learned my addr 172.21.15.92:0/2444298558 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f7b7fe640 1 -- 172.21.15.92:0/639991234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1f80005220 con 0x7f1f8c103650 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 -- 172.21.15.92:0/2444298558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 msgr2=0x55bec90cc010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90cc010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 -- 172.21.15.92:0/2444298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 msgr2=0x55bec90cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 -- 172.21.15.92:0/2444298558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec8ff2530 con 0x55bec8cfa2f0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.914+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90cb4c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c02a930 tx=0x7f1f7c031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581e0650d5ed37f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.915+0000 7f1f78ff9640 1 -- 172.21.15.92:0/2444298558 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c035030 con 0x55bec8cfa2f0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.915+0000 7f1f78ff9640 1 -- 172.21.15.92:0/2444298558 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c004040 con 0x55bec8cfa2f0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.915+0000 7f1f78ff9640 1 -- 172.21.15.92:0/2444298558 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c005590 con 0x55bec8cfa2f0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.915+0000 7f1f7b7fe640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1f740b6510 0x7f1f740b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.915+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1f740b6510 0x7f1f740b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.915+0000 7f1f7b7fe640 1 -- 172.21.15.92:0/639991234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1f800e6e10 con 0x7f1f8c103650 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2444298558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 msgr2=0x55bec90cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1f740b6510 0x7f1f740b8a20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009270 tx=0x7f1f84009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90cb4c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c02a930 tx=0x7f1f7c031620 comp rx=0 tx=0).stop 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2444298558 shutdown_connections 2024-09-21T02:13:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90cb4c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2444298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2444298558 >> 172.21.15.92:0/2444298558 conn(0x55bec8d17300 msgr2=0x55bec90c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2444298558 shutdown_connections 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2444298558 wait complete. 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- start start 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 Processor -- start 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8cfa2f0 0x55bec90d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90d4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d231b0 0x55bec90d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bec8f64ab0 con 0x55bec8d024d0 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bec8fd2730 con 0x55bec8cfa2f0 2024-09-21T02:13:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.916+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bec90ccad0 con 0x55bec8d231b0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f97e8d740 10 client.0 _ll_register_callbacks cb 0x55bec90266b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90d4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90d4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37928/0 (socket says 172.21.15.92:37928) 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 -- 172.21.15.92:0/2510854754 learned_addr learned my addr 172.21.15.92:0/2510854754 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8cfa2f0 0x55bec90d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d231b0 0x55bec90d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 -- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d231b0 msgr2=0x55bec90d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d231b0 0x55bec90d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 -- 172.21.15.92:0/2510854754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8cfa2f0 msgr2=0x55bec90d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8cfa2f0 0x55bec90d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 -- 172.21.15.92:0/2510854754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec8d8e790 con 0x55bec8d024d0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8cfa2f0 0x55bec90d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90d4d10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1f800acc80 tx=0x7f1f800ac6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84f721b6e5dd8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f71ffb640 1 -- 172.21.15.92:0/2510854754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80020070 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f71ffb640 1 -- 172.21.15.92:0/2510854754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1f800a91f0 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f7c004040 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f71ffb640 1 -- 172.21.15.92:0/2510854754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1f800ad680 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f97e8d740 10 client.17994 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bec8e07620 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.917+0000 7f1f71ffb640 1 -- 172.21.15.92:0/2510854754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c004040 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f71ffb640 1 -- 172.21.15.92:0/2510854754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1f800940b0 con 0x55bec8d024d0 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f97e8d740 10 client.17994 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f97e8d740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f97e8d740 20 client.17994 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8ce1d80 0x55bec90e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f97e8d740 4 client.17994 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bec903f640 con 0x55bec8ce1d80 2024-09-21T02:13:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8ce1d80 0x55bec90e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f71ffb640 10 client.17994 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.918+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8ce1d80 0x55bec90e1d20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a93377323ee8f36 server_cookie=e25f68e84246c566 in_seq=0 out_seq=0 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f71ffb640 1 -- 172.21.15.92:0/2510854754 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bec903f640 con 0x55bec8ce1d80 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f71ffb640 10 client.17994 handle_command_reply: tid=0 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 client.17994 shutdown 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 msgr2=0x55bec90d4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90d4d10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1f800acc80 tx=0x7f1f800ac6f0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 shutdown_connections 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8ce1d80 0x55bec90e1d20 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d231b0 0x55bec90d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f71ffb640 0 client.17994 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90d4d10 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/2510854754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8cfa2f0 0x55bec90d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 >> 172.21.15.92:0/2510854754 conn(0x55bec8d17300 msgr2=0x55bec90c74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 shutdown_connections 2024-09-21T02:13:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.919+0000 7f1f97e8d740 1 -- 172.21.15.92:0/2510854754 wait complete. 2024-09-21T02:13:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.920+0000 7f1f97e8d740 20 client.17994 trim_cache size 0 max 16384 2024-09-21T02:13:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 Processor -- start 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 -- start start 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8cfa2f0 0x55bec90c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1aaf00 con 0x55bec8d024d0 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1aca40 con 0x55bec8d231b0 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ab5d0 con 0x55bec8cfa2f0 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37938/0 (socket says 172.21.15.92:37938) 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 -- 172.21.15.92:0/1868075100 learned_addr learned my addr 172.21.15.92:0/1868075100 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8cfa2f0 0x55bec90c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 -- 172.21.15.92:0/1868075100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8cfa2f0 msgr2=0x55bec90c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8cfa2f0 0x55bec90c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 -- 172.21.15.92:0/1868075100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 msgr2=0x55bec90d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.935+0000 7f1f89d9b640 1 -- 172.21.15.92:0/1868075100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec90cb8c0 con 0x55bec8d024d0 2024-09-21T02:13:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.936+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90c7b90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1f800aa1f0 tx=0x7f1f80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522303f2292554cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.936+0000 7f1f73fff640 1 -- 172.21.15.92:0/1868075100 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8005c070 con 0x55bec8d024d0 2024-09-21T02:13:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.936+0000 7f1f73fff640 1 -- 172.21.15.92:0/1868075100 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f800b69a0 con 0x55bec8d024d0 2024-09-21T02:13:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.936+0000 7f1f73fff640 1 -- 172.21.15.92:0/1868075100 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80020070 con 0x55bec8d024d0 2024-09-21T02:13:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1868075100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 msgr2=0x55bec90c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90c7b90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1f800aa1f0 tx=0x7f1f80004730 comp rx=0 tx=0).stop 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1868075100 shutdown_connections 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8d024d0 0x55bec90c7b90 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1868075100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8cfa2f0 0x55bec90c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1868075100 >> 172.21.15.92:0/1868075100 conn(0x55bec8d17300 msgr2=0x55bec90c7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1868075100 shutdown_connections 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.937+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1868075100 wait complete. 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 -- start start 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 Processor -- start 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ac370 con 0x55bec8cfa2f0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1abca0 con 0x55bec8d231b0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1a8ee0 con 0x55bec8d024d0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f97e8d740 10 client.0 _ll_register_callbacks cb 0x7f1f800057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f8a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f8a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37942/0 (socket says 172.21.15.92:37942) 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f8a59c640 1 -- 172.21.15.92:0/1043546236 learned_addr learned my addr 172.21.15.92:0/1043546236 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f8ad9d640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.938+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f8a59c640 1 -- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 msgr2=0x55bec90db150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90db150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f8a59c640 1 -- 172.21.15.92:0/1043546236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 msgr2=0x55bec90db730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90db730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f8a59c640 1 -- 172.21.15.92:0/1043546236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec90dbec0 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f8a59c640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90dc490 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005f70 tx=0x7f1f7c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bd9bccdb8b63c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f71ffb640 1 -- 172.21.15.92:0/1043546236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c035040 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f71ffb640 1 -- 172.21.15.92:0/1043546236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1f7c031ce0 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f800b69a0 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f71ffb640 1 -- 172.21.15.92:0/1043546236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c004030 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f97e8d740 10 client.18000 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bec914c750 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f71ffb640 1 -- 172.21.15.92:0/1043546236 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f800b69a0 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.939+0000 7f1f71ffb640 1 -- 172.21.15.92:0/1043546236 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1f7c04c070 con 0x55bec8cfa2f0 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f97e8d740 10 client.18000 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f97e8d740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f97e8d740 20 client.18000 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8cb71c0 0x55bec914ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f97e8d740 4 client.18000 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bec903b0f0 con 0x55bec8cb71c0 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8cb71c0 0x55bec914ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f71ffb640 10 client.18000 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.940+0000 7f1f89d9b640 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8cb71c0 0x55bec914ed90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e7ffcd6ef6d16b server_cookie=ba3268171dd8e97a in_seq=0 out_seq=0 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f71ffb640 1 -- 172.21.15.92:0/1043546236 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bec903b0f0 con 0x55bec8cb71c0 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f71ffb640 10 client.18000 handle_command_reply: tid=0 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 client.18000 shutdown 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 msgr2=0x55bec90dc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90dc490 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005f70 tx=0x7f1f7c0055f0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 shutdown_connections 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55bec8cb71c0 0x55bec914ed90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec8d231b0 0x55bec90db730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bec8d024d0 0x55bec90db150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 --2- 172.21.15.92:0/1043546236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bec8cfa2f0 0x55bec90dc490 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f71ffb640 0 client.18000 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 >> 172.21.15.92:0/1043546236 conn(0x55bec8d17300 msgr2=0x55bec90ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 shutdown_connections 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 1 -- 172.21.15.92:0/1043546236 wait complete. 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.961+0000 7f1f97e8d740 20 client.18000 trim_cache size 0 max 16384 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:13:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:13:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1f740b6510 msgr2=0x7f1f740b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1f740b6510 0x7f1f740b8a20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009270 tx=0x7f1f84009680 comp rx=0 tx=0).stop 2024-09-21T02:13:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c103650 msgr2=0x7f1f8c19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c103650 0x7f1f8c19ec80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000d6d0 tx=0x7f1f8000db90 comp rx=0 tx=0).stop 2024-09-21T02:13:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 shutdown_connections 2024-09-21T02:13:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1f740b6510 0x7f1f740b8a20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c103ff0 0x7f1f8c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c103650 0x7f1f8c19ec80 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 --2- 172.21.15.92:0/639991234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c1025d0 0x7f1f8c06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 >> 172.21.15.92:0/639991234 conn(0x7f1f8c0fdf30 msgr2=0x7f1f8c106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 shutdown_connections 2024-09-21T02:13:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:27.962+0000 7f1f90f20640 1 -- 172.21.15.92:0/639991234 wait complete. 2024-09-21T02:13:29.270 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:13:29.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:13:29.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:28 smithi136 ceph-mon[34754]: pgmap v615: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:29.296 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:28 smithi092 ceph-mon[31190]: pgmap v615: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:28 smithi099 ceph-mon[34404]: pgmap v615: 97 pgs: 97 active+clean; 2.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:29.524 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 -- 172.21.15.92:0/4217073894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 msgr2=0x7ff3cc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4217073894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc103f70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009900 tx=0x7ff3bc02a410 comp rx=0 tx=0).stop 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 -- 172.21.15.92:0/4217073894 shutdown_connections 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4217073894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc109df0 0x7ff3cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4217073894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc1094a0 0x7ff3cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4217073894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc103f70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 -- 172.21.15.92:0/4217073894 >> 172.21.15.92:0/4217073894 conn(0x7ff3cc07b850 msgr2=0x7ff3cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 -- 172.21.15.92:0/4217073894 shutdown_connections 2024-09-21T02:13:30.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.038+0000 7ff3d1321640 1 -- 172.21.15.92:0/4217073894 wait complete. 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 Processor -- start 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 -- start start 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc1094a0 0x7ff3cc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc109df0 0x7ff3cc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc113ab0 con 0x7ff3cc103b60 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc113930 con 0x7ff3cc109df0 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3d1321640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc113c30 con 0x7ff3cc1094a0 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37966/0 (socket says 172.21.15.92:37966) 2024-09-21T02:13:30.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.039+0000 7ff3caffd640 1 -- 172.21.15.92:0/4018064803 learned_addr learned my addr 172.21.15.92:0/4018064803 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc1094a0 0x7ff3cc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3caffd640 1 -- 172.21.15.92:0/4018064803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc1094a0 msgr2=0x7ff3cc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3caffd640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc1094a0 0x7ff3cc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3caffd640 1 -- 172.21.15.92:0/4018064803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc109df0 msgr2=0x7ff3cc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc109df0 0x7ff3cc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3caffd640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc109df0 0x7ff3cc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3caffd640 1 -- 172.21.15.92:0/4018064803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc1a0410 con 0x7ff3cc103b60 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc1094a0 0x7ff3cc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc109df0 0x7ff3cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3caffd640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc19a4a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc006010 tx=0x7ff3bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3b3fff640 1 -- 172.21.15.92:0/4018064803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc03f030 con 0x7ff3cc103b60 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3b3fff640 1 -- 172.21.15.92:0/4018064803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc003800 con 0x7ff3cc103b60 2024-09-21T02:13:30.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc1a06a0 con 0x7ff3cc103b60 2024-09-21T02:13:30.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3b3fff640 1 -- 172.21.15.92:0/4018064803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc035070 con 0x7ff3cc103b60 2024-09-21T02:13:30.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.040+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1a6e60 con 0x7ff3cc103b60 2024-09-21T02:13:30.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 Processor -- start 2024-09-21T02:13:30.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 -- start start 2024-09-21T02:13:30.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf88350 0x559f4d32f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 0x559f4d32f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d32feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f4d2a5e60 con 0x559f4cf5f490 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559f4d295290 con 0x559f4cf88350 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3d7895740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f4d2990a0 con 0x559f4cf67670 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3b3fff640 1 -- 172.21.15.92:0/4018064803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff3bc031b10 con 0x7ff3cc103b60 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 0x559f4d32f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 0x559f4d32f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37982/0 (socket says 172.21.15.92:37982) 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3caffd640 1 -- 172.21.15.92:0/4291007773 learned_addr learned my addr 172.21.15.92:0/4291007773 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d32feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.043+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf88350 0x559f4d32f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3caffd640 1 -- 172.21.15.92:0/4291007773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 msgr2=0x559f4d32feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3caffd640 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d32feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3caffd640 1 -- 172.21.15.92:0/4291007773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf88350 msgr2=0x559f4d32f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3caffd640 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf88350 0x559f4d32f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3caffd640 1 -- 172.21.15.92:0/4291007773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f4cff3930 con 0x559f4cf5f490 2024-09-21T02:13:30.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3caffd640 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 0x559f4d32f970 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc10bf90 tx=0x7ff3bc1104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11690f34cb26d7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3b3fff640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3b40b6590 0x7ff3b40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3b17fa640 1 -- 172.21.15.92:0/4291007773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc117070 con 0x559f4cf5f490 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3b17fa640 1 -- 172.21.15.92:0/4291007773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc0cd990 con 0x559f4cf5f490 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3b17fa640 1 -- 172.21.15.92:0/4291007773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0cd7d0 con 0x559f4cf5f490 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3b40b6590 0x7ff3b40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.044+0000 7ff3b3fff640 1 -- 172.21.15.92:0/4018064803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff3bc107d10 con 0x7ff3cc103b60 2024-09-21T02:13:30.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 -- 172.21.15.92:0/4291007773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 msgr2=0x559f4d32f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3b40b6590 0x7ff3b40b8aa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4005fd0 tx=0x7ff3c4011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 0x559f4d32f970 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc10bf90 tx=0x7ff3bc1104d0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 -- 172.21.15.92:0/4291007773 shutdown_connections 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d32feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf5f490 0x559f4d32f970 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 --2- 172.21.15.92:0/4291007773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf88350 0x559f4d32f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 -- 172.21.15.92:0/4291007773 >> 172.21.15.92:0/4291007773 conn(0x559f4cf7c740 msgr2=0x559f4d32d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 -- 172.21.15.92:0/4291007773 shutdown_connections 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.045+0000 7ff3d7895740 1 -- 172.21.15.92:0/4291007773 wait complete. 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 -- start start 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 Processor -- start 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d33b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d346c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f4d0f6890 con 0x559f4cf88350 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559f4d22fd40 con 0x559f4cf5f490 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f4d33d9d0 con 0x559f4cf67670 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3d7895740 10 client.0 _ll_register_callbacks cb 0x559f4d293c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d346c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d346c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37990/0 (socket says 172.21.15.92:37990) 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/1223678588 learned_addr learned my addr 172.21.15.92:0/1223678588 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d33b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.046+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 msgr2=0x559f4d33b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d33b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/1223678588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 msgr2=0x559f4d33afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/1223678588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f4d347170 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d33b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d346c30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400ec70 tx=0x7ff3c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c1c68ffeb589c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1223678588 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c402b060 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1223678588 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff3c400eca0 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc0cd990 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1223678588 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3c4026470 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3d7895740 10 client.18012 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f4d349ae0 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1223678588 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc0cd990 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1223678588 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff3c4031590 con 0x559f4cf88350 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3d7895740 10 client.18012 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:30.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.047+0000 7ff3d7895740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3d7895740 20 client.18012 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf46cd0 0x559f4d34d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3d7895740 4 client.18012 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f4d2a7db0 con 0x559f4cf46cd0 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf46cd0 0x559f4d34d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3aeff5640 10 client.18012 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.048+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf46cd0 0x559f4d34d030 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ac1d778cd501e6 server_cookie=3b0636e3bde349c0 in_seq=0 out_seq=0 2024-09-21T02:13:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1223678588 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f4d2a7db0 con 0x559f4cf46cd0 2024-09-21T02:13:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3aeff5640 10 client.18012 handle_command_reply: tid=0 2024-09-21T02:13:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 client.18012 shutdown 2024-09-21T02:13:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 msgr2=0x559f4d346c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d346c30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400ec70 tx=0x7ff3c4004ab0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 shutdown_connections 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf46cd0 0x559f4d34d030 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d346c30 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf67670 0x559f4d33b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3aeff5640 0 client.18012 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1223678588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 >> 172.21.15.92:0/1223678588 conn(0x559f4cf7c740 msgr2=0x559f4d3325c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 shutdown_connections 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 1 -- 172.21.15.92:0/1223678588 wait complete. 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.049+0000 7ff3d7895740 20 client.18012 trim_cache size 0 max 16384 2024-09-21T02:13:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.066+0000 7ff3d7895740 1 Processor -- start 2024-09-21T02:13:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.066+0000 7ff3d7895740 1 -- start start 2024-09-21T02:13:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf5f490 0x559f4d334540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf67670 0x559f4d334a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d32c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3d7895740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ad2c0 con 0x559f4cf88350 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3d7895740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1aee00 con 0x559f4cf67670 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3d7895740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ad990 con 0x559f4cf5f490 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d32c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d32c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38000/0 (socket says 172.21.15.92:38000) 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/312132078 learned_addr learned my addr 172.21.15.92:0/312132078 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3caffd640 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf5f490 0x559f4d334540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf67670 0x559f4d334a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/312132078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf5f490 msgr2=0x559f4d334540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf5f490 0x559f4d334540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/312132078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf67670 msgr2=0x559f4d334a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf67670 0x559f4d334a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 -- 172.21.15.92:0/312132078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f4d336640 con 0x559f4cf88350 2024-09-21T02:13:30.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf67670 0x559f4d334a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:30.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.067+0000 7ff3cb7fe640 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d32c860 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400ffd0 tx=0x7ff3c400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824d83ae54cc6b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3b0ff9640 1 -- 172.21.15.92:0/312132078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c403e070 con 0x559f4cf88350 2024-09-21T02:13:30.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3b0ff9640 1 -- 172.21.15.92:0/312132078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3c4026610 con 0x559f4cf88350 2024-09-21T02:13:30.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3b0ff9640 1 -- 172.21.15.92:0/312132078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4039690 con 0x559f4cf88350 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 -- 172.21.15.92:0/312132078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 msgr2=0x559f4d32c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d32c860 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400ffd0 tx=0x7ff3c400ef30 comp rx=0 tx=0).stop 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 -- 172.21.15.92:0/312132078 shutdown_connections 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf88350 0x559f4d32c860 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf67670 0x559f4d334a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 --2- 172.21.15.92:0/312132078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf5f490 0x559f4d334540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 -- 172.21.15.92:0/312132078 >> 172.21.15.92:0/312132078 conn(0x559f4cf7c740 msgr2=0x559f4d339630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 -- 172.21.15.92:0/312132078 shutdown_connections 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.068+0000 7ff3d7895740 1 -- 172.21.15.92:0/312132078 wait complete. 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 -- start start 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 Processor -- start 2024-09-21T02:13:30.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 0x559f4d33f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf88350 0x559f4d3b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ae730 con 0x559f4cf67670 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ae060 con 0x559f4cf5f490 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ab2a0 con 0x559f4cf88350 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3d7895740 10 client.0 _ll_register_callbacks cb 0x7ff3c400f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3ca7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 0x559f4d33f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3ca7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 0x559f4d33f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38016/0 (socket says 172.21.15.92:38016) 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.069+0000 7ff3ca7fc640 1 -- 172.21.15.92:0/1688181423 learned_addr learned my addr 172.21.15.92:0/1688181423 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3ca7fc640 1 -- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf88350 msgr2=0x559f4d3b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf88350 0x559f4d3b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3ca7fc640 1 -- 172.21.15.92:0/1688181423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 msgr2=0x559f4d33f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3ca7fc640 1 -- 172.21.15.92:0/1688181423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f4d3b6f80 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3ca7fc640 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 0x559f4d33f780 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc06c760 tx=0x7ff3c0011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be6e733990f83d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1688181423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000e860 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1688181423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff3c000ea00 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c4026610 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1688181423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3c000ec00 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3d7895740 10 client.18018 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.070+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f4d3b81e0 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1688181423 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3c4026610 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1688181423 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff3c0005260 con 0x559f4cf67670 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3d7895740 10 client.18018 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3d7895740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:30.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3d7895740 20 client.18018 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf1c080 0x559f4d3baa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3d7895740 4 client.18018 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:13:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f4d2a3860 con 0x559f4cf1c080 2024-09-21T02:13:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf1c080 0x559f4d3baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3aeff5640 10 client.18018 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:30.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.071+0000 7ff3caffd640 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf1c080 0x559f4d3baa50 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a185ac9e181f596 server_cookie=1faccd7295159e63 in_seq=0 out_seq=0 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3aeff5640 1 -- 172.21.15.92:0/1688181423 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f4d2a3860 con 0x559f4cf1c080 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3aeff5640 10 client.18018 handle_command_reply: tid=0 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 client.18018 shutdown 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 msgr2=0x559f4d33f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 0x559f4d33f780 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc06c760 tx=0x7ff3c0011ec0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 shutdown_connections 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x559f4cf1c080 0x559f4d3baa50 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f4cf88350 0x559f4d3b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3aeff5640 0 client.18018 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f4cf67670 0x559f4d33f780 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 --2- 172.21.15.92:0/1688181423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f4cf5f490 0x559f4d33f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 >> 172.21.15.92:0/1688181423 conn(0x559f4cf7c740 msgr2=0x559f4d334540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 shutdown_connections 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 1 -- 172.21.15.92:0/1688181423 wait complete. 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.073+0000 7ff3d7895740 20 client.18018 trim_cache size 0 max 16384 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "bed9224a-80ce-4634-bff3-0a2e25dd6f67", 2024-09-21T02:13:30.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3b40b6590 msgr2=0x7ff3b40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3b40b6590 0x7ff3b40b8aa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4005fd0 tx=0x7ff3c4011040 comp rx=0 tx=0).stop 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 msgr2=0x7ff3cc19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc19a4a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc006010 tx=0x7ff3bc0042f0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 shutdown_connections 2024-09-21T02:13:30.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff3b40b6590 0x7ff3b40b8aa0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc109df0 0x7ff3cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3cc1094a0 0x7ff3cc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 --2- 172.21.15.92:0/4018064803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc103b60 0x7ff3cc19a4a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 >> 172.21.15.92:0/4018064803 conn(0x7ff3cc07b850 msgr2=0x7ff3cc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 shutdown_connections 2024-09-21T02:13:30.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:30.074+0000 7ff3d1321640 1 -- 172.21.15.92:0/4018064803 wait complete. 2024-09-21T02:13:30.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:13:31.081 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:13:31.201 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:30 smithi092 ceph-mon[31190]: pgmap v616: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:13:31.201 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:30 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:13:31.202 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:30 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:13:31.202 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:30 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:13:31.202 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:30 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:13:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:30 smithi136 ceph-mon[34754]: pgmap v616: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:13:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:30 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:13:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:30 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:13:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:30 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:13:31.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:30 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:13:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:30 smithi099 ceph-mon[34404]: pgmap v616: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:13:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:30 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:13:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:30 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:13:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:30 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:13:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:30 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:13:31.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 -- 172.21.15.92:0/1957505334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c108cf0 msgr2=0x7f1c1c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 --2- 172.21.15.92:0/1957505334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c108cf0 0x7f1c1c10f520 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009ca0 tx=0x7f1c1402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 -- 172.21.15.92:0/1957505334 shutdown_connections 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 --2- 172.21.15.92:0/1957505334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c108cf0 0x7f1c1c10f520 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 --2- 172.21.15.92:0/1957505334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c1c1083a0 0x7f1c1c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 --2- 172.21.15.92:0/1957505334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 -- 172.21.15.92:0/1957505334 >> 172.21.15.92:0/1957505334 conn(0x7f1c1c079730 msgr2=0x7f1c1c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 -- 172.21.15.92:0/1957505334 shutdown_connections 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.529+0000 7f1c21806640 1 -- 172.21.15.92:0/1957505334 wait complete. 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 Processor -- start 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 -- start start 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c1083a0 0x7f1c1c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c1c108cf0 0x7f1c1c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c112960 con 0x7f1c1c1083a0 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1127e0 con 0x7f1c1c102a60 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.530+0000 7f1c21806640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c112ae0 con 0x7f1c1c108cf0 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:49420/0 (socket says 172.21.15.92:49420) 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 -- 172.21.15.92:0/2805319912 learned_addr learned my addr 172.21.15.92:0/2805319912 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c1c108cf0 0x7f1c1c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c127fc640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c1083a0 0x7f1c1c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 -- 172.21.15.92:0/2805319912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c1c108cf0 msgr2=0x7f1c1c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c1c108cf0 0x7f1c1c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 -- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c1083a0 msgr2=0x7f1c1c19b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c1083a0 0x7f1c1c19b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 -- 172.21.15.92:0/2805319912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c1a6500 con 0x7f1c1c102a60 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c127fc640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c1083a0 0x7f1c1c19b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c1affd640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c19b270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00d950 tx=0x7f1c0c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c20804640 1 -- 172.21.15.92:0/2805319912 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c017030 con 0x7f1c1c102a60 2024-09-21T02:13:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c1a67f0 con 0x7f1c1c102a60 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.531+0000 7f1c20804640 1 -- 172.21.15.92:0/2805319912 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c004020 con 0x7f1c1c102a60 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.532+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a6d80 con 0x7f1c1c102a60 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.532+0000 7f1c20804640 1 -- 172.21.15.92:0/2805319912 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0047a0 con 0x7f1c1c102a60 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 Processor -- start 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 -- start start 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed166270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd961b0 0x555aed1667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555aed0dce50 con 0x555aecd961b0 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555aed0cc280 con 0x555aecd9e390 2024-09-21T02:13:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555aed0d0090 con 0x555aecdbf070 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c127fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c127fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:49434/0 (socket says 172.21.15.92:49434) 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c127fc640 1 -- 172.21.15.92:0/1868067060 learned_addr learned my addr 172.21.15.92:0/1868067060 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.534+0000 7f1c20804640 1 -- 172.21.15.92:0/2805319912 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f1c0c009a30 con 0x7f1c1c102a60 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed166270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c1affd640 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd961b0 0x555aed1667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c127fc640 1 -- 172.21.15.92:0/1868067060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 msgr2=0x555aed166270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c127fc640 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed166270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c127fc640 1 -- 172.21.15.92:0/1868067060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd961b0 msgr2=0x555aed1667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c127fc640 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd961b0 0x555aed1667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c127fc640 1 -- 172.21.15.92:0/1868067060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aece2a650 con 0x555aecd9e390 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c127fc640 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed166cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c040103d0 tx=0x7f1c04010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc63ee5cd81b6ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c11ffb640 1 -- 172.21.15.92:0/1868067060 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04005220 con 0x555aecd9e390 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c11ffb640 1 -- 172.21.15.92:0/1868067060 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c04004c50 con 0x555aecd9e390 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c20804640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1c080b6540 0x7f1c080b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c11ffb640 1 -- 172.21.15.92:0/1868067060 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04011a60 con 0x555aecd9e390 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1c080b6540 0x7f1c080b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.535+0000 7f1c20804640 1 -- 172.21.15.92:0/2805319912 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1c0c0e6d20 con 0x7f1c1c102a60 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 -- 172.21.15.92:0/1868067060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 msgr2=0x555aed166cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1c080b6540 0x7f1c080b8a50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1c14006010 tx=0x7f1c14009440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed166cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c040103d0 tx=0x7f1c04010890 comp rx=0 tx=0).stop 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 -- 172.21.15.92:0/1868067060 shutdown_connections 2024-09-21T02:13:31.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed166cf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd961b0 0x555aed1667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/1868067060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed166270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 -- 172.21.15.92:0/1868067060 >> 172.21.15.92:0/1868067060 conn(0x555aecdb3460 msgr2=0x555aed164160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 -- 172.21.15.92:0/1868067060 shutdown_connections 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 -- 172.21.15.92:0/1868067060 wait complete. 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.536+0000 7f1c27d7a740 1 -- start start 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 Processor -- start 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed171900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed171e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed17dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555aecf2d700 con 0x555aecd9e390 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555aed067110 con 0x555aecd961b0 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555aed174a50 con 0x555aecdbf070 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c27d7a740 10 client.0 _ll_register_callbacks cb 0x555aed080990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c127fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed171e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c127fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed171e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38062/0 (socket says 172.21.15.92:38062) 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c1affd640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed171900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c127fc640 1 -- 172.21.15.92:0/423430624 learned_addr learned my addr 172.21.15.92:0/423430624 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.537+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed17dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c127fc640 1 -- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 msgr2=0x555aed17dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c127fc640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed17dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c127fc640 1 -- 172.21.15.92:0/423430624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 msgr2=0x555aed171900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c127fc640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed171900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c127fc640 1 -- 172.21.15.92:0/423430624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aed094c40 con 0x555aecd9e390 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c1affd640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed171900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed17dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c127fc640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed171e40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400bd70 tx=0x7f1c040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d97fe7d355154a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1bfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1bfaffd640 1 -- 172.21.15.92:0/423430624 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04004250 con 0x555aecd9e390 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1bfaffd640 1 -- 172.21.15.92:0/423430624 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1c040043f0 con 0x555aecd9e390 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c04004c50 con 0x555aecd9e390 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1bfaffd640 1 -- 172.21.15.92:0/423430624 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c040117e0 con 0x555aecd9e390 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c27d7a740 10 client.18030 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:31.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.538+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555aed180730 con 0x555aecd9e390 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1bfaffd640 1 -- 172.21.15.92:0/423430624 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c04004c50 con 0x555aecd9e390 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1bfaffd640 1 -- 172.21.15.92:0/423430624 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1c04005310 con 0x555aecd9e390 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c27d7a740 10 client.18030 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c27d7a740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c27d7a740 20 client.18030 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd7d9f0 0x555aed183ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c27d7a740 4 client.18030 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aed0deda0 con 0x555aecd7d9f0 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.539+0000 7f1c1affd640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd7d9f0 0x555aed183ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1c1affd640 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd7d9f0 0x555aed183ae0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8f46dd9eb19743 server_cookie=f9034eca2c4a3799 in_seq=0 out_seq=0 2024-09-21T02:13:31.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1bfaffd640 10 client.18030 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1bfaffd640 1 -- 172.21.15.92:0/423430624 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1c0c086e50 con 0x555aecd7d9f0 2024-09-21T02:13:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1bfaffd640 10 client.18030 handle_command_reply: tid=0 2024-09-21T02:13:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1c27d7a740 1 client.18030 shutdown 2024-09-21T02:13:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 msgr2=0x555aed171e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.540+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed171e40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400bd70 tx=0x7f1c040048f0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 shutdown_connections 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd7d9f0 0x555aed183ae0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed17dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed171e40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1bfaffd640 0 client.18030 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/423430624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed171900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 >> 172.21.15.92:0/423430624 conn(0x555aecdb3460 msgr2=0x555aed169400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 shutdown_connections 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 1 -- 172.21.15.92:0/423430624 wait complete. 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.541+0000 7f1c27d7a740 20 client.18030 trim_cache size 0 max 16384 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 Processor -- start 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 -- start start 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecd961b0 0x555aed16d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed173530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 0x555aed17dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1af4e0 con 0x555aecdbf070 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1b1020 con 0x555aecd9e390 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1afbb0 con 0x555aecd961b0 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 0x555aed17dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 0x555aed17dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38066/0 (socket says 172.21.15.92:38066) 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c1b7fe640 1 -- 172.21.15.92:0/2736572926 learned_addr learned my addr 172.21.15.92:0/2736572926 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c1affd640 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecd961b0 0x555aed16d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.558+0000 7f1c127fc640 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed173530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c1b7fe640 1 -- 172.21.15.92:0/2736572926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecd961b0 msgr2=0x555aed16d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecd961b0 0x555aed16d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c1b7fe640 1 -- 172.21.15.92:0/2736572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 msgr2=0x555aed173530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed173530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c1b7fe640 1 -- 172.21.15.92:0/2736572926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aed1690d0 con 0x555aecdbf070 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 0x555aed17dc40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1c14031e40 tx=0x7f1c14041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638e438013abb15d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c117fa640 1 -- 172.21.15.92:0/2736572926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14003970 con 0x555aecdbf070 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c117fa640 1 -- 172.21.15.92:0/2736572926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14036050 con 0x555aecdbf070 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.559+0000 7f1c117fa640 1 -- 172.21.15.92:0/2736572926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140047f0 con 0x555aecdbf070 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 -- 172.21.15.92:0/2736572926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 msgr2=0x555aed17dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 0x555aed17dc40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1c14031e40 tx=0x7f1c14041000 comp rx=0 tx=0).stop 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 -- 172.21.15.92:0/2736572926 shutdown_connections 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecdbf070 0x555aed17dc40 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd9e390 0x555aed173530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/2736572926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecd961b0 0x555aed16d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 -- 172.21.15.92:0/2736572926 >> 172.21.15.92:0/2736572926 conn(0x555aecdb3460 msgr2=0x555aed172e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 -- 172.21.15.92:0/2736572926 shutdown_connections 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 -- 172.21.15.92:0/2736572926 wait complete. 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 -- start start 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.560+0000 7f1c27d7a740 1 Processor -- start 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed175530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed175af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed1e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1b0950 con 0x555aecd9e390 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1b0280 con 0x555aecd961b0 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ad4c0 con 0x555aecdbf070 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c27d7a740 10 client.0 _ll_register_callbacks cb 0x7f1c0400be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed175af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed175af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38074/0 (socket says 172.21.15.92:38074) 2024-09-21T02:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 -- 172.21.15.92:0/3361570343 learned_addr learned my addr 172.21.15.92:0/3361570343 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c1affd640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed175530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c1b7fe640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed1e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 -- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 msgr2=0x555aed1e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed1e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 -- 172.21.15.92:0/3361570343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 msgr2=0x555aed175530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed175530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 -- 172.21.15.92:0/3361570343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aed1e6ab0 con 0x555aecd9e390 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1c127fc640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed175af0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1c040108d0 tx=0x7f1c0400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f7078d84495259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.561+0000 7f1bfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1bfaffd640 1 -- 172.21.15.92:0/3361570343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04035060 con 0x555aecd9e390 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1bfaffd640 1 -- 172.21.15.92:0/3361570343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f1c04003af0 con 0x555aecd9e390 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14036050 con 0x555aecd9e390 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1bfaffd640 1 -- 172.21.15.92:0/3361570343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c04003cf0 con 0x555aecd9e390 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 10 client.18036 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555aed1eddc0 con 0x555aecd9e390 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1bfaffd640 1 -- 172.21.15.92:0/3361570343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c14036050 con 0x555aecd9e390 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1bfaffd640 1 -- 172.21.15.92:0/3361570343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1c0401b960 con 0x555aecd9e390 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 10 client.18036 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 20 client.18036 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd52da0 0x555aed1f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 4 client.18036 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.562+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555aed0da850 con 0x555aecd52da0 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c1affd640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd52da0 0x555aed1f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c1affd640 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd52da0 0x555aed1f0490 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5220a3b672c0b4 server_cookie=df995c639c9f56c1 in_seq=0 out_seq=0 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1bfaffd640 10 client.18036 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1bfaffd640 1 -- 172.21.15.92:0/3361570343 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x555aed0da850 con 0x555aecd52da0 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1bfaffd640 10 client.18036 handle_command_reply: tid=0 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 client.18036 shutdown 2024-09-21T02:13:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 msgr2=0x555aed175af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed175af0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1c040108d0 tx=0x7f1c0400bc80 comp rx=0 tx=0).stop 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 shutdown_connections 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x555aecd52da0 0x555aed1f0490 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555aecdbf070 0x555aed1e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555aecd9e390 0x555aed175af0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1bfaffd640 0 client.18036 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 --2- 172.21.15.92:0/3361570343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aecd961b0 0x555aed175530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.563+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 >> 172.21.15.92:0/3361570343 conn(0x555aecdb3460 msgr2=0x555aed172c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 shutdown_connections 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c27d7a740 1 -- 172.21.15.92:0/3361570343 wait complete. 2024-09-21T02:13:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c27d7a740 20 client.18036 trim_cache size 0 max 16384 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "bed9224a-80ce-4634-bff3-0a2e25dd6f67", 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:13:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:13:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:13:30.072031+0000", 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:13:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:13:31.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1c080b6540 msgr2=0x7f1c080b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1c080b6540 0x7f1c080b8a50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1c14006010 tx=0x7f1c14009440 comp rx=0 tx=0).stop 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 msgr2=0x7f1c1c19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c19b270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00d950 tx=0x7f1c0c00de10 comp rx=0 tx=0).stop 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 shutdown_connections 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f1c080b6540 0x7f1c080b8a50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c1c108cf0 0x7f1c1c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c1c1083a0 0x7f1c1c19b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 --2- 172.21.15.92:0/2805319912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c1c102a60 0x7f1c1c19b270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 >> 172.21.15.92:0/2805319912 conn(0x7f1c1c079730 msgr2=0x7f1c1c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 shutdown_connections 2024-09-21T02:13:31.637 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:13:31.564+0000 7f1c21806640 1 -- 172.21.15.92:0/2805319912 wait complete. 2024-09-21T02:13:32.324 INFO:tasks.cephfs.filesystem:scrub status for tag:bed9224a-80ce-4634-bff3-0a2e25dd6f67 - {'path': '/', 'tag': 'bed9224a-80ce-4634-bff3-0a2e25dd6f67', 'options': 'recursive,force'} 2024-09-21T02:13:33.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:32 smithi136 ceph-mon[34754]: pgmap v617: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:13:33.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:32 smithi092 ceph-mon[31190]: pgmap v617: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:13:33.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:32 smithi099 ceph-mon[34404]: pgmap v617: 97 pgs: 97 active+clean; 2.5 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T02:13:34.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:33 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:13:34.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:33 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:13:34.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:33 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:13:35.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:34 smithi136 ceph-mon[34754]: pgmap v618: 97 pgs: 97 active+clean; 2.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:13:35.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:34 smithi092 ceph-mon[31190]: pgmap v618: 97 pgs: 97 active+clean; 2.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:13:35.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:34 smithi099 ceph-mon[34404]: pgmap v618: 97 pgs: 97 active+clean; 2.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:13:37.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:36 smithi136 ceph-mon[34754]: pgmap v619: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:13:37.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:36 smithi092 ceph-mon[31190]: pgmap v619: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:13:37.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:36 smithi099 ceph-mon[34404]: pgmap v619: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:13:39.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:38 smithi136 ceph-mon[34754]: pgmap v620: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:13:39.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:38 smithi092 ceph-mon[31190]: pgmap v620: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:13:39.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:38 smithi099 ceph-mon[34404]: pgmap v620: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:13:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/3552623540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0102c80 msgr2=0x7fd6c01050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/3552623540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0102c80 0x7fd6c01050a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009900 tx=0x7fd6b002a410 comp rx=0 tx=0).stop 2024-09-21T02:13:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/3552623540 shutdown_connections 2024-09-21T02:13:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/3552623540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c0108300 0x7fd6c010a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/3552623540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c01056e0 0x7fd6c0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/3552623540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0102c80 0x7fd6c01050a0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/3552623540 >> 172.21.15.92:0/3552623540 conn(0x7fd6c00fd690 msgr2=0x7fd6c00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/3552623540 shutdown_connections 2024-09-21T02:13:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.070+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/3552623540 wait complete. 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.071+0000 7fd6c4bcc640 1 Processor -- start 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.071+0000 7fd6c4bcc640 1 -- start start 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.071+0000 7fd6c4bcc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c0102c80 0x7fd6c0199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c01056e0 0x7fd6c019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 0x7fd6c019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c010f310 con 0x7fd6c0108300 2024-09-21T02:13:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c010f190 con 0x7fd6c0102c80 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c010f490 con 0x7fd6c01056e0 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 0x7fd6c019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 0x7fd6c019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56882/0 (socket says 172.21.15.92:56882) 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 -- 172.21.15.92:0/2666896377 learned_addr learned my addr 172.21.15.92:0/2666896377 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c0102c80 0x7fd6c0199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c01056e0 0x7fd6c019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 -- 172.21.15.92:0/2666896377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c01056e0 msgr2=0x7fd6c019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c01056e0 0x7fd6c019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 -- 172.21.15.92:0/2666896377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c0102c80 msgr2=0x7fd6c0199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c0102c80 0x7fd6c0199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 -- 172.21.15.92:0/2666896377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c019b9a0 con 0x7fd6c0108300 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6bed9d640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 0x7fd6c019b180 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8007ee0 tx=0x7fd6b800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6af7fe640 1 -- 172.21.15.92:0/2666896377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800dd90 con 0x7fd6c0108300 2024-09-21T02:13:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6af7fe640 1 -- 172.21.15.92:0/2666896377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b80138a0 con 0x7fd6c0108300 2024-09-21T02:13:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c019bc90 con 0x7fd6c0108300 2024-09-21T02:13:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6af7fe640 1 -- 172.21.15.92:0/2666896377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8012740 con 0x7fd6c0108300 2024-09-21T02:13:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.072+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c01a2380 con 0x7fd6c0108300 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.075+0000 7fd6af7fe640 1 -- 172.21.15.92:0/2666896377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd6b801a020 con 0x7fd6c0108300 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.075+0000 7fd6cbc14740 1 Processor -- start 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.075+0000 7fd6cbc14740 1 -- start start 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbda6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9de4d0 0x556acbda8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556acbd196f0 con 0x556acb9d62f0 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556acbd08b20 con 0x556acb9ff1b0 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556acbd0c930 con 0x556acb9de4d0 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56892/0 (socket says 172.21.15.92:56892) 2024-09-21T02:13:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 -- 172.21.15.92:0/2706992325 learned_addr learned my addr 172.21.15.92:0/2706992325 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbda6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9de4d0 0x556acbda8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6af7fe640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6a80b6510 0x7fd6a80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 -- 172.21.15.92:0/2706992325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9de4d0 msgr2=0x556acbda8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6bed9d640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6a80b6510 0x7fd6a80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9de4d0 0x556acbda8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 -- 172.21.15.92:0/2706992325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 msgr2=0x556acbda6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbda6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 -- 172.21.15.92:0/2706992325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556acbcce530 con 0x556acb9d62f0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6af7fe640 1 -- 172.21.15.92:0/2666896377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd6b80e6ef0 con 0x7fd6c0108300 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.076+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda74c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd6b002a930 tx=0x7fd6b0031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9146bf4335938f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.077+0000 7fd6acff9640 1 -- 172.21.15.92:0/2706992325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0035030 con 0x556acb9d62f0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.077+0000 7fd6bed9d640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6a80b6510 0x7fd6a80b8a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80e4d40 tx=0x7fd6b8078c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.077+0000 7fd6acff9640 1 -- 172.21.15.92:0/2706992325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b0040070 con 0x556acb9d62f0 2024-09-21T02:13:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.077+0000 7fd6acff9640 1 -- 172.21.15.92:0/2706992325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0003950 con 0x556acb9d62f0 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2706992325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 msgr2=0x556acbda74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda74c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd6b002a930 tx=0x7fd6b0031bc0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2706992325 shutdown_connections 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9de4d0 0x556acbda8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda74c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2706992325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbda6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2706992325 >> 172.21.15.92:0/2706992325 conn(0x556acb9f3300 msgr2=0x556acbda28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2706992325 shutdown_connections 2024-09-21T02:13:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2706992325 wait complete. 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- start start 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 Processor -- start 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbdb0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 0x556acbdb1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556acbc40ab0 con 0x556acb9ff1b0 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556acbcae730 con 0x556acb9de4d0 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556acbda8ad0 con 0x556acb9d62f0 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.078+0000 7fd6cbc14740 10 client.0 _ll_register_callbacks cb 0x556acbd026b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 0x556acbdb1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 0x556acbdb1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56898/0 (socket says 172.21.15.92:56898) 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/2958105403 learned_addr learned my addr 172.21.15.92:0/2958105403 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbdb0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 msgr2=0x556acbdb0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/2958105403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 msgr2=0x556acbdb0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbdb0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/2958105403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556acba6a790 con 0x556acb9ff1b0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 0x556acbdb1750 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4012bd0 tx=0x7fd6b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6887643f7ffdb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/2958105403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400ebe0 con 0x556acb9ff1b0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/2958105403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd6b400ed80 con 0x556acb9ff1b0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b0040070 con 0x556acb9ff1b0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/2958105403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6b4027070 con 0x556acb9ff1b0 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6cbc14740 10 client.18048 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.079+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556acbae3620 con 0x556acb9ff1b0 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/2958105403 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b0040070 con 0x556acb9ff1b0 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/2958105403 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6b40047a0 con 0x556acb9ff1b0 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6cbc14740 10 client.18048 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6cbc14740 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6cbc14740 20 client.18048 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9bdd80 0x556acbdbdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6cbc14740 4 client.18048 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556acbd1b640 con 0x556acb9bdd80 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.080+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9bdd80 0x556acbdbdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.081+0000 7fd6be59c640 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9bdd80 0x556acbdbdce0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155cfac641346872 server_cookie=eb18c17d202ea86b in_seq=0 out_seq=0 2024-09-21T02:13:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.081+0000 7fd6a5ffb640 10 client.18048 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.081+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/2958105403 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556acbd1b640 con 0x556acb9bdd80 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.081+0000 7fd6a5ffb640 10 client.18048 handle_command_reply: tid=0 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.081+0000 7fd6cbc14740 1 client.18048 shutdown 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 msgr2=0x556acbdb1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 0x556acbdb1750 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4012bd0 tx=0x7fd6b4004730 comp rx=0 tx=0).stop 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 shutdown_connections 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9bdd80 0x556acbdbdce0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9ff1b0 0x556acbdb1750 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbdb0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/2958105403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6a5ffb640 0 client.18048 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 >> 172.21.15.92:0/2958105403 conn(0x556acb9f3300 msgr2=0x556acbda7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 shutdown_connections 2024-09-21T02:13:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 1 -- 172.21.15.92:0/2958105403 wait complete. 2024-09-21T02:13:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.082+0000 7fd6cbc14740 20 client.18048 trim_cache size 0 max 16384 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 Processor -- start 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 -- start start 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbda6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9ff1b0 0x556acbda7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01ab060 con 0x556acb9d62f0 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01acba0 con 0x556acb9de4d0 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.096+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01ab730 con 0x556acb9ff1b0 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56910/0 (socket says 172.21.15.92:56910) 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/1212534537 learned_addr learned my addr 172.21.15.92:0/1212534537 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9ff1b0 0x556acbda7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6be59c640 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbda6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/1212534537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9ff1b0 msgr2=0x556acbda7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9ff1b0 0x556acbda7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/1212534537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 msgr2=0x556acbda6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbda6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 -- 172.21.15.92:0/1212534537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556acbdae730 con 0x556acb9d62f0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6be59c640 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbda6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda7c90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400b6b0 tx=0x7fd6b400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669cebe576145257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6a7fff640 1 -- 172.21.15.92:0/1212534537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40047a0 con 0x556acb9d62f0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6a7fff640 1 -- 172.21.15.92:0/1212534537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b40044f0 con 0x556acb9d62f0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.097+0000 7fd6a7fff640 1 -- 172.21.15.92:0/1212534537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4003850 con 0x556acb9d62f0 2024-09-21T02:13:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1212534537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 msgr2=0x556acbda7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda7c90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400b6b0 tx=0x7fd6b400b8a0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1212534537 shutdown_connections 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9ff1b0 0x556acbda7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9de4d0 0x556acbda6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1212534537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9d62f0 0x556acbda7c90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1212534537 >> 172.21.15.92:0/1212534537 conn(0x556acb9f3300 msgr2=0x556acbd9d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1212534537 shutdown_connections 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1212534537 wait complete. 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 -- start start 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.098+0000 7fd6cbc14740 1 Processor -- start 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 0x556acbdb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbdb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01ac4d0 con 0x556acb9de4d0 2024-09-21T02:13:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01abe00 con 0x556acb9ff1b0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01a9040 con 0x556acb9d62f0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 10 client.0 _ll_register_callbacks cb 0x7fd6b40129f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 0x556acbdb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 0x556acbdb5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56914/0 (socket says 172.21.15.92:56914) 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 -- 172.21.15.92:0/1984309108 learned_addr learned my addr 172.21.15.92:0/1984309108 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbdb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 -- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 msgr2=0x556acbdb57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 -- 172.21.15.92:0/1984309108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 msgr2=0x556acbdb6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbdb6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 -- 172.21.15.92:0/1984309108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556acbdb7180 con 0x556acb9de4d0 2024-09-21T02:13:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6be59c640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 0x556acbdb5eb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005f70 tx=0x7fd6b002abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39582ebeb3a6836 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/1984309108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0035040 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/1984309108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd6b0004450 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.099+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b40044f0 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/1984309108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6b00035f0 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 10 client.18054 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556acbdb8450 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/1984309108 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b40044f0 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/1984309108 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6b004e040 con 0x556acb9de4d0 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 10 client.18054 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 20 client.18054 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9931c0 0x556acbe2a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 4 client.18054 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:13:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556acbd170f0 con 0x556acb9931c0 2024-09-21T02:13:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.100+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9931c0 0x556acbe2a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.101+0000 7fd6a5ffb640 10 client.18054 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.101+0000 7fd6bdd9b640 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9931c0 0x556acbe2a6f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4db8011da1092e server_cookie=6218518eccc8c2b in_seq=0 out_seq=0 2024-09-21T02:13:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6a5ffb640 1 -- 172.21.15.92:0/1984309108 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556acbd170f0 con 0x556acb9931c0 2024-09-21T02:13:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6a5ffb640 10 client.18054 handle_command_reply: tid=0 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 client.18054 shutdown 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 msgr2=0x556acbdb5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 0x556acbdb5eb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005f70 tx=0x7fd6b002abc0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 shutdown_connections 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556acb9931c0 0x556acbe2a6f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556acb9ff1b0 0x556acbdb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556acb9de4d0 0x556acbdb5eb0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6a5ffb640 0 client.18054 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 --2- 172.21.15.92:0/1984309108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556acb9d62f0 0x556acbdb57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 >> 172.21.15.92:0/1984309108 conn(0x556acb9f3300 msgr2=0x556acbda6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 shutdown_connections 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.114+0000 7fd6cbc14740 1 -- 172.21.15.92:0/1984309108 wait complete. 2024-09-21T02:13:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6cbc14740 20 client.18054 trim_cache size 0 max 16384 2024-09-21T02:13:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:13:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:13:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:13:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6a80b6510 msgr2=0x7fd6a80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6a80b6510 0x7fd6a80b8a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80e4d40 tx=0x7fd6b8078c60 comp rx=0 tx=0).stop 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 msgr2=0x7fd6c019b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 0x7fd6c019b180 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8007ee0 tx=0x7fd6b800d4a0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 shutdown_connections 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6a80b6510 0x7fd6a80b8a20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6c0108300 0x7fd6c019b180 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6c01056e0 0x7fd6c019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 --2- 172.21.15.92:0/2666896377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c0102c80 0x7fd6c0199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 >> 172.21.15.92:0/2666896377 conn(0x7fd6c00fd690 msgr2=0x7fd6c0103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 shutdown_connections 2024-09-21T02:13:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:41.115+0000 7fd6c4bcc640 1 -- 172.21.15.92:0/2666896377 wait complete. 2024-09-21T02:13:41.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:40 smithi136 ceph-mon[34754]: pgmap v621: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:13:41.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:40 smithi092 ceph-mon[31190]: pgmap v621: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:13:41.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:40 smithi099 ceph-mon[34404]: pgmap v621: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 22 op/s 2024-09-21T02:13:43.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:42 smithi136 ceph-mon[34754]: pgmap v622: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:13:43.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:42 smithi092 ceph-mon[31190]: pgmap v622: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:13:43.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:42 smithi099 ceph-mon[34404]: pgmap v622: 97 pgs: 97 active+clean; 2.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:13:44.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:43 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:13:44.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:43 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:13:44.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:43 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:13:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:44 smithi136 ceph-mon[34754]: pgmap v623: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:13:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:44 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:13:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:44 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:13:45.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:44 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:13:45.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:44 smithi092 ceph-mon[31190]: pgmap v623: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:13:45.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:44 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:13:45.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:44 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:13:45.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:44 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:13:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:44 smithi099 ceph-mon[34404]: pgmap v623: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:13:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:44 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:13:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:44 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:13:45.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:44 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:13:47.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:46 smithi136 ceph-mon[34754]: pgmap v624: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:13:47.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:46 smithi092 ceph-mon[31190]: pgmap v624: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:13:47.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:46 smithi099 ceph-mon[34404]: pgmap v624: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:13:49.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:48 smithi136 ceph-mon[34754]: pgmap v625: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:49.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:48 smithi092 ceph-mon[31190]: pgmap v625: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:49.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:48 smithi099 ceph-mon[34404]: pgmap v625: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:51.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:50 smithi136 ceph-mon[34754]: pgmap v626: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:51.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:50 smithi092 ceph-mon[31190]: pgmap v626: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:51.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:50 smithi099 ceph-mon[34404]: pgmap v626: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:13:53.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:52 smithi136 ceph-mon[34754]: pgmap v627: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:53.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:52 smithi092 ceph-mon[31190]: pgmap v627: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:53.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:52 smithi099 ceph-mon[34404]: pgmap v627: 97 pgs: 97 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:437/600: open fd = 3 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:438/600: open fd = 3 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:439/600: open fd = 3 2024-09-21T02:13:53.699 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:440/600: open fd = 3 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:441/600: open fd = 3 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:442/600: open fd = 3 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:443/600: open fd = 3 2024-09-21T02:13:53.700 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:444/600: open fd = 3 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:445/600: open fd = 3 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:446/600: open fd = 3 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.701 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:447/600: open fd = 3 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:448/600: open fd = 3 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:449/600: open fd = 3 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:450/600: open fd = 3 2024-09-21T02:13:53.702 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:451/600: open fd = 3 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:452/600: open fd = 3 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:453/600: open fd = 3 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.703 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:454/600: open fd = 3 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:455/600: open fd = 3 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:456/600: open fd = 3 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:457/600: open fd = 3 2024-09-21T02:13:53.704 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:458/600: open fd = 3 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:459/600: open fd = 3 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:460/600: open fd = 3 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.705 INFO:tasks.workunit.client.0.smithi092.stdout:461/600: open fd = 3 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:462/600: open fd = 3 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:463/600: open fd = 3 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.706 INFO:tasks.workunit.client.0.smithi092.stdout:464/600: open fd = 3 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:465/600: open fd = 3 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:466/600: open fd = 3 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:467/600: open fd = 3 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.707 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:468/600: open fd = 3 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:469/600: open fd = 3 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:470/600: open fd = 3 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.708 INFO:tasks.workunit.client.0.smithi092.stdout:471/600: open fd = 3 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:472/600: open fd = 3 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:473/600: open fd = 3 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:474/600: open fd = 3 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.709 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:475/600: open fd = 3 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:476/600: open fd = 3 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:477/600: open fd = 3 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:478/600: open fd = 3 2024-09-21T02:13:53.710 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:479/600: open fd = 3 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:480/600: open fd = 3 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:481/600: open fd = 3 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.711 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:482/600: open fd = 3 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:483/600: open fd = 3 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:484/600: open fd = 3 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.712 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:485/600: open fd = 3 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:486/600: open fd = 3 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:487/600: open fd = 3 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:488/600: open fd = 3 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.713 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:489/600: open fd = 3 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:490/600: open fd = 3 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:491/600: open fd = 3 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:492/600: open fd = 3 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.714 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:493/600: open fd = 3 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:494/600: open fd = 3 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:495/600: open fd = 3 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:496/600: open fd = 3 2024-09-21T02:13:53.715 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:497/600: open fd = 3 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:498/600: open fd = 3 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:13:53.716 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:13:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/228201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 msgr2=0x7f4f5410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/228201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f5410a980 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ca0 tx=0x7f4f4c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/228201504 shutdown_connections 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/228201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f5410a980 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/228201504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f54103640 0x7f4f54103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/228201504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f541025c0 0x7f4f54102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/228201504 >> 172.21.15.92:0/228201504 conn(0x7f4f540fdf00 msgr2=0x7f4f541003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/228201504 shutdown_connections 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.227+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/228201504 wait complete. 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 Processor -- start 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 -- start start 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f541025c0 0x7f4f54106190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103640 0x7f4f54104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f54104f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410e210 con 0x7f4f54103fe0 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410e090 con 0x7f4f54103640 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f5ad2b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410e390 con 0x7f4f541025c0 2024-09-21T02:13:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f54104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f54104f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38424/0 (socket says 172.21.15.92:38424) 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 -- 172.21.15.92:0/857399914 learned_addr learned my addr 172.21.15.92:0/857399914 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f53fff640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103640 0x7f4f54104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f541025c0 0x7f4f54106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 -- 172.21.15.92:0/857399914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f541025c0 msgr2=0x7f4f54106190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f541025c0 0x7f4f54106190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 -- 172.21.15.92:0/857399914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103640 msgr2=0x7f4f54104a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103640 0x7f4f54104a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 -- 172.21.15.92:0/857399914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f54105630 con 0x7f4f54103fe0 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.228+0000 7f4f592c8640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f54104f90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0097b0 tx=0x7f4f4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.229+0000 7f4f51ffb640 1 -- 172.21.15.92:0/857399914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c004240 con 0x7f4f54103fe0 2024-09-21T02:13:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.229+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541a3f80 con 0x7f4f54103fe0 2024-09-21T02:13:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.229+0000 7f4f51ffb640 1 -- 172.21.15.92:0/857399914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4c0317c0 con 0x7f4f54103fe0 2024-09-21T02:13:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.229+0000 7f4f51ffb640 1 -- 172.21.15.92:0/857399914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c02fc30 con 0x7f4f54103fe0 2024-09-21T02:13:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.229+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a4330 con 0x7f4f54103fe0 2024-09-21T02:13:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.231+0000 7f4f61c9a740 1 Processor -- start 2024-09-21T02:13:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f51ffb640 1 -- 172.21.15.92:0/857399914 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4f4c0305c0 con 0x7f4f54103fe0 2024-09-21T02:13:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 -- start start 2024-09-21T02:13:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 0x561cee2c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561cee2356f0 con 0x561cedf1b1b0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561cee224b20 con 0x561cedefa4d0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cee228930 con 0x561cedef22f0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 0x561cee2c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 0x561cee2c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38440/0 (socket says 172.21.15.92:38440) 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.232+0000 7f4f53fff640 1 -- 172.21.15.92:0/453945933 learned_addr learned my addr 172.21.15.92:0/453945933 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 -- 172.21.15.92:0/453945933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 msgr2=0x561cee2c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 -- 172.21.15.92:0/453945933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 msgr2=0x561cee2c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f51ffb640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f400b6510 0x7f4f400b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 -- 172.21.15.92:0/453945933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cee1ea530 con 0x561cedf1b1b0 2024-09-21T02:13:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f400b6510 0x7f4f400b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f53fff640 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 0x561cee2c2e50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0135a0 tx=0x7f4f3c00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97500095ad6f541d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f51ffb640 1 -- 172.21.15.92:0/857399914 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4f4c108900 con 0x7f4f54103fe0 2024-09-21T02:13:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f4affd640 1 -- 172.21.15.92:0/453945933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c019030 con 0x561cedf1b1b0 2024-09-21T02:13:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f4affd640 1 -- 172.21.15.92:0/453945933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0054d0 con 0x561cedf1b1b0 2024-09-21T02:13:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.233+0000 7f4f4affd640 1 -- 172.21.15.92:0/453945933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c004e60 con 0x561cedf1b1b0 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 -- 172.21.15.92:0/453945933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 msgr2=0x561cee2c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 0x561cee2c2e50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0135a0 tx=0x7f4f3c00b6e0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f400b6510 0x7f4f400b8a20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4f440045d0 tx=0x7f4f4400c4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 -- 172.21.15.92:0/453945933 shutdown_connections 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/453945933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedf1b1b0 0x561cee2c2e50 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 -- 172.21.15.92:0/453945933 >> 172.21.15.92:0/453945933 conn(0x561cedf0f300 msgr2=0x561cee2be8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 -- 172.21.15.92:0/453945933 shutdown_connections 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.234+0000 7f4f61c9a740 1 -- 172.21.15.92:0/453945933 wait complete. 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 -- start start 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 Processor -- start 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedf1b1b0 0x561cee2cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561cee15cab0 con 0x561cedefa4d0 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561cee1ca730 con 0x561cedf1b1b0 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cee2c4b30 con 0x561cedef22f0 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f61c9a740 10 client.0 _ll_register_callbacks cb 0x561cee21e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38452/0 (socket says 172.21.15.92:38452) 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.235+0000 7f4f53fff640 1 -- 172.21.15.92:0/2594127275 learned_addr learned my addr 172.21.15.92:0/2594127275 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f53fff640 1 -- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 msgr2=0x561cee2cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f53fff640 1 -- 172.21.15.92:0/2594127275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedf1b1b0 msgr2=0x561cee2cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f592c8640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedf1b1b0 0x561cee2cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedf1b1b0 0x561cee2cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f53fff640 1 -- 172.21.15.92:0/2594127275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cedf86790 con 0x561cedefa4d0 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2ccc70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0057c0 tx=0x7f4f3c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26dff1e5d5c2520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f33fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f33fff640 1 -- 172.21.15.92:0/2594127275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c02b060 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f33fff640 1 -- 172.21.15.92:0/2594127275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4f3c0046c0 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c0054d0 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f33fff640 1 -- 172.21.15.92:0/2594127275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f3c019040 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f61c9a740 10 client.18066 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cedfff620 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.236+0000 7f4f33fff640 1 -- 172.21.15.92:0/2594127275 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c0054d0 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f33fff640 1 -- 172.21.15.92:0/2594127275 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f3c011ca0 con 0x561cedefa4d0 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f61c9a740 10 client.18066 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f61c9a740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f61c9a740 20 client.18066 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561ceded9d80 0x561cee2d99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f61c9a740 4 client.18066 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cee237640 con 0x561ceded9d80 2024-09-21T02:13:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.237+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561ceded9d80 0x561cee2d99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.238+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561ceded9d80 0x561cee2d99d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c4acba593fd404 server_cookie=ee942f784b242931 in_seq=0 out_seq=0 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.238+0000 7f4f33fff640 10 client.18066 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.238+0000 7f4f33fff640 1 -- 172.21.15.92:0/2594127275 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561cee237640 con 0x561ceded9d80 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.238+0000 7f4f33fff640 10 client.18066 handle_command_reply: tid=0 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.238+0000 7f4f61c9a740 1 client.18066 shutdown 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.238+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 msgr2=0x561cee2ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2ccc70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0057c0 tx=0x7f4f3c004e00 comp rx=0 tx=0).stop 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 shutdown_connections 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561ceded9d80 0x561cee2d99d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedf1b1b0 0x561cee2cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2ccc70 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2594127275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedef22f0 0x561cee2cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f33fff640 0 client.18066 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 >> 172.21.15.92:0/2594127275 conn(0x561cedf0f300 msgr2=0x561cee2bcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 shutdown_connections 2024-09-21T02:13:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2594127275 wait complete. 2024-09-21T02:13:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.239+0000 7f4f61c9a740 20 client.18066 trim_cache size 0 max 16384 2024-09-21T02:13:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.253+0000 7f4f61c9a740 1 Processor -- start 2024-09-21T02:13:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.253+0000 7f4f61c9a740 1 -- start start 2024-09-21T02:13:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedef22f0 0x561cee2c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ad160 con 0x561cedefa4d0 2024-09-21T02:13:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541aeca0 con 0x561cedef22f0 2024-09-21T02:13:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ad830 con 0x561cedf1b1b0 2024-09-21T02:13:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38460/0 (socket says 172.21.15.92:38460) 2024-09-21T02:13:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 -- 172.21.15.92:0/1290004526 learned_addr learned my addr 172.21.15.92:0/1290004526 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f592c8640 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 -- 172.21.15.92:0/1290004526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 msgr2=0x561cee2cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 -- 172.21.15.92:0/1290004526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedef22f0 msgr2=0x561cee2c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedef22f0 0x561cee2c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 -- 172.21.15.92:0/1290004526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cee2c4a10 con 0x561cedefa4d0 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f592c8640 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f53fff640 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2b95c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c01e4e0 tx=0x7f4f3c0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12107711a5dbc8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f4a7fc640 1 -- 172.21.15.92:0/1290004526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c02b070 con 0x561cedefa4d0 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f4a7fc640 1 -- 172.21.15.92:0/1290004526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c01e6d0 con 0x561cedefa4d0 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.254+0000 7f4f4a7fc640 1 -- 172.21.15.92:0/1290004526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c005520 con 0x561cedefa4d0 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 -- 172.21.15.92:0/1290004526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 msgr2=0x561cee2b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2b95c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c01e4e0 tx=0x7f4f3c0059b0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 -- 172.21.15.92:0/1290004526 shutdown_connections 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedefa4d0 0x561cee2b95c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/1290004526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedef22f0 0x561cee2c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 -- 172.21.15.92:0/1290004526 >> 172.21.15.92:0/1290004526 conn(0x561cedf0f300 msgr2=0x561cee2cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 -- 172.21.15.92:0/1290004526 shutdown_connections 2024-09-21T02:13:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 -- 172.21.15.92:0/1290004526 wait complete. 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.255+0000 7f4f61c9a740 1 -- start start 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 Processor -- start 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 0x561cee2d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ae5d0 con 0x561cedef22f0 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541adf00 con 0x561cedefa4d0 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ab140 con 0x561cedf1b1b0 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f61c9a740 10 client.0 _ll_register_callbacks cb 0x7f4f4400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f58ac7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 0x561cee2d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f58ac7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 0x561cee2d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38468/0 (socket says 172.21.15.92:38468) 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f58ac7640 1 -- 172.21.15.92:0/2126160157 learned_addr learned my addr 172.21.15.92:0/2126160157 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:13:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f592c8640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.256+0000 7f4f58ac7640 1 -- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 msgr2=0x561cee2d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f58ac7640 1 -- 172.21.15.92:0/2126160157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 msgr2=0x561cee2d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f58ac7640 1 -- 172.21.15.92:0/2126160157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cee2d2a60 con 0x561cedef22f0 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f58ac7640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 0x561cee2d10d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x561cee2da0b0 tx=0x7f4f44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb8b475dbc06280 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f33fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f33fff640 1 -- 172.21.15.92:0/2126160157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4401b450 con 0x561cedef22f0 2024-09-21T02:13:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f33fff640 1 -- 172.21.15.92:0/2126160157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4f4401b5f0 con 0x561cedef22f0 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c01e6d0 con 0x561cedef22f0 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f33fff640 1 -- 172.21.15.92:0/2126160157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f4401b7f0 con 0x561cedef22f0 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 10 client.18072 fetch_fsmap learned FSMap version 29 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cee2d3dc0 con 0x561cedef22f0 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f33fff640 1 -- 172.21.15.92:0/2126160157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c01e6d0 con 0x561cedef22f0 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f33fff640 1 -- 172.21.15.92:0/2126160157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f44018040 con 0x561cedef22f0 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 10 client.18072 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:13:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 20 client.18072 populate_metadata read hostname 'smithi092' 2024-09-21T02:13:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561cedeaf1c0 0x561cee346480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 4 client.18072 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:13:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.257+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cee2330f0 con 0x561cedeaf1c0 2024-09-21T02:13:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.258+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561cedeaf1c0 0x561cee346480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.258+0000 7f4f33fff640 10 client.18072 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.258+0000 7f4f53fff640 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561cedeaf1c0 0x561cee346480 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907be02aa786eb88 server_cookie=b2dc676ffca6fb08 in_seq=0 out_seq=0 2024-09-21T02:13:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f33fff640 1 -- 172.21.15.92:0/2126160157 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cee2330f0 con 0x561cedeaf1c0 2024-09-21T02:13:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f33fff640 10 client.18072 handle_command_reply: tid=0 2024-09-21T02:13:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 client.18072 shutdown 2024-09-21T02:13:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 msgr2=0x561cee2d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 0x561cee2d10d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x561cee2da0b0 tx=0x7f4f44004730 comp rx=0 tx=0).stop 2024-09-21T02:13:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 shutdown_connections 2024-09-21T02:13:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x561cedeaf1c0 0x561cee346480 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cedf1b1b0 0x561cee2d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cedefa4d0 0x561cee2d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f33fff640 0 client.18072 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 --2- 172.21.15.92:0/2126160157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cedef22f0 0x561cee2d10d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 >> 172.21.15.92:0/2126160157 conn(0x561cedf0f300 msgr2=0x561cee2b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 shutdown_connections 2024-09-21T02:13:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 1 -- 172.21.15.92:0/2126160157 wait complete. 2024-09-21T02:13:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.274+0000 7f4f61c9a740 20 client.18072 trim_cache size 0 max 16384 2024-09-21T02:13:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:13:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:13:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:13:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:13:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f400b6510 msgr2=0x7f4f400b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f400b6510 0x7f4f400b8a20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4f440045d0 tx=0x7f4f4400c4b0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 msgr2=0x7f4f54104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f54104f90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0097b0 tx=0x7f4f4c003c20 comp rx=0 tx=0).stop 2024-09-21T02:13:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 shutdown_connections 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4f400b6510 0x7f4f400b8a20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f54103fe0 0x7f4f54104f90 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103640 0x7f4f54104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 --2- 172.21.15.92:0/857399914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f541025c0 0x7f4f54106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 >> 172.21.15.92:0/857399914 conn(0x7f4f540fdf00 msgr2=0x7f4f540ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 shutdown_connections 2024-09-21T02:13:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:13:54.275+0000 7f4f5ad2b640 1 -- 172.21.15.92:0/857399914 wait complete. 2024-09-21T02:13:55.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:54 smithi136 ceph-mon[34754]: pgmap v628: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:13:55.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:54 smithi092 ceph-mon[31190]: pgmap v628: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:13:55.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:54 smithi099 ceph-mon[34404]: pgmap v628: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:13:57.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:56 smithi136 ceph-mon[34754]: pgmap v629: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:13:57.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:56 smithi092 ceph-mon[31190]: pgmap v629: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:13:57.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:56 smithi099 ceph-mon[34404]: pgmap v629: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:13:59.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:13:58 smithi136 ceph-mon[34754]: pgmap v630: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:13:59.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:13:58 smithi092 ceph-mon[31190]: pgmap v630: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:13:59.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:13:58 smithi099 ceph-mon[34404]: pgmap v630: 97 pgs: 97 active+clean; 2.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:14:01.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:01 smithi136 ceph-mon[34754]: pgmap v631: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:14:01.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:01 smithi092 ceph-mon[31190]: pgmap v631: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:14:01.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:01 smithi099 ceph-mon[34404]: pgmap v631: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:14:02.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:14:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 -- 172.21.15.92:0/2158990841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa701083a0 msgr2=0x7ffa701087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 --2- 172.21.15.92:0/2158990841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa701083a0 0x7ffa701087b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009900 tx=0x7ffa6002a410 comp rx=0 tx=0).stop 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 -- 172.21.15.92:0/2158990841 shutdown_connections 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 --2- 172.21.15.92:0/2158990841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa70108cf0 0x7ffa7010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 --2- 172.21.15.92:0/2158990841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa701083a0 0x7ffa701087b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 --2- 172.21.15.92:0/2158990841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa70102a60 0x7ffa70102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 -- 172.21.15.92:0/2158990841 >> 172.21.15.92:0/2158990841 conn(0x7ffa70079730 msgr2=0x7ffa70079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 -- 172.21.15.92:0/2158990841 shutdown_connections 2024-09-21T02:14:03.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 -- 172.21.15.92:0/2158990841 wait complete. 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.091+0000 7ffa75e92640 1 Processor -- start 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 -- start start 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa70102a60 0x7ffa7019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa701083a0 0x7ffa7019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 0x7ffa7019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70112960 con 0x7ffa70108cf0 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701127e0 con 0x7ffa701083a0 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa75e92640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70112ae0 con 0x7ffa70102a60 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa6ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 0x7ffa7019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa701083a0 0x7ffa7019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa6ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 0x7ffa7019bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57398/0 (socket says 172.21.15.92:57398) 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa6ffff640 1 -- 172.21.15.92:0/3786260184 learned_addr learned my addr 172.21.15.92:0/3786260184 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa6effd640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa701083a0 0x7ffa7019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:53322/0 (socket says 172.21.15.92:53322) 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.092+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa70102a60 0x7ffa7019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6ffff640 1 -- 172.21.15.92:0/3786260184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa70102a60 msgr2=0x7ffa7019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6ffff640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa70102a60 0x7ffa7019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6ffff640 1 -- 172.21.15.92:0/3786260184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa701083a0 msgr2=0x7ffa7019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6ffff640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa701083a0 0x7ffa7019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6ffff640 1 -- 172.21.15.92:0/3786260184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa701a6600 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6ffff640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 0x7ffa7019bd70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800cad0 tx=0x7ffa6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6cff9640 1 -- 172.21.15.92:0/3786260184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800ae20 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa701a68f0 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6cff9640 1 -- 172.21.15.92:0/3786260184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa68005020 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa6cff9640 1 -- 172.21.15.92:0/3786260184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800f6b0 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.093+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a6db0 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 Processor -- start 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa6cff9640 1 -- 172.21.15.92:0/3786260184 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ffa680051d0 con 0x7ffa70108cf0 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 -- start start 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49d4070 0x55dff4d7b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d7bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d7c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dff4cf1e50 con 0x55dff49b3390 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dff4ce1280 con 0x55dff49ab1b0 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa7c406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dff4ce5090 con 0x55dff49d4070 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d7c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d7c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57410/0 (socket says 172.21.15.92:57410) 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.096+0000 7ffa6effd640 1 -- 172.21.15.92:0/1750631759 learned_addr learned my addr 172.21.15.92:0/1750631759 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6cff9640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ffa580b6510 0x7ffa580b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d7bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49d4070 0x55dff4d7b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6ffff640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ffa580b6510 0x7ffa580b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 -- 172.21.15.92:0/1750631759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49d4070 msgr2=0x55dff4d7b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49d4070 0x55dff4d7b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 -- 172.21.15.92:0/1750631759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 msgr2=0x55dff4d7bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d7bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6cff9640 1 -- 172.21.15.92:0/3786260184 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ffa680e7230 con 0x7ffa70108cf0 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 -- 172.21.15.92:0/1750631759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff4a3f650 con 0x55dff49b3390 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6effd640 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d7c0d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009900 tx=0x7ffa60009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd1d37ff1e508a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa6ffff640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ffa580b6510 0x7ffa580b8a20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffa680ad150 tx=0x7ffa68079050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa55ffb640 1 -- 172.21.15.92:0/1750631759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60035070 con 0x55dff49b3390 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa55ffb640 1 -- 172.21.15.92:0/1750631759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa60031900 con 0x55dff49b3390 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.097+0000 7ffa55ffb640 1 -- 172.21.15.92:0/1750631759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa600055f0 con 0x55dff49b3390 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 -- 172.21.15.92:0/1750631759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 msgr2=0x55dff4d7c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d7c0d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009900 tx=0x7ffa60009560 comp rx=0 tx=0).stop 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 -- 172.21.15.92:0/1750631759 shutdown_connections 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d7c0d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d7bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 --2- 172.21.15.92:0/1750631759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49d4070 0x55dff4d7b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 -- 172.21.15.92:0/1750631759 >> 172.21.15.92:0/1750631759 conn(0x55dff49c8460 msgr2=0x55dff4d79540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 -- 172.21.15.92:0/1750631759 shutdown_connections 2024-09-21T02:14:03.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.098+0000 7ffa7c406740 1 -- 172.21.15.92:0/1750631759 wait complete. 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.099+0000 7ffa7c406740 1 -- start start 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.099+0000 7ffa7c406740 1 Processor -- start 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.099+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d86d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.099+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49b3390 0x55dff4d872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 0x55dff4d93060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa7c406740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dff4b42700 con 0x55dff49d4070 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa7c406740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dff4c7c110 con 0x55dff49ab1b0 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa7c406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dff4d89d70 con 0x55dff49b3390 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa7c406740 10 client.0 _ll_register_callbacks cb 0x55dff4c95990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 0x55dff4d93060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 0x55dff4d93060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57414/0 (socket says 172.21.15.92:57414) 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 -- 172.21.15.92:0/2379405022 learned_addr learned my addr 172.21.15.92:0/2379405022 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49b3390 0x55dff4d872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d86d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 -- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49b3390 msgr2=0x55dff4d872d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49b3390 0x55dff4d872d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 -- 172.21.15.92:0/2379405022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 msgr2=0x55dff4d86d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d86d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 -- 172.21.15.92:0/2379405022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff4ca9c40 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa6effd640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 0x55dff4d93060 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ffa600052d0 tx=0x7ffa60031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ab8937a8ce0f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa4effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa4effd640 1 -- 172.21.15.92:0/2379405022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60035030 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa4effd640 1 -- 172.21.15.92:0/2379405022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ffa60004050 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.100+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dff4cf3da0 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa4effd640 1 -- 172.21.15.92:0/2379405022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa600315f0 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 10 client.18084 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dff4d95b00 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa4effd640 1 -- 172.21.15.92:0/2379405022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dff4cf3da0 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa4effd640 1 -- 172.21.15.92:0/2379405022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa600035f0 con 0x55dff49d4070 2024-09-21T02:14:03.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 10 client.18084 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 20 client.18084 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49929f0 0x55dff4d98e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 4 client.18084 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff4cef850 con 0x55dff49929f0 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.101+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49929f0 0x55dff4d98e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.102+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49929f0 0x55dff4d98e60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb62daefc09ee6c7 server_cookie=ad67978a22802af3 in_seq=0 out_seq=0 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.102+0000 7ffa4effd640 10 client.18084 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa4effd640 1 -- 172.21.15.92:0/2379405022 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dff4cef850 con 0x55dff49929f0 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa4effd640 10 client.18084 handle_command_reply: tid=0 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 client.18084 shutdown 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 msgr2=0x55dff4d93060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 0x55dff4d93060 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ffa600052d0 tx=0x7ffa60031400 comp rx=0 tx=0).stop 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 shutdown_connections 2024-09-21T02:14:03.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49929f0 0x55dff4d98e60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa4effd640 0 client.18084 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49d4070 0x55dff4d93060 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49b3390 0x55dff4d872d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 --2- 172.21.15.92:0/2379405022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d86d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 >> 172.21.15.92:0/2379405022 conn(0x55dff49c8460 msgr2=0x55dff4d7e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 shutdown_connections 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 1 -- 172.21.15.92:0/2379405022 wait complete. 2024-09-21T02:14:03.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.103+0000 7ffa7c406740 20 client.18084 trim_cache size 0 max 16384 2024-09-21T02:14:03.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.120+0000 7ffa7c406740 1 Processor -- start 2024-09-21T02:14:03.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.120+0000 7ffa7c406740 1 -- start start 2024-09-21T02:14:03.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 0x55dff4d88d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49ab1b0 0x55dff4d78b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49b3390 0x55dff4d790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa7c406740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701af4d0 con 0x55dff49929f0 2024-09-21T02:14:03.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa7c406740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b1010 con 0x55dff49b3390 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa7c406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701afba0 con 0x55dff49ab1b0 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 0x55dff4d88d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 0x55dff4d88d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57416/0 (socket says 172.21.15.92:57416) 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 -- 172.21.15.92:0/4180528523 learned_addr learned my addr 172.21.15.92:0/4180528523 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49ab1b0 0x55dff4d78b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49b3390 0x55dff4d790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 -- 172.21.15.92:0/4180528523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49ab1b0 msgr2=0x55dff4d78b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49ab1b0 0x55dff4d78b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 -- 172.21.15.92:0/4180528523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49b3390 msgr2=0x55dff4d790b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49b3390 0x55dff4d790b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 -- 172.21.15.92:0/4180528523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff4d894d0 con 0x55dff49929f0 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.121+0000 7ffa6effd640 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 0x55dff4d88d80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ffa60003a90 tx=0x7ffa60009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89c4d6d18d0972f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.122+0000 7ffa557fa640 1 -- 172.21.15.92:0/4180528523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6005e070 con 0x55dff49929f0 2024-09-21T02:14:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.122+0000 7ffa557fa640 1 -- 172.21.15.92:0/4180528523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6005f040 con 0x55dff49929f0 2024-09-21T02:14:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.122+0000 7ffa557fa640 1 -- 172.21.15.92:0/4180528523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60035040 con 0x55dff49929f0 2024-09-21T02:14:03.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 -- 172.21.15.92:0/4180528523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 msgr2=0x55dff4d88d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 0x55dff4d88d80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ffa60003a90 tx=0x7ffa60009560 comp rx=0 tx=0).stop 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 -- 172.21.15.92:0/4180528523 shutdown_connections 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49b3390 0x55dff4d790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49ab1b0 0x55dff4d78b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 --2- 172.21.15.92:0/4180528523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49929f0 0x55dff4d88d80 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 -- 172.21.15.92:0/4180528523 >> 172.21.15.92:0/4180528523 conn(0x55dff49c8460 msgr2=0x55dff4d7d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 -- 172.21.15.92:0/4180528523 shutdown_connections 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.123+0000 7ffa7c406740 1 -- 172.21.15.92:0/4180528523 wait complete. 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 -- start start 2024-09-21T02:14:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 Processor -- start 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49929f0 0x55dff4d9c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d99ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d9a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b0940 con 0x55dff49b3390 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b0270 con 0x55dff49ab1b0 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701ad500 con 0x55dff49929f0 2024-09-21T02:14:03.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.124+0000 7ffa7c406740 10 client.0 _ll_register_callbacks cb 0x7ffa6400a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d9a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d9a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57422/0 (socket says 172.21.15.92:57422) 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6effd640 1 -- 172.21.15.92:0/596365058 learned_addr learned my addr 172.21.15.92:0/596365058 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d99ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6effd640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49929f0 0x55dff4d9c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 -- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49929f0 msgr2=0x55dff4d9c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49929f0 0x55dff4d9c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 -- 172.21.15.92:0/596365058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 msgr2=0x55dff4d9a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d9a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 -- 172.21.15.92:0/596365058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff4d9a760 con 0x55dff49ab1b0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6effd640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d9a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa6f7fe640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d99ae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55dff4d89da0 tx=0x7ffa64010b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce6e7f6ab727b885 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa4effd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:14:03.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa4effd640 1 -- 172.21.15.92:0/596365058 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64028020 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa4effd640 1 -- 172.21.15.92:0/596365058 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ffa6400fd60 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.125+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6005f040 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa4effd640 1 -- 172.21.15.92:0/596365058 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa64033050 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 10 client.26903 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dff4e03520 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa4effd640 1 -- 172.21.15.92:0/596365058 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6005f040 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa4effd640 1 -- 172.21.15.92:0/596365058 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa6401e400 con 0x55dff49ab1b0 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 10 client.26903 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 10 client.26903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 20 client.26903 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49d4070 0x55dff4e05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 4 client.26903 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dff4cecc80 con 0x55dff49d4070 2024-09-21T02:14:03.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.126+0000 7ffa6effd640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49d4070 0x55dff4e05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa6effd640 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49d4070 0x55dff4e05ce0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457abf936584e75b server_cookie=b02b65482161e0b5 in_seq=0 out_seq=0 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa4effd640 10 client.26903 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa4effd640 1 -- 172.21.15.92:0/596365058 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55dff4cecc80 con 0x55dff49d4070 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa4effd640 10 client.26903 handle_command_reply: tid=0 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 client.26903 shutdown 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 msgr2=0x55dff4d99ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d99ae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55dff4d89da0 tx=0x7ffa64010b20 comp rx=0 tx=0).stop 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 shutdown_connections 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dff49d4070 0x55dff4e05ce0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dff49b3390 0x55dff4d9a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dff49ab1b0 0x55dff4d99ae0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 --2- 172.21.15.92:0/596365058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dff49929f0 0x55dff4d9c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 >> 172.21.15.92:0/596365058 conn(0x55dff49c8460 msgr2=0x55dff4d7cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.127+0000 7ffa4effd640 0 client.26903 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 shutdown_connections 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa7c406740 1 -- 172.21.15.92:0/596365058 wait complete. 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa7c406740 20 client.26903 trim_cache size 0 max 16384 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:14:03.132 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.133 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.134 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:13:30.072031+0000", 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.135 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.137 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.138 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:03.139 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ffa580b6510 msgr2=0x7ffa580b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ffa580b6510 0x7ffa580b8a20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffa680ad150 tx=0x7ffa68079050 comp rx=0 tx=0).stop 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 msgr2=0x7ffa7019bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 0x7ffa7019bd70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800cad0 tx=0x7ffa6800cf90 comp rx=0 tx=0).stop 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 shutdown_connections 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ffa580b6510 0x7ffa580b8a20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa70108cf0 0x7ffa7019bd70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa701083a0 0x7ffa7019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 --2- 172.21.15.92:0/3786260184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa70102a60 0x7ffa7019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.128+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 >> 172.21.15.92:0/3786260184 conn(0x7ffa70079730 msgr2=0x7ffa70104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.129+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 shutdown_connections 2024-09-21T02:14:03.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:03.129+0000 7ffa75e92640 1 -- 172.21.15.92:0/3786260184 wait complete. 2024-09-21T02:14:03.271 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:03 smithi092 ceph-mon[31190]: pgmap v632: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:03.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:03 smithi136 ceph-mon[34754]: pgmap v632: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:03.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:03 smithi099 ceph-mon[34404]: pgmap v632: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:03.927 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:14:03.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:14:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 -- 172.21.15.92:0/2225709133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 msgr2=0x7f250c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 --2- 172.21.15.92:0/2225709133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c110620 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2504009ca0 tx=0x7f250402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 -- 172.21.15.92:0/2225709133 shutdown_connections 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 --2- 172.21.15.92:0/2225709133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c110620 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 --2- 172.21.15.92:0/2225709133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f250c1094a0 0x7f250c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 --2- 172.21.15.92:0/2225709133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 0x7f250c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 -- 172.21.15.92:0/2225709133 >> 172.21.15.92:0/2225709133 conn(0x7f250c07b850 msgr2=0x7f250c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 -- 172.21.15.92:0/2225709133 shutdown_connections 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.658+0000 7f2512c31640 1 -- 172.21.15.92:0/2225709133 wait complete. 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 Processor -- start 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 -- start start 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 0x7f250c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f250c1094a0 0x7f250c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f250c113ab0 con 0x7f250c109df0 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f250c113930 con 0x7f250c1094a0 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2512c31640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f250c113c30 con 0x7f250c103b60 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57428/0 (socket says 172.21.15.92:57428) 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 -- 172.21.15.92:0/2236995965 learned_addr learned my addr 172.21.15.92:0/2236995965 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f2503fff640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f250c1094a0 0x7f250c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25109cd640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 0x7f250c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 -- 172.21.15.92:0/2236995965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 msgr2=0x7f250c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 0x7f250c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 -- 172.21.15.92:0/2236995965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f250c1094a0 msgr2=0x7f250c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f250c1094a0 0x7f250c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25111ce640 1 -- 172.21.15.92:0/2236995965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c1a03b0 con 0x7f250c109df0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.659+0000 7f25109cd640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 0x7f250c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.660+0000 7f25111ce640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c19fc80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2504005cc0 tx=0x7f250402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.660+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504003bf0 con 0x7f250c109df0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.660+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504003d90 con 0x7f250c109df0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.660+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c1a0640 con 0x7f250c109df0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.660+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250402fbc0 con 0x7f250c109df0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.660+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1a6e00 con 0x7f250c109df0 2024-09-21T02:14:04.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.661+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c103f70 con 0x7f250c109df0 2024-09-21T02:14:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.663+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f250402fd60 con 0x7f250c109df0 2024-09-21T02:14:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.664+0000 7f2501ffb640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24f40b6510 0x7f24f40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.664+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2504108a40 con 0x7f250c109df0 2024-09-21T02:14:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.664+0000 7f25109cd640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24f40b6510 0x7f24f40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.665+0000 7f25109cd640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24f40b6510 0x7f24f40b8a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2508005d00 tx=0x7f2508005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.667+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25040cf2b0 con 0x7f250c109df0 2024-09-21T02:14:04.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.880+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f250c10bbe0 con 0x7f250c109df0 2024-09-21T02:14:04.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.881+0000 7f2501ffb640 1 -- 172.21.15.92:0/2236995965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f25040d3690 con 0x7f250c109df0 2024-09-21T02:14:04.882 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:14:04.882 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:14:04.882 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:14:04.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.883+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24f40b6510 msgr2=0x7f24f40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.883+0000 7f2512c31640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24f40b6510 0x7f24f40b8a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2508005d00 tx=0x7f2508005ed0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.883+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 msgr2=0x7f250c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.883+0000 7f2512c31640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c19fc80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2504005cc0 tx=0x7f250402f380 comp rx=0 tx=0).stop 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 shutdown_connections 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f24f40b6510 0x7f24f40b8a20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f250c109df0 0x7f250c19fc80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f250c1094a0 0x7f250c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 --2- 172.21.15.92:0/2236995965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f250c103b60 0x7f250c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 >> 172.21.15.92:0/2236995965 conn(0x7f250c07b850 msgr2=0x7f250c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 shutdown_connections 2024-09-21T02:14:04.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:04.884+0000 7f2512c31640 1 -- 172.21.15.92:0/2236995965 wait complete. 2024-09-21T02:14:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:05 smithi136 ceph-mon[34754]: pgmap v633: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:05.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:05 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2236995965' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:14:05.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:05 smithi092 ceph-mon[31190]: pgmap v633: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:05.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2236995965' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:14:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:05 smithi099 ceph-mon[34404]: pgmap v633: 97 pgs: 97 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:05.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:05 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2236995965' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:14:05.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:14:05.848 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 -- 172.21.15.92:0/2107611783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8102fd0 msgr2=0x7ff7a810fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 --2- 172.21.15.92:0/2107611783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8102fd0 0x7ff7a810fcf0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009900 tx=0x7ff79c02a480 comp rx=0 tx=0).stop 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 -- 172.21.15.92:0/2107611783 shutdown_connections 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 --2- 172.21.15.92:0/2107611783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8110230 0x7ff7a8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 --2- 172.21.15.92:0/2107611783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8102fd0 0x7ff7a810fcf0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 --2- 172.21.15.92:0/2107611783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7a8102680 0x7ff7a8102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 -- 172.21.15.92:0/2107611783 >> 172.21.15.92:0/2107611783 conn(0x7ff7a8079730 msgr2=0x7ff7a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 -- 172.21.15.92:0/2107611783 shutdown_connections 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 -- 172.21.15.92:0/2107611783 wait complete. 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 Processor -- start 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.403+0000 7ff7ae694640 1 -- start start 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7ae694640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8102680 0x7ff7a819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7ae694640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7a8102fd0 0x7ff7a819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7ae694640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 0x7ff7a81a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7ae694640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8115ca0 con 0x7ff7a8110230 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7ae694640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8115b20 con 0x7ff7a8102680 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7ae694640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8115e20 con 0x7ff7a8102fd0 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 0x7ff7a81a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 0x7ff7a81a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57444/0 (socket says 172.21.15.92:57444) 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 -- 172.21.15.92:0/3912997650 learned_addr learned my addr 172.21.15.92:0/3912997650 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7a8102fd0 0x7ff7a819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8102680 0x7ff7a819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 -- 172.21.15.92:0/3912997650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7a8102fd0 msgr2=0x7ff7a819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7a8102fd0 0x7ff7a819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 -- 172.21.15.92:0/3912997650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8102680 msgr2=0x7ff7a819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8102680 0x7ff7a819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7acc31640 1 -- 172.21.15.92:0/3912997650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a81a2680 con 0x7ff7a8110230 2024-09-21T02:14:06.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.404+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8102680 0x7ff7a819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.405+0000 7ff7acc31640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 0x7ff7a81a1f50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000cad0 tx=0x7ff7a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.405+0000 7ff7a57fa640 1 -- 172.21.15.92:0/3912997650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000ae20 con 0x7ff7a8110230 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.405+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a81a2970 con 0x7ff7a8110230 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.405+0000 7ff7a57fa640 1 -- 172.21.15.92:0/3912997650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a0005020 con 0x7ff7a8110230 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.405+0000 7ff7a57fa640 1 -- 172.21.15.92:0/3912997650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000f6b0 con 0x7ff7a8110230 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.405+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a90d0 con 0x7ff7a8110230 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.407+0000 7ff7b4c08740 1 Processor -- start 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a57fa640 1 -- 172.21.15.92:0/3912997650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7ff7a000f850 con 0x7ff7a8110230 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 -- start start 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d3186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d318be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d319120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559c2d28eea0 con 0x559c2cf50390 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559c2d27e2d0 con 0x559c2cf481b0 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c2d2820e0 con 0x559c2cf71070 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d319120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d319120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57454/0 (socket says 172.21.15.92:57454) 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 -- 172.21.15.92:0/311691696 learned_addr learned my addr 172.21.15.92:0/311691696 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d318be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d3186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a57fa640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7900b6510 0x7ff7900b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 -- 172.21.15.92:0/311691696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 msgr2=0x559c2d3186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d3186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 -- 172.21.15.92:0/311691696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 msgr2=0x559c2d318be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.408+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d318be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff7a77fe640 1 -- 172.21.15.92:0/311691696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2cfdc650 con 0x559c2cf50390 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff7acc31640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7900b6510 0x7ff7900b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d319120 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff79c02a9a0 tx=0x7ff79c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd891ca87356800 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff7a57fa640 1 -- 172.21.15.92:0/3912997650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff7a00e7410 con 0x7ff7a8110230 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff78e7fc640 1 -- 172.21.15.92:0/311691696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c035070 con 0x559c2cf50390 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff78e7fc640 1 -- 172.21.15.92:0/311691696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c0054d0 con 0x559c2cf50390 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.409+0000 7ff78e7fc640 1 -- 172.21.15.92:0/311691696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c004a00 con 0x559c2cf50390 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 -- 172.21.15.92:0/311691696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 msgr2=0x559c2d319120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7acc31640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7900b6510 0x7ff7900b8a20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000caa0 tx=0x7ff7a007a850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d319120 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff79c02a9a0 tx=0x7ff79c005ef0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 -- 172.21.15.92:0/311691696 shutdown_connections 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d319120 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d318be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/311691696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d3186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 -- 172.21.15.92:0/311691696 >> 172.21.15.92:0/311691696 conn(0x559c2cf65460 msgr2=0x559c2d316590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 -- 172.21.15.92:0/311691696 shutdown_connections 2024-09-21T02:14:06.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 -- 172.21.15.92:0/311691696 wait complete. 2024-09-21T02:14:06.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.410+0000 7ff7b4c08740 1 -- start start 2024-09-21T02:14:06.417 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 Processor -- start 2024-09-21T02:14:06.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d323d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d324280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d330140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559c2d0df700 con 0x559c2cf71070 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559c2d219160 con 0x559c2cf481b0 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c2d326e20 con 0x559c2cf50390 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7b4c08740 10 client.0 _ll_register_callbacks cb 0x559c2d2329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d330140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d330140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57456/0 (socket says 172.21.15.92:57456) 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 -- 172.21.15.92:0/3448064721 learned_addr learned my addr 172.21.15.92:0/3448064721 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d323d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d324280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 -- 172.21.15.92:0/3448064721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 msgr2=0x559c2d324280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d324280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 -- 172.21.15.92:0/3448064721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 msgr2=0x559c2d323d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d323d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.411+0000 7ff7a77fe640 1 -- 172.21.15.92:0/3448064721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2d246c90 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d330140 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0057c0 tx=0x7ff79c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1174675b9c8c13c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7877fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7877fe640 1 -- 172.21.15.92:0/3448064721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c035070 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7877fe640 1 -- 172.21.15.92:0/3448064721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff79c03d860 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c0054d0 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7877fe640 1 -- 172.21.15.92:0/3448064721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79c03da60 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 10 client.18108 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c2d332b40 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7877fe640 1 -- 172.21.15.92:0/3448064721 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c0054d0 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7877fe640 1 -- 172.21.15.92:0/3448064721 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff79c04d440 con 0x559c2cf71070 2024-09-21T02:14:06.421 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 10 client.18108 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 10 client.18108 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 20 client.18108 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.412+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf2f9f0 0x559c2d335ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.413+0000 7ff7b4c08740 4 client.18108 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.413+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c2d290df0 con 0x559c2cf2f9f0 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.413+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf2f9f0 0x559c2d335ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.413+0000 7ff7877fe640 10 client.18108 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.413+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf2f9f0 0x559c2d335ea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e9df9c7307bab3a server_cookie=81f05ada8dfa637d in_seq=0 out_seq=0 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7877fe640 1 -- 172.21.15.92:0/3448064721 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c2d290df0 con 0x559c2cf2f9f0 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7877fe640 10 client.18108 handle_command_reply: tid=0 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 client.18108 shutdown 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 msgr2=0x559c2d330140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d330140 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0057c0 tx=0x7ff79c031400 comp rx=0 tx=0).stop 2024-09-21T02:14:06.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 shutdown_connections 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf2f9f0 0x559c2d335ea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d330140 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d324280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/3448064721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d323d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7877fe640 0 client.18108 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 >> 172.21.15.92:0/3448064721 conn(0x559c2cf65460 msgr2=0x559c2d31b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 shutdown_connections 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 1 -- 172.21.15.92:0/3448064721 wait complete. 2024-09-21T02:14:06.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.414+0000 7ff7b4c08740 20 client.18108 trim_cache size 0 max 16384 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 Processor -- start 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 -- start start 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d31a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d325970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d315bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81b1690 con 0x559c2cf50390 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81b31d0 con 0x559c2cf481b0 2024-09-21T02:14:06.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.427+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81b1d60 con 0x559c2cf71070 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d325970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d325970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57460/0 (socket says 172.21.15.92:57460) 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 -- 172.21.15.92:0/787677036 learned_addr learned my addr 172.21.15.92:0/787677036 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d31a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d315bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 -- 172.21.15.92:0/787677036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 msgr2=0x559c2d315bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d315bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 -- 172.21.15.92:0/787677036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 msgr2=0x559c2d31a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d31a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 -- 172.21.15.92:0/787677036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2d31a990 con 0x559c2cf50390 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d31a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d325970 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff7980058c0 tx=0x7ff79800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2336c94636d5a7a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff78dffb640 1 -- 172.21.15.92:0/787677036 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79801c430 con 0x559c2cf50390 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff78dffb640 1 -- 172.21.15.92:0/787677036 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79801ca80 con 0x559c2cf50390 2024-09-21T02:14:06.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.428+0000 7ff78dffb640 1 -- 172.21.15.92:0/787677036 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79800eb30 con 0x559c2cf50390 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 -- 172.21.15.92:0/787677036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 msgr2=0x559c2d325970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d325970 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff7980058c0 tx=0x7ff79800f7f0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 -- 172.21.15.92:0/787677036 shutdown_connections 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf71070 0x559c2d315bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf50390 0x559c2d325970 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/787677036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d31a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 -- 172.21.15.92:0/787677036 >> 172.21.15.92:0/787677036 conn(0x559c2cf65460 msgr2=0x559c2d325260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 -- 172.21.15.92:0/787677036 shutdown_connections 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 -- 172.21.15.92:0/787677036 wait complete. 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.429+0000 7ff7b4c08740 1 -- start start 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 Processor -- start 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d3396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d336ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d337100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81b2b00 con 0x559c2cf71070 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81b2430 con 0x559c2cf481b0 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81af670 con 0x559c2cf50390 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7b4c08740 10 client.0 _ll_register_callbacks cb 0x7ff79800b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d337100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d337100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57476/0 (socket says 172.21.15.92:57476) 2024-09-21T02:14:06.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 -- 172.21.15.92:0/2278905667 learned_addr learned my addr 172.21.15.92:0/2278905667 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d3396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d336ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 -- 172.21.15.92:0/2278905667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 msgr2=0x559c2d336ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d336ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 -- 172.21.15.92:0/2278905667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 msgr2=0x559c2d3396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d3396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.430+0000 7ff7a77fe640 1 -- 172.21.15.92:0/2278905667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2d337890 con 0x559c2cf71070 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7a77fe640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d337100 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ff79c02a9a0 tx=0x7ff79c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b30ae8d05655bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7877fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7877fe640 1 -- 172.21.15.92:0/2278905667 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c035040 con 0x559c2cf71070 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7877fe640 1 -- 172.21.15.92:0/2278905667 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff79c04dc80 con 0x559c2cf71070 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79801ca80 con 0x559c2cf71070 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7877fe640 1 -- 172.21.15.92:0/2278905667 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79c031a20 con 0x559c2cf71070 2024-09-21T02:14:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 10 client.18114 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c2d3a0580 con 0x559c2cf71070 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7877fe640 1 -- 172.21.15.92:0/2278905667 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79801ca80 con 0x559c2cf71070 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7877fe640 1 -- 172.21.15.92:0/2278905667 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff79c061040 con 0x559c2cf71070 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 10 client.18114 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 10 client.18114 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 20 client.18114 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf04da0 0x559c2d3a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 4 client.18114 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.431+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c2d28c8a0 con 0x559c2cf04da0 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.432+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf04da0 0x559c2d3a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.432+0000 7ff7877fe640 10 client.18114 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.432+0000 7ff7a7fff640 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf04da0 0x559c2d3a2cb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=508f00507683743 server_cookie=f8aae766d5d469fb in_seq=0 out_seq=0 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.432+0000 7ff7877fe640 1 -- 172.21.15.92:0/2278905667 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c2d28c8a0 con 0x559c2cf04da0 2024-09-21T02:14:06.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.432+0000 7ff7877fe640 10 client.18114 handle_command_reply: tid=0 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.432+0000 7ff7b4c08740 1 client.18114 shutdown 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 msgr2=0x559c2d337100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d337100 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ff79c02a9a0 tx=0x7ff79c041040 comp rx=0 tx=0).stop 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 shutdown_connections 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x559c2cf04da0 0x559c2d3a2cb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559c2cf71070 0x559c2d337100 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c2cf50390 0x559c2d336ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 --2- 172.21.15.92:0/2278905667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c2cf481b0 0x559c2d3396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 >> 172.21.15.92:0/2278905667 conn(0x559c2cf65460 msgr2=0x559c2d319e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7877fe640 0 client.18114 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 shutdown_connections 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 1 -- 172.21.15.92:0/2278905667 wait complete. 2024-09-21T02:14:06.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7b4c08740 20 client.18114 trim_cache size 0 max 16384 2024-09-21T02:14:06.439 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:06.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7900b6510 msgr2=0x7ff7900b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7900b6510 0x7ff7900b8a20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000caa0 tx=0x7ff7a007a850 comp rx=0 tx=0).stop 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 msgr2=0x7ff7a81a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 0x7ff7a81a1f50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000cad0 tx=0x7ff7a000cf90 comp rx=0 tx=0).stop 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 shutdown_connections 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7ff7900b6510 0x7ff7900b8a20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7a8110230 0x7ff7a81a1f50 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7a8102fd0 0x7ff7a819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.433+0000 7ff7ae694640 1 --2- 172.21.15.92:0/3912997650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7a8102680 0x7ff7a819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.434+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 >> 172.21.15.92:0/3912997650 conn(0x7ff7a8079730 msgr2=0x7ff7a8111740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.434+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 shutdown_connections 2024-09-21T02:14:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:06.434+0000 7ff7ae694640 1 -- 172.21.15.92:0/3912997650 wait complete. 2024-09-21T02:14:07.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:14:07.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:07 smithi136 ceph-mon[34754]: pgmap v634: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:14:07.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:07 smithi092 ceph-mon[31190]: pgmap v634: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:14:07.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:07 smithi099 ceph-mon[34404]: pgmap v634: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:14:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 -- 172.21.15.92:0/3551009226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103fd0 msgr2=0x7f2a7c10a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 --2- 172.21.15.92:0/3551009226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103fd0 0x7f2a7c10a970 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009ca0 tx=0x7f2a7402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 -- 172.21.15.92:0/3551009226 shutdown_connections 2024-09-21T02:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 --2- 172.21.15.92:0/3551009226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103fd0 0x7f2a7c10a970 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 --2- 172.21.15.92:0/3551009226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c103630 0x7f2a7c103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 --2- 172.21.15.92:0/3551009226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a7c1025b0 0x7f2a7c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 -- 172.21.15.92:0/3551009226 >> 172.21.15.92:0/3551009226 conn(0x7f2a7c0fdf30 msgr2=0x7f2a7c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 -- 172.21.15.92:0/3551009226 shutdown_connections 2024-09-21T02:14:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.376+0000 7f2a82f82640 1 -- 172.21.15.92:0/3551009226 wait complete. 2024-09-21T02:14:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 Processor -- start 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 -- start start 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a7c1025b0 0x7f2a7c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 0x7f2a7c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c103fd0 0x7f2a7c1a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10e1f0 con 0x7f2a7c103630 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10e070 con 0x7f2a7c103fd0 2024-09-21T02:14:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a82f82640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10e370 con 0x7f2a7c1025b0 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.377+0000 7f2a7bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 0x7f2a7c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 0x7f2a7c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57508/0 (socket says 172.21.15.92:57508) 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 -- 172.21.15.92:0/2758277368 learned_addr learned my addr 172.21.15.92:0/2758277368 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a7c1025b0 0x7f2a7c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a8151f640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c103fd0 0x7f2a7c1a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 -- 172.21.15.92:0/2758277368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a7c1025b0 msgr2=0x7f2a7c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a7c1025b0 0x7f2a7c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 -- 172.21.15.92:0/2758277368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c103fd0 msgr2=0x7f2a7c1a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c103fd0 0x7f2a7c1a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 -- 172.21.15.92:0/2758277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c1a1a00 con 0x7f2a7c103630 2024-09-21T02:14:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 0x7f2a7c19ecb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a79ffb640 1 -- 172.21.15.92:0/2758277368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800cd80 con 0x7f2a7c103630 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c1a1cf0 con 0x7f2a7c103630 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a79ffb640 1 -- 172.21.15.92:0/2758277368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a680108e0 con 0x7f2a7c103630 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.378+0000 7f2a79ffb640 1 -- 172.21.15.92:0/2758277368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68018980 con 0x7f2a7c103630 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.379+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c1a2190 con 0x7f2a7c103630 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 Processor -- start 2024-09-21T02:14:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 -- start start 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580fff4d0 0x5605813c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56058133a6f0 con 0x560580fff4d0 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560581329b20 con 0x560580ff72f0 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.381+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56058132d930 con 0x5605810201b0 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38582/0 (socket says 172.21.15.92:38582) 2024-09-21T02:14:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 -- 172.21.15.92:0/94198223 learned_addr learned my addr 172.21.15.92:0/94198223 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a79ffb640 1 -- 172.21.15.92:0/2758277368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2a68010430 con 0x7f2a7c103630 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580fff4d0 0x5605813c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 -- 172.21.15.92:0/94198223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 msgr2=0x5605813c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 -- 172.21.15.92:0/94198223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580fff4d0 msgr2=0x5605813c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580fff4d0 0x5605813c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 -- 172.21.15.92:0/94198223 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605812ef530 con 0x5605810201b0 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580fff4d0 0x5605813c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.382+0000 7f2a8151f640 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813c7e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a74035000 tx=0x7f2a74009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e4c5ca255b58c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.383+0000 7f2a66ffd640 1 -- 172.21.15.92:0/94198223 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74031650 con 0x5605810201b0 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.383+0000 7f2a66ffd640 1 -- 172.21.15.92:0/94198223 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a74036030 con 0x5605810201b0 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.383+0000 7f2a66ffd640 1 -- 172.21.15.92:0/94198223 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74004840 con 0x5605810201b0 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.383+0000 7f2a79ffb640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a6c0b6510 0x7f2a6c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.383+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a6c0b6510 0x7f2a6c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.383+0000 7f2a79ffb640 1 -- 172.21.15.92:0/2758277368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2a680e75d0 con 0x7f2a7c103630 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 -- 172.21.15.92:0/94198223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 msgr2=0x5605813c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a6c0b6510 0x7f2a6c0b8a20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a70005d00 tx=0x7f2a7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813c7e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a74035000 tx=0x7f2a74009800 comp rx=0 tx=0).stop 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 -- 172.21.15.92:0/94198223 shutdown_connections 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580fff4d0 0x5605813c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/94198223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813c7e50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 -- 172.21.15.92:0/94198223 >> 172.21.15.92:0/94198223 conn(0x560581014300 msgr2=0x5605813c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 -- 172.21.15.92:0/94198223 shutdown_connections 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 -- 172.21.15.92:0/94198223 wait complete. 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.384+0000 7f2a89fcd740 1 -- start start 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 Processor -- start 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580fff4d0 0x5605813d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560581261ab0 con 0x560580ff72f0 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5605812cf730 con 0x560580fff4d0 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5605813c9950 con 0x5605810201b0 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a89fcd740 10 client.0 _ll_register_callbacks cb 0x5605813236b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a80d1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a80d1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57524/0 (socket says 172.21.15.92:57524) 2024-09-21T02:14:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a80d1e640 1 -- 172.21.15.92:0/228669769 learned_addr learned my addr 172.21.15.92:0/228669769 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a8151f640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.385+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580fff4d0 0x5605813d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a80d1e640 1 -- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 msgr2=0x5605813d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a80d1e640 1 -- 172.21.15.92:0/228669769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580fff4d0 msgr2=0x5605813d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580fff4d0 0x5605813d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a80d1e640 1 -- 172.21.15.92:0/228669769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56058108b790 con 0x560580ff72f0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813d13e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001f550 tx=0x7f2a70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fffb30518be5d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a5bfff640 1 -- 172.21.15.92:0/228669769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005020 con 0x560580ff72f0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a5bfff640 1 -- 172.21.15.92:0/228669769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2a70017030 con 0x560580ff72f0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a74036030 con 0x560580ff72f0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a5bfff640 1 -- 172.21.15.92:0/228669769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a700051c0 con 0x560580ff72f0 2024-09-21T02:14:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a89fcd740 10 client.18123 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.386+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560581104620 con 0x560580ff72f0 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a5bfff640 1 -- 172.21.15.92:0/228669769 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a74036030 con 0x560580ff72f0 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a5bfff640 1 -- 172.21.15.92:0/228669769 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a7001c690 con 0x560580ff72f0 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a89fcd740 10 client.18123 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a89fcd740 10 client.18123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a89fcd740 20 client.18123 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fded80 0x5605813deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a89fcd740 4 client.18123 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56058133c640 con 0x560580fded80 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.387+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fded80 0x5605813deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a5bfff640 10 client.18123 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fded80 0x5605813deb90 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5e1e999cdf6e55 server_cookie=eb49454e81a8f7ac in_seq=0 out_seq=0 2024-09-21T02:14:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a5bfff640 1 -- 172.21.15.92:0/228669769 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56058133c640 con 0x560580fded80 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a5bfff640 10 client.18123 handle_command_reply: tid=0 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a89fcd740 1 client.18123 shutdown 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 msgr2=0x5605813d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.388+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813d13e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001f550 tx=0x7f2a70004730 comp rx=0 tx=0).stop 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 shutdown_connections 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fded80 0x5605813deb90 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a5bfff640 0 client.18123 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605810201b0 0x5605813d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580fff4d0 0x5605813d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/228669769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813d13e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 >> 172.21.15.92:0/228669769 conn(0x560581014300 msgr2=0x5605813c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 shutdown_connections 2024-09-21T02:14:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 1 -- 172.21.15.92:0/228669769 wait complete. 2024-09-21T02:14:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.389+0000 7f2a89fcd740 20 client.18123 trim_cache size 0 max 16384 2024-09-21T02:14:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.411+0000 7f2a89fcd740 1 Processor -- start 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.411+0000 7f2a89fcd740 1 -- start start 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.411+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.411+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605810201b0 0x5605813cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1aafa0 con 0x560580ff72f0 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1acae0 con 0x5605810201b0 2024-09-21T02:14:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1ab670 con 0x560580fff4d0 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57536/0 (socket says 172.21.15.92:57536) 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 -- 172.21.15.92:0/4183033775 learned_addr learned my addr 172.21.15.92:0/4183033775 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a8151f640 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605810201b0 0x5605813cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 -- 172.21.15.92:0/4183033775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 msgr2=0x5605813cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 -- 172.21.15.92:0/4183033775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605810201b0 msgr2=0x5605813cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605810201b0 0x5605813cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 -- 172.21.15.92:0/4183033775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605813c9d50 con 0x560580ff72f0 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a8151f640 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605810201b0 0x5605813cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813cc670 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001cb00 tx=0x7f2a7000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772c4aca1d14bee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a667fc640 1 -- 172.21.15.92:0/4183033775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7003f070 con 0x560580ff72f0 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a667fc640 1 -- 172.21.15.92:0/4183033775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7001c800 con 0x560580ff72f0 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.412+0000 7f2a667fc640 1 -- 172.21.15.92:0/4183033775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70036050 con 0x560580ff72f0 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 -- 172.21.15.92:0/4183033775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 msgr2=0x5605813cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813cc670 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001cb00 tx=0x7f2a7000bbf0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 -- 172.21.15.92:0/4183033775 shutdown_connections 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605810201b0 0x5605813cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/4183033775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560580ff72f0 0x5605813cc670 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 -- 172.21.15.92:0/4183033775 >> 172.21.15.92:0/4183033775 conn(0x560581014300 msgr2=0x5605813cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 -- 172.21.15.92:0/4183033775 shutdown_connections 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 -- 172.21.15.92:0/4183033775 wait complete. 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 -- start start 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.414+0000 7f2a89fcd740 1 Processor -- start 2024-09-21T02:14:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 0x5605813d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1ac410 con 0x5605810201b0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1abd40 con 0x560580ff72f0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1a8f80 con 0x560580fff4d0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a89fcd740 10 client.0 _ll_register_callbacks cb 0x7f2a7001f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 0x5605813d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 0x5605813d8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57548/0 (socket says 172.21.15.92:57548) 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 -- 172.21.15.92:0/524667327 learned_addr learned my addr 172.21.15.92:0/524667327 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a7bfff640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 -- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 msgr2=0x5605813d7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813d7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 -- 172.21.15.92:0/524667327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 msgr2=0x5605813d8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813d8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.415+0000 7f2a8151f640 1 -- 172.21.15.92:0/524667327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605813d8620 con 0x5605810201b0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a8151f640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 0x5605813d8090 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2a7402f4c0 tx=0x7f2a74003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b303dcdcb940e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a5bfff640 1 -- 172.21.15.92:0/524667327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74055020 con 0x5605810201b0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a5bfff640 1 -- 172.21.15.92:0/524667327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2a7402f820 con 0x5605810201b0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a7001c800 con 0x5605810201b0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a5bfff640 1 -- 172.21.15.92:0/524667327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a7404a040 con 0x5605810201b0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a89fcd740 10 client.18129 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605813d7450 con 0x5605810201b0 2024-09-21T02:14:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.416+0000 7f2a5bfff640 1 -- 172.21.15.92:0/524667327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7001c800 con 0x5605810201b0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a5bfff640 1 -- 172.21.15.92:0/524667327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a74043070 con 0x5605810201b0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a89fcd740 10 client.18129 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a89fcd740 10 client.18129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a89fcd740 20 client.18129 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fb41c0 0x56058144ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a89fcd740 4 client.18129 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605813380f0 con 0x560580fb41c0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fb41c0 0x56058144ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a5bfff640 10 client.18129 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.417+0000 7f2a80d1e640 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fb41c0 0x56058144ba20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490887b079e7dc08 server_cookie=ec84c8a751137baa in_seq=0 out_seq=0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.432+0000 7f2a5bfff640 1 -- 172.21.15.92:0/524667327 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605813380f0 con 0x560580fb41c0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.432+0000 7f2a5bfff640 10 client.18129 handle_command_reply: tid=0 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 client.18129 shutdown 2024-09-21T02:14:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 msgr2=0x5605813d8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 0x5605813d8090 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2a7402f4c0 tx=0x7f2a74003ba0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 shutdown_connections 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560580fb41c0 0x56058144ba20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605810201b0 0x5605813d8090 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560580fff4d0 0x5605813d7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 --2- 172.21.15.92:0/524667327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560580ff72f0 0x5605813d8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a5bfff640 0 client.18129 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 >> 172.21.15.92:0/524667327 conn(0x560581014300 msgr2=0x5605813c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 shutdown_connections 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 1 -- 172.21.15.92:0/524667327 wait complete. 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.433+0000 7f2a89fcd740 20 client.18129 trim_cache size 0 max 16384 2024-09-21T02:14:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:14:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:14:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:14:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:14:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a6c0b6510 msgr2=0x7f2a6c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a6c0b6510 0x7f2a6c0b8a20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a70005d00 tx=0x7f2a7000b410 comp rx=0 tx=0).stop 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 msgr2=0x7f2a7c19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 0x7f2a7c19ecb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 shutdown_connections 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a6c0b6510 0x7f2a6c0b8a20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7c103fd0 0x7f2a7c1a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a7c103630 0x7f2a7c19ecb0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 --2- 172.21.15.92:0/2758277368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a7c1025b0 0x7f2a7c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 >> 172.21.15.92:0/2758277368 conn(0x7f2a7c0fdf30 msgr2=0x7f2a7c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 shutdown_connections 2024-09-21T02:14:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:07.434+0000 7f2a82f82640 1 -- 172.21.15.92:0/2758277368 wait complete. 2024-09-21T02:14:07.854 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/1118148426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a410de10 msgr2=0x7fc0a4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/1118148426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a410de10 0x7fc0a4110300 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009ca0 tx=0x7fc09c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/1118148426 shutdown_connections 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/1118148426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a410de10 0x7fc0a4110300 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/1118148426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4105300 0x7fc0a410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/1118148426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a41049b0 0x7fc0a4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/1118148426 >> 172.21.15.92:0/1118148426 conn(0x7fc0a4079730 msgr2=0x7fc0a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/1118148426 shutdown_connections 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.844+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/1118148426 wait complete. 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 Processor -- start 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 -- start start 2024-09-21T02:14:07.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a41049b0 0x7fc0a419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 0x7fc0a419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a410de10 0x7fc0a419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4113a90 con 0x7fc0a4105300 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4113910 con 0x7fc0a41049b0 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0ab2ef640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4113c10 con 0x7fc0a410de10 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 0x7fc0a419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 0x7fc0a419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57566/0 (socket says 172.21.15.92:57566) 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 -- 172.21.15.92:0/763472186 learned_addr learned my addr 172.21.15.92:0/763472186 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a908b640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a41049b0 0x7fc0a419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a988c640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a410de10 0x7fc0a419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 -- 172.21.15.92:0/763472186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a410de10 msgr2=0x7fc0a419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a410de10 0x7fc0a419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 -- 172.21.15.92:0/763472186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a41049b0 msgr2=0x7fc0a419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a41049b0 0x7fc0a419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a888a640 1 -- 172.21.15.92:0/763472186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a41a03d0 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.845+0000 7fc0a988c640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a410de10 0x7fc0a419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.846+0000 7fc0a888a640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 0x7fc0a419aa00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc09800aec0 tx=0x7fc09800cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.846+0000 7fc0a27fc640 1 -- 172.21.15.92:0/763472186 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098007bb0 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.846+0000 7fc0a27fc640 1 -- 172.21.15.92:0/763472186 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc098007d50 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.846+0000 7fc0a27fc640 1 -- 172.21.15.92:0/763472186 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098011750 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.846+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a41a06c0 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.846+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a41025d0 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a27fc640 1 -- 172.21.15.92:0/763472186 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc098005560 con 0x7fc0a4105300 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 Processor -- start 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 -- start start 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 0x5616947162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616943461b0 0x561694716800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694716d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56169468cea0 con 0x56169434e390 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56169467c2d0 con 0x5616943461b0 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0b1863740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616946800e0 con 0x56169436f070 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a988c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 0x5616947162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694716d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a988c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 0x5616947162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38638/0 (socket says 172.21.15.92:38638) 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a988c640 1 -- 172.21.15.92:0/2736679724 learned_addr learned my addr 172.21.15.92:0/2736679724 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a908b640 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616943461b0 0x561694716800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.849+0000 7fc0a27fc640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc08c0b6510 0x7fc08c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a27fc640 1 -- 172.21.15.92:0/763472186 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc0980eaf30 con 0x7fc0a4105300 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a888a640 1 -- 172.21.15.92:0/2736679724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 msgr2=0x5616947162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a888a640 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 0x5616947162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a888a640 1 -- 172.21.15.92:0/2736679724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616943461b0 msgr2=0x561694716800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a888a640 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616943461b0 0x561694716800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a888a640 1 -- 172.21.15.92:0/2736679724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616943da650 con 0x56169434e390 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a988c640 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 0x5616947162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a908b640 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616943461b0 0x561694716800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a988c640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc08c0b6510 0x7fc08c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc0a888a640 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694716d40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc0980e89e0 tx=0x7fc098004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255585118dd11527 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc08b7fe640 1 -- 172.21.15.92:0/2736679724 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980b09f0 con 0x56169434e390 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc08b7fe640 1 -- 172.21.15.92:0/2736679724 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0980b0bb0 con 0x56169434e390 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.850+0000 7fc08b7fe640 1 -- 172.21.15.92:0/2736679724 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980b0810 con 0x56169434e390 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 -- 172.21.15.92:0/2736679724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 msgr2=0x561694716d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694716d40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc0980e89e0 tx=0x7fc098004910 comp rx=0 tx=0).stop 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0a988c640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc08c0b6510 0x7fc08c0b8a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0094e0 tx=0x7fc09c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 -- 172.21.15.92:0/2736679724 shutdown_connections 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694716d40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616943461b0 0x561694716800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2736679724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169436f070 0x5616947162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 -- 172.21.15.92:0/2736679724 >> 172.21.15.92:0/2736679724 conn(0x561694363460 msgr2=0x5616947141b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 -- 172.21.15.92:0/2736679724 shutdown_connections 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 -- 172.21.15.92:0/2736679724 wait complete. 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 -- start start 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.851+0000 7fc0b1863740 1 Processor -- start 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616943461b0 0x5616947219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694721ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169436f070 0x56169472dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5616944dd700 con 0x56169434e390 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561694617160 con 0x56169436f070 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561694724a40 con 0x5616943461b0 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0b1863740 10 client.0 _ll_register_callbacks cb 0x5616946309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694721ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.860 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694721ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57594/0 (socket says 172.21.15.92:57594) 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 -- 172.21.15.92:0/1372405319 learned_addr learned my addr 172.21.15.92:0/1372405319 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a908b640 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616943461b0 0x5616947219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 -- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616943461b0 msgr2=0x5616947219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616943461b0 0x5616947219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 -- 172.21.15.92:0/1372405319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169436f070 msgr2=0x56169472dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169436f070 0x56169472dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.852+0000 7fc0a888a640 1 -- 172.21.15.92:0/1372405319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561694644c90 con 0x56169434e390 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0a888a640 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694721ee0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fc0980b0da0 tx=0x7fc0980b05b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abd6c5f4937a631 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc088ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc088ff9640 1 -- 172.21.15.92:0/1372405319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980b09f0 con 0x56169434e390 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc088ff9640 1 -- 172.21.15.92:0/1372405319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc0980eeb40 con 0x56169434e390 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0980b0bb0 con 0x56169434e390 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc088ff9640 1 -- 172.21.15.92:0/1372405319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc0980b5b80 con 0x56169434e390 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 10 client.18141 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561694730780 con 0x56169434e390 2024-09-21T02:14:07.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc088ff9640 1 -- 172.21.15.92:0/1372405319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0980b0bb0 con 0x56169434e390 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc088ff9640 1 -- 172.21.15.92:0/1372405319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc0980afd70 con 0x56169434e390 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 10 client.18141 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 10 client.18141 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 20 client.18141 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169432d9f0 0x561694733ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 4 client.18141 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.853+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56169468edf0 con 0x56169432d9f0 2024-09-21T02:14:07.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.854+0000 7fc0a908b640 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169432d9f0 0x561694733ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.854+0000 7fc088ff9640 10 client.18141 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.854+0000 7fc0a908b640 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169432d9f0 0x561694733ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69672e9f44b08b52 server_cookie=83d8f3cd217ab30b in_seq=0 out_seq=0 2024-09-21T02:14:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc088ff9640 1 -- 172.21.15.92:0/1372405319 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56169468edf0 con 0x56169432d9f0 2024-09-21T02:14:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc088ff9640 10 client.18141 handle_command_reply: tid=0 2024-09-21T02:14:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 client.18141 shutdown 2024-09-21T02:14:07.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 msgr2=0x561694721ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694721ee0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fc0980b0da0 tx=0x7fc0980b05b0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 shutdown_connections 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169432d9f0 0x561694733ae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169436f070 0x56169472dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc088ff9640 0 client.18141 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56169434e390 0x561694721ee0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 --2- 172.21.15.92:0/1372405319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616943461b0 0x5616947219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 >> 172.21.15.92:0/1372405319 conn(0x561694363460 msgr2=0x561694719450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 shutdown_connections 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 1 -- 172.21.15.92:0/1372405319 wait complete. 2024-09-21T02:14:07.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.855+0000 7fc0b1863740 20 client.18141 trim_cache size 0 max 16384 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.868+0000 7fc0b1863740 1 Processor -- start 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.868+0000 7fc0b1863740 1 -- start start 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694723880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694717a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x56169472dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0b1863740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ad350 con 0x5616943461b0 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0b1863740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41aee90 con 0x56169432d9f0 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0b1863740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ada20 con 0x56169434e390 2024-09-21T02:14:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694717a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694717a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57606/0 (socket says 172.21.15.92:57606) 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 -- 172.21.15.92:0/2052855007 learned_addr learned my addr 172.21.15.92:0/2052855007 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a988c640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x56169472dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 -- 172.21.15.92:0/2052855007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 msgr2=0x56169472dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a908b640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694723880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x56169472dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 -- 172.21.15.92:0/2052855007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 msgr2=0x561694723880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694723880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 -- 172.21.15.92:0/2052855007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616947185b0 con 0x5616943461b0 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a908b640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694723880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:07.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a988c640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x56169472dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:07.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc0a888a640 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694717a00 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc0980e8d80 tx=0x7fc098004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6eba458b54be9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc08affd640 1 -- 172.21.15.92:0/2052855007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980b09f0 con 0x5616943461b0 2024-09-21T02:14:07.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc08affd640 1 -- 172.21.15.92:0/2052855007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0980b0620 con 0x5616943461b0 2024-09-21T02:14:07.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.869+0000 7fc08affd640 1 -- 172.21.15.92:0/2052855007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980ee7e0 con 0x5616943461b0 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 -- 172.21.15.92:0/2052855007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 msgr2=0x561694717a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694717a00 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc0980e8d80 tx=0x7fc098004910 comp rx=0 tx=0).stop 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 -- 172.21.15.92:0/2052855007 shutdown_connections 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x56169472dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694717a00 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 --2- 172.21.15.92:0/2052855007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694723880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 -- 172.21.15.92:0/2052855007 >> 172.21.15.92:0/2052855007 conn(0x561694363460 msgr2=0x5616947172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 -- 172.21.15.92:0/2052855007 shutdown_connections 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.870+0000 7fc0b1863740 1 -- 172.21.15.92:0/2052855007 wait complete. 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 -- start start 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 Processor -- start 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694734b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694735040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x561694735580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ae7c0 con 0x5616943461b0 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ae0f0 con 0x56169432d9f0 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a41ab330 con 0x56169434e390 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0b1863740 10 client.0 _ll_register_callbacks cb 0x7fc09400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694735040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0a888a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694735040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57618/0 (socket says 172.21.15.92:57618) 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0a888a640 1 -- 172.21.15.92:0/3066325275 learned_addr learned my addr 172.21.15.92:0/3066325275 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.871+0000 7fc0a988c640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x561694735580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a908b640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694734b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a888a640 1 -- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 msgr2=0x561694735580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a888a640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x561694735580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a888a640 1 -- 172.21.15.92:0/3066325275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 msgr2=0x561694734b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a888a640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694734b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a888a640 1 -- 172.21.15.92:0/3066325275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169479d260 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0a888a640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694735040 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc0980b0da0 tx=0x7fc0980b0510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53efc9bb778c9f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc088ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc088ff9640 1 -- 172.21.15.92:0/3066325275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980b17a0 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc088ff9640 1 -- 172.21.15.92:0/3066325275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc0980046c0 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0980b0620 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc088ff9640 1 -- 172.21.15.92:0/3066325275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc0980baac0 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0b1863740 10 client.18147 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56169479d260 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.872+0000 7fc088ff9640 1 -- 172.21.15.92:0/3066325275 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0980b0620 con 0x5616943461b0 2024-09-21T02:14:07.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc088ff9640 1 -- 172.21.15.92:0/3066325275 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc0980b6ea0 con 0x5616943461b0 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 10 client.18147 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 10 client.18147 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 20 client.18147 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169436f070 0x5616947a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 4 client.18147 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56169468a8a0 con 0x56169436f070 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0a908b640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169436f070 0x5616947a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc088ff9640 10 client.18147 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0a908b640 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169436f070 0x5616947a0660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93cc61e0cf2cd130 server_cookie=e4622aca404978ca in_seq=0 out_seq=0 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc088ff9640 1 -- 172.21.15.92:0/3066325275 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56169468a8a0 con 0x56169436f070 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc088ff9640 10 client.18147 handle_command_reply: tid=0 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.873+0000 7fc0b1863740 1 client.18147 shutdown 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 msgr2=0x561694735040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.878 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694735040 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc0980b0da0 tx=0x7fc0980b0510 comp rx=0 tx=0).stop 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 shutdown_connections 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169434e390 0x561694735580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616943461b0 0x561694735040 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x56169436f070 0x5616947a0660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 --2- 172.21.15.92:0/3066325275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56169432d9f0 0x561694734b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc088ff9640 0 client.18147 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 >> 172.21.15.92:0/3066325275 conn(0x561694363460 msgr2=0x561694723810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 shutdown_connections 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 1 -- 172.21.15.92:0/3066325275 wait complete. 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0b1863740 20 client.18147 trim_cache size 0 max 16384 2024-09-21T02:14:07.879 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc08c0b6510 msgr2=0x7fc08c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc08c0b6510 0x7fc08c0b8a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0094e0 tx=0x7fc09c0095c0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 msgr2=0x7fc0a419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 0x7fc0a419aa00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc09800aec0 tx=0x7fc09800cd40 comp rx=0 tx=0).stop 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 shutdown_connections 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc08c0b6510 0x7fc08c0b8a20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a410de10 0x7fc0a419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a4105300 0x7fc0a419aa00 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 --2- 172.21.15.92:0/763472186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a41049b0 0x7fc0a419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.874+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 >> 172.21.15.92:0/763472186 conn(0x7fc0a4079730 msgr2=0x7fc0a40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.875+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 shutdown_connections 2024-09-21T02:14:07.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:07.875+0000 7fc0ab2ef640 1 -- 172.21.15.92:0/763472186 wait complete. 2024-09-21T02:14:08.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:14:08.881 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:09.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:09 smithi092 ceph-mon[31190]: pgmap v635: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:14:09.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:09 smithi136 ceph-mon[34754]: pgmap v635: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:14:09.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:09 smithi099 ceph-mon[34404]: pgmap v635: 97 pgs: 97 active+clean; 2.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:14:09.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 -- 172.21.15.92:0/2412431099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 msgr2=0x7f75b8110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/2412431099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b8110640 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009ca0 tx=0x7f75b002a8e0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 -- 172.21.15.92:0/2412431099 shutdown_connections 2024-09-21T02:14:09.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/2412431099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b8110640 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/2412431099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 0x7f75b81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/2412431099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b8103b80 0x7f75b8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 -- 172.21.15.92:0/2412431099 >> 172.21.15.92:0/2412431099 conn(0x7f75b80ff610 msgr2=0x7f75b8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 -- 172.21.15.92:0/2412431099 shutdown_connections 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.353+0000 7f75bf1fc640 1 -- 172.21.15.92:0/2412431099 wait complete. 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 Processor -- start 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 -- start start 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b8103b80 0x7f75b819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 0x7f75b819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8113b30 con 0x7f75b8109e10 2024-09-21T02:14:09.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81139b0 con 0x7f75b81094c0 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bf1fc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8113cb0 con 0x7f75b8103b80 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57640/0 (socket says 172.21.15.92:57640) 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 -- 172.21.15.92:0/547046959 learned_addr learned my addr 172.21.15.92:0/547046959 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bcf98640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b8103b80 0x7f75b819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75b7fff640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 0x7f75b819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 -- 172.21.15.92:0/547046959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b8103b80 msgr2=0x7f75b819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b8103b80 0x7f75b819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 -- 172.21.15.92:0/547046959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 msgr2=0x7f75b819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 0x7f75b819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 -- 172.21.15.92:0/547046959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81a03f0 con 0x7f75b8109e10 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75b7fff640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 0x7f75b819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.354+0000 7f75bd799640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b819fcc0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005cc0 tx=0x7f75b00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.355+0000 7f75b5ffb640 1 -- 172.21.15.92:0/547046959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0003de0 con 0x7f75b8109e10 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.355+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b81a0680 con 0x7f75b8109e10 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.355+0000 7f75b5ffb640 1 -- 172.21.15.92:0/547046959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b00044f0 con 0x7f75b8109e10 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.355+0000 7f75b5ffb640 1 -- 172.21.15.92:0/547046959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0030d60 con 0x7f75b8109e10 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.355+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a6e40 con 0x7f75b8109e10 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75b5ffb640 1 -- 172.21.15.92:0/547046959 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f75b0004040 con 0x7f75b8109e10 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 Processor -- start 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 -- start start 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604001e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56040015eea0 con 0x5603ffe181b0 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56040014e2d0 con 0x5603ffe20390 2024-09-21T02:14:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75c5770740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5604001520e0 con 0x5603ffe41070 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001e8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57644/0 (socket says 172.21.15.92:57644) 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 -- 172.21.15.92:0/513578171 learned_addr learned my addr 172.21.15.92:0/513578171 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75b5ffb640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75a00b6510 0x7f75a00b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75b7fff640 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604001e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75b7fff640 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bd799640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75a00b6510 0x7f75a00b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 -- 172.21.15.92:0/513578171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 msgr2=0x5604001e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604001e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 -- 172.21.15.92:0/513578171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 msgr2=0x5604001e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.358+0000 7f75bcf98640 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001e9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f75bcf98640 1 -- 172.21.15.92:0/513578171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ffeac650 con 0x5603ffe181b0 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f75b7fff640 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604001e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f75b5ffb640 1 -- 172.21.15.92:0/547046959 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f75b01085f0 con 0x7f75b8109e10 2024-09-21T02:14:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f75bcf98640 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001e8be0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f75a800b760 tx=0x7f75a800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dfb067c9f266f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f759effd640 1 -- 172.21.15.92:0/513578171 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003dd0 con 0x5603ffe181b0 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f75bd799640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75a00b6510 0x7f75a00b8a20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ce460 tx=0x7f75b00c3010 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f759effd640 1 -- 172.21.15.92:0/513578171 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a8015070 con 0x5603ffe181b0 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.359+0000 7f759effd640 1 -- 172.21.15.92:0/513578171 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8004d50 con 0x5603ffe181b0 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 -- 172.21.15.92:0/513578171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 msgr2=0x5604001e8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001e8be0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f75a800b760 tx=0x7f75a800bc20 comp rx=0 tx=0).stop 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 -- 172.21.15.92:0/513578171 shutdown_connections 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001e8be0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 --2- 172.21.15.92:0/513578171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604001e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 -- 172.21.15.92:0/513578171 >> 172.21.15.92:0/513578171 conn(0x5603ffe35460 msgr2=0x5604001e6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 -- 172.21.15.92:0/513578171 shutdown_connections 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 -- 172.21.15.92:0/513578171 wait complete. 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 -- start start 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.360+0000 7f75c5770740 1 Processor -- start 2024-09-21T02:14:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604002001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5603fffaf700 con 0x5603ffe181b0 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5604000e9160 con 0x5603ffe20390 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5604001f6e80 con 0x5603ffe41070 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75c5770740 10 client.0 _ll_register_callbacks cb 0x5604001029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57646/0 (socket says 172.21.15.92:57646) 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 -- 172.21.15.92:0/1176652865 learned_addr learned my addr 172.21.15.92:0/1176652865 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 -- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 msgr2=0x5604002001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604002001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 -- 172.21.15.92:0/1176652865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 msgr2=0x5604001f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.361+0000 7f75b7fff640 1 -- 172.21.15.92:0/1176652865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560400116c90 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001f3d70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00cf90 tx=0x7f75ac011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818b83c4d6c3f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f7597fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f7597fff640 1 -- 172.21.15.92:0/1176652865 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac018070 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f7597fff640 1 -- 172.21.15.92:0/1176652865 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f75ac00f790 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a8015070 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f7597fff640 1 -- 172.21.15.92:0/1176652865 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75ac024050 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 10 client.18159 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560400202b80 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f7597fff640 1 -- 172.21.15.92:0/1176652865 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a8015070 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f7597fff640 1 -- 172.21.15.92:0/1176652865 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f75ac014a80 con 0x5603ffe181b0 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 10 client.18159 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 10 client.18159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 20 client.18159 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffdff9f0 0x560400206010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 4 client.18159 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.362+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560400160df0 con 0x5603ffdff9f0 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.363+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffdff9f0 0x560400206010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.363+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffdff9f0 0x560400206010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fa6b51db4df7943 server_cookie=c7cc79dbe37a2b07 in_seq=0 out_seq=0 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.363+0000 7f7597fff640 10 client.18159 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f7597fff640 1 -- 172.21.15.92:0/1176652865 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560400160df0 con 0x5603ffdff9f0 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f7597fff640 10 client.18159 handle_command_reply: tid=0 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 client.18159 shutdown 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 msgr2=0x5604001f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001f3d70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00cf90 tx=0x7f75ac011970 comp rx=0 tx=0).stop 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 shutdown_connections 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffdff9f0 0x560400206010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffe41070 0x5604002001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f7597fff640 0 client.18159 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe20390 0x5604001f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 --2- 172.21.15.92:0/1176652865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe181b0 0x5604001f3d70 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 >> 172.21.15.92:0/1176652865 conn(0x5603ffe35460 msgr2=0x5604001eb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 shutdown_connections 2024-09-21T02:14:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 1 -- 172.21.15.92:0/1176652865 wait complete. 2024-09-21T02:14:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.364+0000 7f75c5770740 20 client.18159 trim_cache size 0 max 16384 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 Processor -- start 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 -- start start 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81af470 con 0x5603ffe20390 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81b0fb0 con 0x5603ffe181b0 2024-09-21T02:14:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75c5770740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81afb40 con 0x5603ffdff9f0 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57650/0 (socket says 172.21.15.92:57650) 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75b7fff640 1 -- 172.21.15.92:0/1270364193 learned_addr learned my addr 172.21.15.92:0/1270364193 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.378+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75b7fff640 1 -- 172.21.15.92:0/1270364193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 msgr2=0x5604001e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001e9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75b7fff640 1 -- 172.21.15.92:0/1270364193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 msgr2=0x5604001e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001e5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75b7fff640 1 -- 172.21.15.92:0/1270364193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604001f4cc0 con 0x5603ffe20390 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001e6100 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f75ac014630 tx=0x7f75ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38d108ef642ab4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f759e7fc640 1 -- 172.21.15.92:0/1270364193 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac018040 con 0x5603ffe20390 2024-09-21T02:14:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f759e7fc640 1 -- 172.21.15.92:0/1270364193 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75ac01cd50 con 0x5603ffe20390 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.379+0000 7f759e7fc640 1 -- 172.21.15.92:0/1270364193 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00f590 con 0x5603ffe20390 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 -- 172.21.15.92:0/1270364193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 msgr2=0x5604001e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001e6100 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f75ac014630 tx=0x7f75ac0046c0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 -- 172.21.15.92:0/1270364193 shutdown_connections 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001e6100 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 --2- 172.21.15.92:0/1270364193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 -- 172.21.15.92:0/1270364193 >> 172.21.15.92:0/1270364193 conn(0x5603ffe35460 msgr2=0x5604001e96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 -- 172.21.15.92:0/1270364193 shutdown_connections 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.380+0000 7f75c5770740 1 -- 172.21.15.92:0/1270364193 wait complete. 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 -- start start 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 Processor -- start 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81b08e0 con 0x5603ffe20390 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81b0210 con 0x5603ffe181b0 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ad450 con 0x5603ffdff9f0 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75c5770740 10 client.0 _ll_register_callbacks cb 0x7f75ac00c510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57656/0 (socket says 172.21.15.92:57656) 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75b7fff640 1 -- 172.21.15.92:0/1815885836 learned_addr learned my addr 172.21.15.92:0/1815885836 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.381+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75b7fff640 1 -- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 msgr2=0x5604001fc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001fc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75b7fff640 1 -- 172.21.15.92:0/1815885836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 msgr2=0x5604001f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:09.387 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75b7fff640 1 -- 172.21.15.92:0/1815885836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604001f8ba0 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75b7fff640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001f8410 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0143f0 tx=0x7f75ac014920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20120b2080e97d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f7597fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f7597fff640 1 -- 172.21.15.92:0/1815885836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac018040 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f7597fff640 1 -- 172.21.15.92:0/1815885836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f75ac014bc0 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75ac01cd50 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f7597fff640 1 -- 172.21.15.92:0/1815885836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75ac01c8a0 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75c5770740 10 client.18165 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560400274da0 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.382+0000 7f7597fff640 1 -- 172.21.15.92:0/1815885836 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75ac01cd50 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f7597fff640 1 -- 172.21.15.92:0/1815885836 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f75ac00fb60 con 0x5603ffe20390 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75c5770740 10 client.18165 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75c5770740 10 client.18165 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75c5770740 20 client.18165 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:09.388 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75c5770740 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffe41070 0x560400277490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75c5770740 4 client.18165 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56040015c8a0 con 0x5603ffe41070 2024-09-21T02:14:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffe41070 0x560400277490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f7597fff640 10 client.18165 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:09.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.383+0000 7f75bcf98640 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffe41070 0x560400277490 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2123c671e2e41699 server_cookie=a378f6b52738a431 in_seq=0 out_seq=0 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f7597fff640 1 -- 172.21.15.92:0/1815885836 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56040015c8a0 con 0x5603ffe41070 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f7597fff640 10 client.18165 handle_command_reply: tid=0 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 client.18165 shutdown 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 msgr2=0x5604001f8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001f8410 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0143f0 tx=0x7f75ac014920 comp rx=0 tx=0).stop 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 shutdown_connections 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x5603ffe41070 0x560400277490 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f7597fff640 0 client.18165 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603ffe20390 0x5604001f8410 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ffe181b0 0x5604001f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 --2- 172.21.15.92:0/1815885836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ffdff9f0 0x5604001fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 >> 172.21.15.92:0/1815885836 conn(0x5603ffe35460 msgr2=0x5604001f51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 shutdown_connections 2024-09-21T02:14:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 1 -- 172.21.15.92:0/1815885836 wait complete. 2024-09-21T02:14:09.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.384+0000 7f75c5770740 20 client.18165 trim_cache size 0 max 16384 2024-09-21T02:14:09.391 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75a00b6510 msgr2=0x7f75a00b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75a00b6510 0x7f75a00b8a20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ce460 tx=0x7f75b00c3010 comp rx=0 tx=0).stop 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 msgr2=0x7f75b819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b819fcc0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005cc0 tx=0x7f75b00039f0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 shutdown_connections 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f75a00b6510 0x7f75a00b8a20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75b8109e10 0x7f75b819fcc0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75b81094c0 0x7f75b819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 --2- 172.21.15.92:0/547046959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b8103b80 0x7f75b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 >> 172.21.15.92:0/547046959 conn(0x7f75b80ff610 msgr2=0x7f75b8106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 shutdown_connections 2024-09-21T02:14:09.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:09.385+0000 7f75bf1fc640 1 -- 172.21.15.92:0/547046959 wait complete. 2024-09-21T02:14:10.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:14:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.834+0000 7f803ac37640 1 -- 172.21.15.92:0/156500493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8034103bb0 msgr2=0x7f8034103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.834+0000 7f803ac37640 1 --2- 172.21.15.92:0/156500493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8034103bb0 0x7f8034103fc0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f802c009ca0 tx=0x7f802c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 -- 172.21.15.92:0/156500493 shutdown_connections 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 --2- 172.21.15.92:0/156500493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8034109e40 0x7f8034110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 --2- 172.21.15.92:0/156500493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80341094f0 0x7f8034109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 --2- 172.21.15.92:0/156500493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8034103bb0 0x7f8034103fc0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 -- 172.21.15.92:0/156500493 >> 172.21.15.92:0/156500493 conn(0x7f80340ff600 msgr2=0x7f8034101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 -- 172.21.15.92:0/156500493 shutdown_connections 2024-09-21T02:14:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 -- 172.21.15.92:0/156500493 wait complete. 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 Processor -- start 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.835+0000 7f803ac37640 1 -- start start 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f803ac37640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8034103bb0 0x7f803419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f803ac37640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 0x7f803419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f803ac37640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8034109e40 0x7f803419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f803ac37640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8034113b00 con 0x7f80341094f0 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f803ac37640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8034113980 con 0x7f8034103bb0 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f803ac37640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8034113c80 con 0x7f8034109e40 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 0x7f803419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 0x7f803419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57672/0 (socket says 172.21.15.92:57672) 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 -- 172.21.15.92:0/3466204729 learned_addr learned my addr 172.21.15.92:0/3466204729 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f80391d4640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8034109e40 0x7f803419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f80389d3640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8034103bb0 0x7f803419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 -- 172.21.15.92:0/3466204729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8034109e40 msgr2=0x7f803419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8034109e40 0x7f803419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 -- 172.21.15.92:0/3466204729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8034103bb0 msgr2=0x7f803419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8034103bb0 0x7f803419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.836+0000 7f8033fff640 1 -- 172.21.15.92:0/3466204729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80341a0440 con 0x7f80341094f0 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.837+0000 7f8033fff640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 0x7f803419aa70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f802800d750 tx=0x7f802800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.837+0000 7f8031ffb640 1 -- 172.21.15.92:0/3466204729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028017070 con 0x7f80341094f0 2024-09-21T02:14:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.837+0000 7f8031ffb640 1 -- 172.21.15.92:0/3466204729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80280054d0 con 0x7f80341094f0 2024-09-21T02:14:10.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.837+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80341a0730 con 0x7f80341094f0 2024-09-21T02:14:10.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.837+0000 7f8031ffb640 1 -- 172.21.15.92:0/3466204729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028004c30 con 0x7f80341094f0 2024-09-21T02:14:10.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.837+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80341a6e90 con 0x7f80341094f0 2024-09-21T02:14:10.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.839+0000 7f8031ffb640 1 -- 172.21.15.92:0/3466204729 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8028004dd0 con 0x7f80341094f0 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.839+0000 7f80411ab740 1 Processor -- start 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.839+0000 7f80411ab740 1 -- start start 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80411ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfaa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80411ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbda1b0 0x563c4bfaabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80411ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80411ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563c4bf20ea0 con 0x563c4bc03070 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80411ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c4bf102d0 con 0x563c4bbda1b0 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80411ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c4bf140e0 con 0x563c4bbe2390 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80391d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfaa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80389d3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbda1b0 0x563c4bfaabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80391d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfaa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57684/0 (socket says 172.21.15.92:57684) 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80391d4640 1 -- 172.21.15.92:0/3958474052 learned_addr learned my addr 172.21.15.92:0/3958474052 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:10.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f8033fff640 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f8031ffb640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f801c0b6510 0x7f801c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80389d3640 1 -- 172.21.15.92:0/3958474052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 msgr2=0x563c4bfab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80389d3640 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80389d3640 1 -- 172.21.15.92:0/3958474052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 msgr2=0x563c4bfaa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80389d3640 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfaa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f80389d3640 1 -- 172.21.15.92:0/3958474052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4bc6e650 con 0x563c4bbda1b0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.840+0000 7f8031ffb640 1 -- 172.21.15.92:0/3466204729 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f80280e6c90 con 0x7f80341094f0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.841+0000 7f80391d4640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f801c0b6510 0x7f801c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.841+0000 7f80389d3640 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbda1b0 0x563c4bfaabe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f802c000f80 tx=0x7f802c0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3c0cc46de60701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.841+0000 7f801affd640 1 -- 172.21.15.92:0/3958474052 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c031db0 con 0x563c4bbda1b0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.841+0000 7f801affd640 1 -- 172.21.15.92:0/3958474052 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f802c036030 con 0x563c4bbda1b0 2024-09-21T02:14:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.841+0000 7f801affd640 1 -- 172.21.15.92:0/3958474052 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004280 con 0x563c4bbda1b0 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 -- 172.21.15.92:0/3958474052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbda1b0 msgr2=0x563c4bfaabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80391d4640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f801c0b6510 0x7f801c0b8a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f802400b820 tx=0x7f80240046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbda1b0 0x563c4bfaabe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f802c000f80 tx=0x7f802c0094a0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 -- 172.21.15.92:0/3958474052 shutdown_connections 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbda1b0 0x563c4bfaabe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 --2- 172.21.15.92:0/3958474052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfaa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 -- 172.21.15.92:0/3958474052 >> 172.21.15.92:0/3958474052 conn(0x563c4bbf7460 msgr2=0x563c4bfa8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 -- 172.21.15.92:0/3958474052 shutdown_connections 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 -- 172.21.15.92:0/3958474052 wait complete. 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 -- start start 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.842+0000 7f80411ab740 1 Processor -- start 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 0x563c4bfba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563c4bd71700 con 0x563c4bbda1b0 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c4beab160 con 0x563c4bc03070 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c4bfbcfa0 con 0x563c4bbe2390 2024-09-21T02:14:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80411ab740 10 client.0 _ll_register_callbacks cb 0x563c4bec49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 0x563c4bfba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 0x563c4bfba120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57696/0 (socket says 172.21.15.92:57696) 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 -- 172.21.15.92:0/323153704 learned_addr learned my addr 172.21.15.92:0/323153704 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f8033fff640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80391d4640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 -- 172.21.15.92:0/323153704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 msgr2=0x563c4bfba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 -- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 msgr2=0x563c4bfbabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfbabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.843+0000 7f80389d3640 1 -- 172.21.15.92:0/323153704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4bed8c90 con 0x563c4bbda1b0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f80389d3640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 0x563c4bfba120 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f802c009600 tx=0x7f802c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e1491f3b8bb0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f8013fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f8013fff640 1 -- 172.21.15.92:0/323153704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c04a040 con 0x563c4bbda1b0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f8013fff640 1 -- 172.21.15.92:0/323153704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f802c031650 con 0x563c4bbda1b0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f802c036030 con 0x563c4bbda1b0 2024-09-21T02:14:10.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f8013fff640 1 -- 172.21.15.92:0/323153704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f802c031850 con 0x563c4bbda1b0 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f80411ab740 10 client.18177 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c4bfc8cb0 con 0x563c4bbda1b0 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f8013fff640 1 -- 172.21.15.92:0/323153704 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f802c036030 con 0x563c4bbda1b0 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.844+0000 7f8013fff640 1 -- 172.21.15.92:0/323153704 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f802c04d030 con 0x563c4bbda1b0 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f80411ab740 10 client.18177 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f80411ab740 10 client.18177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f80411ab740 20 client.18177 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f80411ab740 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bbc19f0 0x563c4bfcc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f80411ab740 4 client.18177 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c4bf22df0 con 0x563c4bbc19f0 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f8033fff640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bbc19f0 0x563c4bfcc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f8013fff640 10 client.18177 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:10.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.845+0000 7f8033fff640 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bbc19f0 0x563c4bfcc010 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85dd73698414adda server_cookie=2e75194dd9cced80 in_seq=0 out_seq=0 2024-09-21T02:14:10.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f8013fff640 1 -- 172.21.15.92:0/323153704 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c4bf22df0 con 0x563c4bbc19f0 2024-09-21T02:14:10.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f8013fff640 10 client.18177 handle_command_reply: tid=0 2024-09-21T02:14:10.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 client.18177 shutdown 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 msgr2=0x563c4bfba120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 0x563c4bfba120 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f802c009600 tx=0x7f802c003c20 comp rx=0 tx=0).stop 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 shutdown_connections 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfbabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbe2390 0x563c4bfba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bbc19f0 0x563c4bfcc010 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f8013fff640 0 client.18177 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 --2- 172.21.15.92:0/323153704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbda1b0 0x563c4bfba120 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.846+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 >> 172.21.15.92:0/323153704 conn(0x563c4bbf7460 msgr2=0x563c4bfac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.847+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 shutdown_connections 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.847+0000 7f80411ab740 1 -- 172.21.15.92:0/323153704 wait complete. 2024-09-21T02:14:10.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.847+0000 7f80411ab740 20 client.18177 trim_cache size 0 max 16384 2024-09-21T02:14:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 Processor -- start 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 -- start start 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfb89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 0x563c4bfb8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfb9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80341ad1e0 con 0x563c4bbe2390 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80341aed20 con 0x563c4bc03070 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80411ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80341ad8b0 con 0x563c4bbda1b0 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 0x563c4bfb8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 0x563c4bfb8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57698/0 (socket says 172.21.15.92:57698) 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 -- 172.21.15.92:0/253892168 learned_addr learned my addr 172.21.15.92:0/253892168 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80389d3640 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfb89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 -- 172.21.15.92:0/253892168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 msgr2=0x563c4bfb89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfb89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 -- 172.21.15.92:0/253892168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 msgr2=0x563c4bfb9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfb9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f8033fff640 1 -- 172.21.15.92:0/253892168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4bfab4e0 con 0x563c4bbe2390 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.860+0000 7f80389d3640 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfb89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.861+0000 7f8033fff640 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 0x563c4bfb8ee0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8028009610 tx=0x7f80280d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72a9b929fefa882 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.861+0000 7f801a7fc640 1 -- 172.21.15.92:0/253892168 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028020040 con 0x563c4bbe2390 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.861+0000 7f801a7fc640 1 -- 172.21.15.92:0/253892168 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80280ac9c0 con 0x563c4bbe2390 2024-09-21T02:14:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.861+0000 7f801a7fc640 1 -- 172.21.15.92:0/253892168 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028009c30 con 0x563c4bbe2390 2024-09-21T02:14:10.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.861+0000 7f80411ab740 1 -- 172.21.15.92:0/253892168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 msgr2=0x563c4bfb8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.861+0000 7f80411ab740 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 0x563c4bfb8ee0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8028009610 tx=0x7f80280d0040 comp rx=0 tx=0).stop 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 -- 172.21.15.92:0/253892168 shutdown_connections 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bc03070 0x563c4bfb9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bbe2390 0x563c4bfb8ee0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 --2- 172.21.15.92:0/253892168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfb89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 -- 172.21.15.92:0/253892168 >> 172.21.15.92:0/253892168 conn(0x563c4bbf7460 msgr2=0x563c4bfaffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 -- 172.21.15.92:0/253892168 shutdown_connections 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 -- 172.21.15.92:0/253892168 wait complete. 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 -- start start 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 Processor -- start 2024-09-21T02:14:10.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfc2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.862+0000 7f80411ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbe2390 0x563c4bfbdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80411ab740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfbe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80411ab740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80341ae650 con 0x563c4bc03070 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80411ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80341adf80 con 0x563c4bbe2390 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80411ab740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80341ab1c0 con 0x563c4bbda1b0 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80411ab740 10 client.0 _ll_register_callbacks cb 0x7f8024004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfbe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfbe500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57702/0 (socket says 172.21.15.92:57702) 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 -- 172.21.15.92:0/1393352214 learned_addr learned my addr 172.21.15.92:0/1393352214 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80389d3640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfc2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f8033fff640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbe2390 0x563c4bfbdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 -- 172.21.15.92:0/1393352214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 msgr2=0x563c4bfc2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfc2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 -- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbe2390 msgr2=0x563c4bfbdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbe2390 0x563c4bfbdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 -- 172.21.15.92:0/1393352214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4bfbec90 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80391d4640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfbe500 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8024003fc0 tx=0x7f802400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f55c485527d4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f8013fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f8013fff640 1 -- 172.21.15.92:0/1393352214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024029440 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f8013fff640 1 -- 172.21.15.92:0/1393352214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f802401f420 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.863+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80280ac9c0 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f8013fff640 1 -- 172.21.15.92:0/1393352214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f802401f620 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 10 client.18183 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c4c036730 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f8013fff640 1 -- 172.21.15.92:0/1393352214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80280ac9c0 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f8013fff640 1 -- 172.21.15.92:0/1393352214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8024027020 con 0x563c4bc03070 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 10 client.18183 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 10 client.18183 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:14:10.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 20 client.18183 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bb96da0 0x563c4c038f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 4 client.18183 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c4bf1e8a0 con 0x563c4bb96da0 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.864+0000 7f80389d3640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bb96da0 0x563c4c038f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80389d3640 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bb96da0 0x563c4c038f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f239e6d9a5e1d1f8 server_cookie=a8d70513d5c32c83 in_seq=0 out_seq=0 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f8013fff640 10 client.18183 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f8013fff640 1 -- 172.21.15.92:0/1393352214 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c4bf1e8a0 con 0x563c4bb96da0 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f8013fff640 10 client.18183 handle_command_reply: tid=0 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 client.18183 shutdown 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 msgr2=0x563c4bfbe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfbe500 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8024003fc0 tx=0x7f802400b2e0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 shutdown_connections 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x563c4bb96da0 0x563c4c038f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563c4bc03070 0x563c4bfbe500 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c4bbe2390 0x563c4bfbdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 --2- 172.21.15.92:0/1393352214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c4bbda1b0 0x563c4bfc2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 >> 172.21.15.92:0/1393352214 conn(0x563c4bbf7460 msgr2=0x563c4bfb9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f8013fff640 0 client.18183 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 shutdown_connections 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 1 -- 172.21.15.92:0/1393352214 wait complete. 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.865+0000 7f80411ab740 20 client.18183 trim_cache size 0 max 16384 2024-09-21T02:14:10.871 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f801c0b6510 msgr2=0x7f801c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f801c0b6510 0x7f801c0b8a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f802400b820 tx=0x7f80240046c0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 msgr2=0x7f803419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 0x7f803419aa70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f802800d750 tx=0x7f802800dc10 comp rx=0 tx=0).stop 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 shutdown_connections 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f801c0b6510 0x7f801c0b8a20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8034109e40 0x7f803419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80341094f0 0x7f803419aa70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 --2- 172.21.15.92:0/3466204729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8034103bb0 0x7f803419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 >> 172.21.15.92:0/3466204729 conn(0x7f80340ff600 msgr2=0x7f8034101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 shutdown_connections 2024-09-21T02:14:10.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:10.866+0000 7f803ac37640 1 -- 172.21.15.92:0/3466204729 wait complete. 2024-09-21T02:14:11.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:11 smithi136 ceph-mon[34754]: pgmap v636: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:14:11.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:11 smithi092 ceph-mon[31190]: pgmap v636: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:14:11.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:11 smithi099 ceph-mon[34404]: pgmap v636: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:14:11.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:14:11.785 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 -- 172.21.15.92:0/2699189893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 msgr2=0x7f57c0104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 --2- 172.21.15.92:0/2699189893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c0104aa0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009900 tx=0x7f57a802a410 comp rx=0 tx=0).stop 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 -- 172.21.15.92:0/2699189893 shutdown_connections 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 --2- 172.21.15.92:0/2699189893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c010dc40 0x7f57c0110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 --2- 172.21.15.92:0/2699189893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c01050c0 0x7f57c010d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 --2- 172.21.15.92:0/2699189893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c0104aa0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 -- 172.21.15.92:0/2699189893 >> 172.21.15.92:0/2699189893 conn(0x7f57c00fe2c0 msgr2=0x7f57c0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 -- 172.21.15.92:0/2699189893 shutdown_connections 2024-09-21T02:14:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.303+0000 7f57c49fe640 1 -- 172.21.15.92:0/2699189893 wait complete. 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 Processor -- start 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 -- start start 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c01050c0 0x7f57c019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c010dc40 0x7f57c019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01138e0 con 0x7f57c0104690 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0113760 con 0x7f57c010dc40 2024-09-21T02:14:12.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57c49fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57c0113a60 con 0x7f57c01050c0 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57728/0 (socket says 172.21.15.92:57728) 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57bed9d640 1 -- 172.21.15.92:0/402369195 learned_addr learned my addr 172.21.15.92:0/402369195 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57bf59e640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c010dc40 0x7f57c019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.304+0000 7f57be59c640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c01050c0 0x7f57c019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bed9d640 1 -- 172.21.15.92:0/402369195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c01050c0 msgr2=0x7f57c019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bed9d640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c01050c0 0x7f57c019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bed9d640 1 -- 172.21.15.92:0/402369195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c010dc40 msgr2=0x7f57c019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bed9d640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c010dc40 0x7f57c019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bed9d640 1 -- 172.21.15.92:0/402369195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c01a01a0 con 0x7f57c0104690 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bf59e640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c010dc40 0x7f57c019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57bed9d640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c019a2f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f57a8006010 tx=0x7f57a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57b7fff640 1 -- 172.21.15.92:0/402369195 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a803f030 con 0x7f57c0104690 2024-09-21T02:14:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c01a0430 con 0x7f57c0104690 2024-09-21T02:14:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57b7fff640 1 -- 172.21.15.92:0/402369195 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a8003800 con 0x7f57c0104690 2024-09-21T02:14:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57b7fff640 1 -- 172.21.15.92:0/402369195 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8035070 con 0x7f57c0104690 2024-09-21T02:14:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.305+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c01a6cb0 con 0x7f57c0104690 2024-09-21T02:14:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57b7fff640 1 -- 172.21.15.92:0/402369195 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f57a80323a0 con 0x7f57c0104690 2024-09-21T02:14:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 Processor -- start 2024-09-21T02:14:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 -- start start 2024-09-21T02:14:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a51460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a51ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5588a89c7ea0 con 0x5588a86811b0 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588a89b72d0 con 0x5588a86aa070 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57cb773740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588a89bb0e0 con 0x5588a8689390 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.308+0000 7f57bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57742/0 (socket says 172.21.15.92:57742) 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 -- 172.21.15.92:0/2262510315 learned_addr learned my addr 172.21.15.92:0/2262510315 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bf59e640 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a51460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57be59c640 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a51ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57b7fff640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57ac0b6540 0x7f57ac0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 -- 172.21.15.92:0/2262510315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 msgr2=0x5588a8a51ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a51ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 -- 172.21.15.92:0/2262510315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 msgr2=0x5588a8a51460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a51460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 -- 172.21.15.92:0/2262510315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a8715650 con 0x5588a86811b0 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bf59e640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57ac0b6540 0x7f57ac0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a519a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f57a808d8a0 tx=0x7f57a80fe610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750e6d19b80197d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57b7fff640 1 -- 172.21.15.92:0/402369195 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f57a8107a00 con 0x7f57c0104690 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.309+0000 7f57b57fa640 1 -- 172.21.15.92:0/2262510315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a80cd4c0 con 0x5588a86811b0 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.310+0000 7f57bf59e640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57ac0b6540 0x7f57ac0b8a50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f57b800a5a0 tx=0x7f57b800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.310+0000 7f57b57fa640 1 -- 172.21.15.92:0/2262510315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a80cd680 con 0x5588a86811b0 2024-09-21T02:14:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.310+0000 7f57b57fa640 1 -- 172.21.15.92:0/2262510315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8003cb0 con 0x5588a86811b0 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.310+0000 7f57cb773740 1 -- 172.21.15.92:0/2262510315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 msgr2=0x5588a8a519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.310+0000 7f57cb773740 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a519a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f57a808d8a0 tx=0x7f57a80fe610 comp rx=0 tx=0).stop 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- 172.21.15.92:0/2262510315 shutdown_connections 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a51ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a519a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 --2- 172.21.15.92:0/2262510315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a51460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- 172.21.15.92:0/2262510315 >> 172.21.15.92:0/2262510315 conn(0x5588a869e460 msgr2=0x5588a8a4f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- 172.21.15.92:0/2262510315 shutdown_connections 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- 172.21.15.92:0/2262510315 wait complete. 2024-09-21T02:14:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- start start 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 Processor -- start 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a5cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a5d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 0x5588a8a6cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5588a8818700 con 0x5588a86aa070 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588a8952160 con 0x5588a86811b0 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588a8a63e10 con 0x5588a8689390 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.311+0000 7f57cb773740 10 client.0 _ll_register_callbacks cb 0x5588a896b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 0x5588a8a6cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 0x5588a8a6cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57756/0 (socket says 172.21.15.92:57756) 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 -- 172.21.15.92:0/2627451777 learned_addr learned my addr 172.21.15.92:0/2627451777 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57be59c640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a5cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 -- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 msgr2=0x5588a8a5d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a5d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 -- 172.21.15.92:0/2627451777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 msgr2=0x5588a8a5cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a5cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 -- 172.21.15.92:0/2627451777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a897fc90 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a5cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57bf59e640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 0x5588a8a6cff0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f57b8018ac0 tx=0x7f57b8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bd1d3af8a04f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57a67fc640 1 -- 172.21.15.92:0/2627451777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8007950 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57a67fc640 1 -- 172.21.15.92:0/2627451777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f57b8007af0 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.312+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a80cd680 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57a67fc640 1 -- 172.21.15.92:0/2627451777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57b8007cf0 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 10 client.18195 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588a8a6fa30 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57a67fc640 1 -- 172.21.15.92:0/2627451777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a80cd680 con 0x5588a86aa070 2024-09-21T02:14:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57a67fc640 1 -- 172.21.15.92:0/2627451777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f57b8023480 con 0x5588a86aa070 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 10 client.18195 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 20 client.18195 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a86689f0 0x5588a8a72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 4 client.18195 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a89c9df0 con 0x5588a86689f0 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.313+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a86689f0 0x5588a8a72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.314+0000 7f57a67fc640 10 client.18195 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:12.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.314+0000 7f57bed9d640 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a86689f0 0x5588a8a72fe0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325bedaf1933f88a server_cookie=f9c4525aa8e831a4 in_seq=0 out_seq=0 2024-09-21T02:14:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57a67fc640 1 -- 172.21.15.92:0/2627451777 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588a89c9df0 con 0x5588a86689f0 2024-09-21T02:14:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57a67fc640 10 client.18195 handle_command_reply: tid=0 2024-09-21T02:14:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 client.18195 shutdown 2024-09-21T02:14:12.319 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 msgr2=0x5588a8a6cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 0x5588a8a6cff0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f57b8018ac0 tx=0x7f57b8009850 comp rx=0 tx=0).stop 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 shutdown_connections 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a86689f0 0x5588a8a72fe0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57a67fc640 0 client.18195 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86aa070 0x5588a8a6cff0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a5d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 --2- 172.21.15.92:0/2627451777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a5cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 >> 172.21.15.92:0/2627451777 conn(0x5588a869e460 msgr2=0x5588a8a52490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 shutdown_connections 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 1 -- 172.21.15.92:0/2627451777 wait complete. 2024-09-21T02:14:12.320 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.315+0000 7f57cb773740 20 client.18195 trim_cache size 0 max 16384 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 Processor -- start 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 -- start start 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a5e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a6cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a59d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01ad120 con 0x5588a86811b0 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01aec60 con 0x5588a86aa070 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57cb773740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01ad7f0 con 0x5588a8689390 2024-09-21T02:14:12.330 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.328+0000 7f57bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a5e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a5e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57768/0 (socket says 172.21.15.92:57768) 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 -- 172.21.15.92:0/478374185 learned_addr learned my addr 172.21.15.92:0/478374185 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57be59c640 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a6cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bf59e640 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a59d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 -- 172.21.15.92:0/478374185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 msgr2=0x5588a8a6cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a6cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 -- 172.21.15.92:0/478374185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 msgr2=0x5588a8a59d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a59d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 -- 172.21.15.92:0/478374185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a8a4fc40 con 0x5588a86811b0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bf59e640 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a59d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57bed9d640 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a5e770 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f57a808f950 tx=0x7f57a810b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330e4b7912c58e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57b4ff9640 1 -- 172.21.15.92:0/478374185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8049040 con 0x5588a86811b0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57b4ff9640 1 -- 172.21.15.92:0/478374185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a80cd680 con 0x5588a86811b0 2024-09-21T02:14:12.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.329+0000 7f57b4ff9640 1 -- 172.21.15.92:0/478374185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a808f7b0 con 0x5588a86811b0 2024-09-21T02:14:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 -- 172.21.15.92:0/478374185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 msgr2=0x5588a8a5e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a5e770 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f57a808f950 tx=0x7f57a810b430 comp rx=0 tx=0).stop 2024-09-21T02:14:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 -- 172.21.15.92:0/478374185 shutdown_connections 2024-09-21T02:14:12.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86aa070 0x5588a8a59d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a8689390 0x5588a8a6cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 --2- 172.21.15.92:0/478374185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a86811b0 0x5588a8a5e770 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 -- 172.21.15.92:0/478374185 >> 172.21.15.92:0/478374185 conn(0x5588a869e460 msgr2=0x5588a8a5e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 -- 172.21.15.92:0/478374185 shutdown_connections 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 -- 172.21.15.92:0/478374185 wait complete. 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.330+0000 7f57cb773740 1 -- start start 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 Processor -- start 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a65190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 0x5588a8a656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a86aa070 0x5588a8a65c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01ae590 con 0x5588a8689390 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01adec0 con 0x5588a86811b0 2024-09-21T02:14:12.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57c01ab100 con 0x5588a86aa070 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57cb773740 10 client.0 _ll_register_callbacks cb 0x7f57b8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 0x5588a8a656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 0x5588a8a656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57780/0 (socket says 172.21.15.92:57780) 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 -- 172.21.15.92:0/1989028315 learned_addr learned my addr 172.21.15.92:0/1989028315 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57bed9d640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a65190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57bf59e640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a86aa070 0x5588a8a65c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 -- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a86aa070 msgr2=0x5588a8a65c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a86aa070 0x5588a8a65c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 -- 172.21.15.92:0/1989028315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 msgr2=0x5588a8a65190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a65190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 -- 172.21.15.92:0/1989028315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a8adcba0 con 0x5588a8689390 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57bf59e640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a86aa070 0x5588a8a65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57bed9d640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a65190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57be59c640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 0x5588a8a656d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f57b00038a0 tx=0x7f57b0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16b6d6bc20faefe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.331+0000 7f57a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57a67fc640 1 -- 172.21.15.92:0/1989028315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0010b10 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57a67fc640 1 -- 172.21.15.92:0/1989028315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f57b0010cb0 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a80cd680 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57a67fc640 1 -- 172.21.15.92:0/1989028315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57b0005020 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 10 client.18201 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588a8adde00 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57a67fc640 1 -- 172.21.15.92:0/1989028315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a80cd680 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57a67fc640 1 -- 172.21.15.92:0/1989028315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f57b001a070 con 0x5588a8689390 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 10 client.18201 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 20 client.18201 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a863dda0 0x5588a8ae0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 4 client.18201 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.332+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588a89c58a0 con 0x5588a863dda0 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57bed9d640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a863dda0 0x5588a8ae0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57a67fc640 10 client.18201 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57bed9d640 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a863dda0 0x5588a8ae0670 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c25b5500eb0104d server_cookie=ae77cf863dd64ee2 in_seq=0 out_seq=0 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57a67fc640 1 -- 172.21.15.92:0/1989028315 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588a89c58a0 con 0x5588a863dda0 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57a67fc640 10 client.18201 handle_command_reply: tid=0 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 client.18201 shutdown 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 msgr2=0x5588a8a656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 0x5588a8a656d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f57b00038a0 tx=0x7f57b0003d60 comp rx=0 tx=0).stop 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 shutdown_connections 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x5588a863dda0 0x5588a8ae0670 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a86aa070 0x5588a8a65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5588a8689390 0x5588a8a656d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 --2- 172.21.15.92:0/1989028315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a86811b0 0x5588a8a65190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 >> 172.21.15.92:0/1989028315 conn(0x5588a869e460 msgr2=0x5588a8a4f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.333+0000 7f57a67fc640 0 client.18201 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 shutdown_connections 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57cb773740 1 -- 172.21.15.92:0/1989028315 wait complete. 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57cb773740 20 client.18201 trim_cache size 0 max 16384 2024-09-21T02:14:12.340 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57ac0b6540 msgr2=0x7f57ac0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57ac0b6540 0x7f57ac0b8a50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f57b800a5a0 tx=0x7f57b800e040 comp rx=0 tx=0).stop 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 msgr2=0x7f57c019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c019a2f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f57a8006010 tx=0x7f57a80042f0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 shutdown_connections 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f57ac0b6540 0x7f57ac0b8a50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57c010dc40 0x7f57c019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57c01050c0 0x7f57c019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 --2- 172.21.15.92:0/402369195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0104690 0x7f57c019a2f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.334+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 >> 172.21.15.92:0/402369195 conn(0x7f57c00fe2c0 msgr2=0x7f57c01077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.335+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 shutdown_connections 2024-09-21T02:14:12.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:12.335+0000 7f57c49fe640 1 -- 172.21.15.92:0/402369195 wait complete. 2024-09-21T02:14:13.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:13 smithi136 ceph-mon[34754]: pgmap v637: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:13.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:13 smithi092 ceph-mon[31190]: pgmap v637: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:13.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:13 smithi099 ceph-mon[34404]: pgmap v637: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:14.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:14:14.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:14:14.267 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:14.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 -- 172.21.15.92:0/2806040796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88006b2d0 msgr2=0x7fd8801022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 --2- 172.21.15.92:0/2806040796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88006b2d0 0x7fd8801022a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002a410 comp rx=0 tx=0).stop 2024-09-21T02:14:14.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 -- 172.21.15.92:0/2806040796 shutdown_connections 2024-09-21T02:14:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 --2- 172.21.15.92:0/2806040796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102bf0 0x7fd880108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 --2- 172.21.15.92:0/2806040796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8801027e0 0x7fd880107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 --2- 172.21.15.92:0/2806040796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd88006b2d0 0x7fd8801022a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.770+0000 7fd884d2b640 1 -- 172.21.15.92:0/2806040796 >> 172.21.15.92:0/2806040796 conn(0x7fd880079ec0 msgr2=0x7fd88007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.771+0000 7fd884d2b640 1 -- 172.21.15.92:0/2806040796 shutdown_connections 2024-09-21T02:14:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.771+0000 7fd884d2b640 1 -- 172.21.15.92:0/2806040796 wait complete. 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 Processor -- start 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 -- start start 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88006b2d0 0x7fd88019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 0x7fd88019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102bf0 0x7fd88019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd880112a60 con 0x7fd8801027e0 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801128e0 con 0x7fd88006b2d0 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd884d2b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd880112be0 con 0x7fd880102bf0 2024-09-21T02:14:14.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 0x7fd88019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 0x7fd88019b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34350/0 (socket says 172.21.15.92:34350) 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd875d9b640 1 -- 172.21.15.92:0/998043562 learned_addr learned my addr 172.21.15.92:0/998043562 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd87e59c640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88006b2d0 0x7fd88019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.772+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102bf0 0x7fd88019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd875d9b640 1 -- 172.21.15.92:0/998043562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102bf0 msgr2=0x7fd88019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd875d9b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102bf0 0x7fd88019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd875d9b640 1 -- 172.21.15.92:0/998043562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88006b2d0 msgr2=0x7fd88019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd875d9b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88006b2d0 0x7fd88019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd875d9b640 1 -- 172.21.15.92:0/998043562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801a65a0 con 0x7fd8801027e0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd875d9b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 0x7fd88019b850 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d740 tx=0x7fd86800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd877fff640 1 -- 172.21.15.92:0/998043562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017070 con 0x7fd8801027e0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801a6890 con 0x7fd8801027e0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd877fff640 1 -- 172.21.15.92:0/998043562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86800de50 con 0x7fd8801027e0 2024-09-21T02:14:14.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd877fff640 1 -- 172.21.15.92:0/998043562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86800f620 con 0x7fd8801027e0 2024-09-21T02:14:14.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.773+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a6e20 con 0x7fd8801027e0 2024-09-21T02:14:14.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd877fff640 1 -- 172.21.15.92:0/998043562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd86801a070 con 0x7fd8801027e0 2024-09-21T02:14:14.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 Processor -- start 2024-09-21T02:14:14.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 -- start start 2024-09-21T02:14:14.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc0f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d83f490 0x55853dc0f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc0feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55853db85e60 con 0x55853d847670 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55853db75290 con 0x55853d83f490 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd88b29f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55853db790a0 con 0x55853d868350 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.776+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc0feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc0feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34362/0 (socket says 172.21.15.92:34362) 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 -- 172.21.15.92:0/1029252097 learned_addr learned my addr 172.21.15.92:0/1029252097 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd877fff640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd86c0b6540 0x7fd86c0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc0f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd87e59c640 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d83f490 0x55853dc0f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd86c0b6540 0x7fd86c0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd877fff640 1 -- 172.21.15.92:0/998043562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd8680e6d40 con 0x7fd8801027e0 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 -- 172.21.15.92:0/1029252097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 msgr2=0x55853dc0f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc0f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 -- 172.21.15.92:0/1029252097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d83f490 msgr2=0x55853dc0f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d83f490 0x55853dc0f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 -- 172.21.15.92:0/1029252097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853d8d3930 con 0x55853d847670 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd87e59c640 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d83f490 0x55853dc0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd875d9b640 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc0feb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd8680aa120 tx=0x7fd8680a8e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b6040b74afc93e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd86c0b6540 0x7fd86c0b8a50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd878012a70 tx=0x7fd878004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.777+0000 7fd874d99640 1 -- 172.21.15.92:0/1029252097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868021070 con 0x55853d847670 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd874d99640 1 -- 172.21.15.92:0/1029252097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8680ac9c0 con 0x55853d847670 2024-09-21T02:14:14.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd874d99640 1 -- 172.21.15.92:0/1029252097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680942f0 con 0x55853d847670 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 -- 172.21.15.92:0/1029252097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 msgr2=0x55853dc0feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc0feb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd8680aa120 tx=0x7fd8680a8e80 comp rx=0 tx=0).stop 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 -- 172.21.15.92:0/1029252097 shutdown_connections 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc0feb0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d83f490 0x55853dc0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1029252097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc0f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.778+0000 7fd88b29f740 1 -- 172.21.15.92:0/1029252097 >> 172.21.15.92:0/1029252097 conn(0x55853d85c740 msgr2=0x55853dc0d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 -- 172.21.15.92:0/1029252097 shutdown_connections 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 -- 172.21.15.92:0/1029252097 wait complete. 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 -- start start 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 Processor -- start 2024-09-21T02:14:14.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc1afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc1b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 0x55853dc26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55853d9d6890 con 0x55853d868350 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55853db0fd40 con 0x55853d847670 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55853dc1dc10 con 0x55853d83f490 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd88b29f740 10 client.0 _ll_register_callbacks cb 0x55853db73c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.779+0000 7fd87ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 0x55853dc26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 0x55853dc26f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34368/0 (socket says 172.21.15.92:34368) 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 -- 172.21.15.92:0/3378315396 learned_addr learned my addr 172.21.15.92:0/3378315396 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87e59c640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc1afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd875d9b640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc1b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 -- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 msgr2=0x55853dc1afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc1afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 -- 172.21.15.92:0/3378315396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 msgr2=0x55853dc1b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc1b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 -- 172.21.15.92:0/3378315396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853dc27490 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 0x55853dc26f00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd878007fd0 tx=0x7fd878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e469ddf634b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd865ffb640 1 -- 172.21.15.92:0/3378315396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878024030 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd865ffb640 1 -- 172.21.15.92:0/3378315396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd87800fbd0 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8680ac9c0 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd865ffb640 1 -- 172.21.15.92:0/3378315396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd878030050 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd88b29f740 10 client.18213 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.780+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55853dc29dc0 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd865ffb640 1 -- 172.21.15.92:0/3378315396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8680ac9c0 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd865ffb640 1 -- 172.21.15.92:0/3378315396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd87801e040 con 0x55853d868350 2024-09-21T02:14:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd88b29f740 10 client.18213 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd88b29f740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd88b29f740 20 client.18213 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd88b29f740 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d826cd0 0x55853dc2d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd88b29f740 4 client.18213 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853db87db0 con 0x55853d826cd0 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.781+0000 7fd87e59c640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d826cd0 0x55853dc2d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.782+0000 7fd865ffb640 10 client.18213 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.782+0000 7fd87e59c640 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d826cd0 0x55853dc2d2a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13148efb0950acc server_cookie=4279da83920bbf4c in_seq=0 out_seq=0 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.782+0000 7fd865ffb640 1 -- 172.21.15.92:0/3378315396 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55853db87db0 con 0x55853d826cd0 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.782+0000 7fd865ffb640 10 client.18213 handle_command_reply: tid=0 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.782+0000 7fd88b29f740 1 client.18213 shutdown 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 msgr2=0x55853dc26f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 0x55853dc26f00 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd878007fd0 tx=0x7fd878004a40 comp rx=0 tx=0).stop 2024-09-21T02:14:14.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 shutdown_connections 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d826cd0 0x55853dc2d2a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d868350 0x55853dc26f00 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc1b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd865ffb640 0 client.18213 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 --2- 172.21.15.92:0/3378315396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc1afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 >> 172.21.15.92:0/3378315396 conn(0x55853d85c740 msgr2=0x55853dc125c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 shutdown_connections 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 1 -- 172.21.15.92:0/3378315396 wait complete. 2024-09-21T02:14:14.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.783+0000 7fd88b29f740 20 client.18213 trim_cache size 0 max 16384 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 Processor -- start 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 -- start start 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 0x55853dc26e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc0d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc1c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801af560 con 0x55853d83f490 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b10a0 con 0x55853d847670 2024-09-21T02:14:14.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.800+0000 7fd88b29f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801afc30 con 0x55853d868350 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 0x55853dc26e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 0x55853dc26e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34374/0 (socket says 172.21.15.92:34374) 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 -- 172.21.15.92:0/1535382196 learned_addr learned my addr 172.21.15.92:0/1535382196 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc1c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 -- 172.21.15.92:0/1535382196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 msgr2=0x55853dc1c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd875d9b640 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc0d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc1c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 -- 172.21.15.92:0/1535382196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 msgr2=0x55853dc0d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc0d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 -- 172.21.15.92:0/1535382196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853dc11cc0 con 0x55853d83f490 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc1c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd87e59c640 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 0x55853dc26e00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd87002a930 tx=0x7fd870031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8cfb21594519f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd867fff640 1 -- 172.21.15.92:0/1535382196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87004a020 con 0x55853d83f490 2024-09-21T02:14:14.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd867fff640 1 -- 172.21.15.92:0/1535382196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8700329a0 con 0x55853d83f490 2024-09-21T02:14:14.804 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.801+0000 7fd867fff640 1 -- 172.21.15.92:0/1535382196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870035040 con 0x55853d83f490 2024-09-21T02:14:14.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 -- 172.21.15.92:0/1535382196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 msgr2=0x55853dc26e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 0x55853dc26e00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd87002a930 tx=0x7fd870031400 comp rx=0 tx=0).stop 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 -- 172.21.15.92:0/1535382196 shutdown_connections 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d868350 0x55853dc1c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d847670 0x55853dc0d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 --2- 172.21.15.92:0/1535382196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d83f490 0x55853dc26e00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 -- 172.21.15.92:0/1535382196 >> 172.21.15.92:0/1535382196 conn(0x55853d85c740 msgr2=0x55853dc1c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 -- 172.21.15.92:0/1535382196 shutdown_connections 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 -- 172.21.15.92:0/1535382196 wait complete. 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 -- start start 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.802+0000 7fd88b29f740 1 Processor -- start 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d868350 0x55853dc1dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b09d0 con 0x55853d847670 2024-09-21T02:14:14.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b0300 con 0x55853d868350 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ad540 con 0x55853d83f490 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd88b29f740 10 client.0 _ll_register_callbacks cb 0x7fd878008ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc23890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34378/0 (socket says 172.21.15.92:34378) 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 -- 172.21.15.92:0/2315478010 learned_addr learned my addr 172.21.15.92:0/2315478010 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd87ed9d640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d868350 0x55853dc1dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd87e59c640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 -- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 msgr2=0x55853dc232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 -- 172.21.15.92:0/2315478010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d868350 msgr2=0x55853dc1dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d868350 0x55853dc1dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 -- 172.21.15.92:0/2315478010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853dc1e780 con 0x55853d847670 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd875d9b640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc23890 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd8680e4870 tx=0x7fd868010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6917381d451c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:14.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd865ffb640 1 -- 172.21.15.92:0/2315478010 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868021040 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.803+0000 7fd865ffb640 1 -- 172.21.15.92:0/2315478010 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd8680942f0 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55853db83860 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd865ffb640 1 -- 172.21.15.92:0/2315478010 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8680ac9c0 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 10 client.18219 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55853dc98800 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd865ffb640 1 -- 172.21.15.92:0/2315478010 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55853db83860 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd865ffb640 1 -- 172.21.15.92:0/2315478010 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8680e4ce0 con 0x55853d847670 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 10 client.18219 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 20 client.18219 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d7fc080 0x55853dc9aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 4 client.18219 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55853db80c90 con 0x55853d7fc080 2024-09-21T02:14:14.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.804+0000 7fd87e59c640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d7fc080 0x55853dc9aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.805+0000 7fd865ffb640 10 client.18219 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:14.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.805+0000 7fd87e59c640 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d7fc080 0x55853dc9aaf0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f612aed0b36750 server_cookie=c09065160d7ad05f in_seq=0 out_seq=0 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd865ffb640 1 -- 172.21.15.92:0/2315478010 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55853db80c90 con 0x55853d7fc080 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd865ffb640 10 client.18219 handle_command_reply: tid=0 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 client.18219 shutdown 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 msgr2=0x55853dc23890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc23890 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd8680e4870 tx=0x7fd868010c70 comp rx=0 tx=0).stop 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 shutdown_connections 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55853d7fc080 0x55853dc9aaf0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55853d868350 0x55853dc1dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55853d847670 0x55853dc23890 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd865ffb640 0 client.18219 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 --2- 172.21.15.92:0/2315478010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55853d83f490 0x55853dc232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 >> 172.21.15.92:0/2315478010 conn(0x55853d85c740 msgr2=0x55853dc1c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 shutdown_connections 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 1 -- 172.21.15.92:0/2315478010 wait complete. 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.807+0000 7fd88b29f740 20 client.18219 trim_cache size 0 max 16384 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a", 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:14:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd86c0b6540 msgr2=0x7fd86c0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd86c0b6540 0x7fd86c0b8a50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd878012a70 tx=0x7fd878004ab0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 msgr2=0x7fd88019b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 0x7fd88019b850 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d740 tx=0x7fd86800dc00 comp rx=0 tx=0).stop 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 shutdown_connections 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd86c0b6540 0x7fd86c0b8a50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102bf0 0x7fd88019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8801027e0 0x7fd88019b850 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 --2- 172.21.15.92:0/998043562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88006b2d0 0x7fd88019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 >> 172.21.15.92:0/998043562 conn(0x7fd880079ec0 msgr2=0x7fd880100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 shutdown_connections 2024-09-21T02:14:14.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:14.808+0000 7fd884d2b640 1 -- 172.21.15.92:0/998043562 wait complete. 2024-09-21T02:14:15.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:15 smithi136 ceph-mon[34754]: pgmap v638: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:15.304 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:15 smithi092 ceph-mon[31190]: pgmap v638: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:15.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:15 smithi099 ceph-mon[34404]: pgmap v638: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:15.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:14:15.755 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:16.153 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:16 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:14:16.153 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:14:16.153 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:16 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:14:16.153 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:16 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:14:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 -- 172.21.15.92:0/4121066044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0102a60 msgr2=0x7f12e0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 --2- 172.21.15.92:0/4121066044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0102a60 0x7f12e0102e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009900 tx=0x7f12d002a410 comp rx=0 tx=0).stop 2024-09-21T02:14:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 -- 172.21.15.92:0/4121066044 shutdown_connections 2024-09-21T02:14:16.243 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 --2- 172.21.15.92:0/4121066044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12e0108cf0 0x7f12e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 --2- 172.21.15.92:0/4121066044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e01083a0 0x7f12e01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 --2- 172.21.15.92:0/4121066044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0102a60 0x7f12e0102e70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 -- 172.21.15.92:0/4121066044 >> 172.21.15.92:0/4121066044 conn(0x7f12e0079730 msgr2=0x7f12e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 -- 172.21.15.92:0/4121066044 shutdown_connections 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.242+0000 7f12e64d6640 1 -- 172.21.15.92:0/4121066044 wait complete. 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 Processor -- start 2024-09-21T02:14:16.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 -- start start 2024-09-21T02:14:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e0102a60 0x7f12e019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12e01083a0 0x7f12e019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 0x7f12e019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0112960 con 0x7f12e0108cf0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01127e0 con 0x7f12e0102a60 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e64d6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0112ae0 con 0x7f12e01083a0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e4a73640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 0x7f12e019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e4a73640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 0x7f12e019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34402/0 (socket says 172.21.15.92:34402) 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12e4a73640 1 -- 172.21.15.92:0/3547583802 learned_addr learned my addr 172.21.15.92:0/3547583802 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.243+0000 7f12dffff640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e0102a60 0x7f12e019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12df7fe640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12e01083a0 0x7f12e019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e4a73640 1 -- 172.21.15.92:0/3547583802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12e01083a0 msgr2=0x7f12e019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e4a73640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12e01083a0 0x7f12e019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e4a73640 1 -- 172.21.15.92:0/3547583802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e0102a60 msgr2=0x7f12e019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e4a73640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e0102a60 0x7f12e019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e4a73640 1 -- 172.21.15.92:0/3547583802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e01a65f0 con 0x7f12e0108cf0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e4a73640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 0x7f12e019bdb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f12d800cad0 tx=0x7f12d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12dd7fa640 1 -- 172.21.15.92:0/3547583802 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d800ae20 con 0x7f12e0108cf0 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e01a68e0 con 0x7f12e0108cf0 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12dd7fa640 1 -- 172.21.15.92:0/3547583802 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d8005020 con 0x7f12e0108cf0 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12dd7fa640 1 -- 172.21.15.92:0/3547583802 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d800f6b0 con 0x7f12e0108cf0 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.244+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e01a6e70 con 0x7f12e0108cf0 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 Processor -- start 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12dd7fa640 1 -- 172.21.15.92:0/3547583802 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f12d80051d0 con 0x7f12e0108cf0 2024-09-21T02:14:16.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 -- start start 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 0x55dc56100650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc56100b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc561010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dc56076e50 con 0x55dc55d59070 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dc56066280 con 0x55dc55d38390 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12eca4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc5606a090 con 0x55dc55d301b0 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 0x55dc56100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 0x55dc56100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34412/0 (socket says 172.21.15.92:34412) 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12df7fe640 1 -- 172.21.15.92:0/4200743914 learned_addr learned my addr 172.21.15.92:0/4200743914 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.247+0000 7f12dffff640 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc56100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12df7fe640 1 -- 172.21.15.92:0/4200743914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 msgr2=0x55dc56100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12df7fe640 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc56100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12df7fe640 1 -- 172.21.15.92:0/4200743914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 msgr2=0x55dc561010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12df7fe640 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc561010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12df7fe640 1 -- 172.21.15.92:0/4200743914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc55dc4650 con 0x55dc55d59070 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12dffff640 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc56100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12dd7fa640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f12c80b6510 0x7f12c80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12dffff640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f12c80b6510 0x7f12c80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12df7fe640 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 0x55dc56100650 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f12d400bd00 tx=0x7f12d4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ef5e9cde9c6c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12dd7fa640 1 -- 172.21.15.92:0/3547583802 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f12d80e72a0 con 0x7f12e0108cf0 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12c67fc640 1 -- 172.21.15.92:0/4200743914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d401a400 con 0x55dc55d59070 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12c67fc640 1 -- 172.21.15.92:0/4200743914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d40128a0 con 0x55dc55d59070 2024-09-21T02:14:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.248+0000 7f12c67fc640 1 -- 172.21.15.92:0/4200743914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400f800 con 0x55dc55d59070 2024-09-21T02:14:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 -- 172.21.15.92:0/4200743914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 msgr2=0x55dc56100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 0x55dc56100650 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f12d400bd00 tx=0x7f12d4011750 comp rx=0 tx=0).stop 2024-09-21T02:14:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12dffff640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f12c80b6510 0x7f12c80b8a20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000c00 tx=0x7f12d0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 -- 172.21.15.92:0/4200743914 shutdown_connections 2024-09-21T02:14:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc561010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc56100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 --2- 172.21.15.92:0/4200743914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d59070 0x55dc56100650 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 -- 172.21.15.92:0/4200743914 >> 172.21.15.92:0/4200743914 conn(0x55dc55d4d460 msgr2=0x55dc560fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 -- 172.21.15.92:0/4200743914 shutdown_connections 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.249+0000 7f12eca4a740 1 -- 172.21.15.92:0/4200743914 wait complete. 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 -- start start 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 Processor -- start 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d301b0 0x55dc5610bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc5610c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d59070 0x55dc5610c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dc55ec7700 con 0x55dc55d38390 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dc56001110 con 0x55dc55d301b0 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc5610ee30 con 0x55dc55d59070 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12eca4a740 10 client.0 _ll_register_callbacks cb 0x55dc5601a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc5610c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc5610c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34424/0 (socket says 172.21.15.92:34424) 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12df7fe640 1 -- 172.21.15.92:0/1089010003 learned_addr learned my addr 172.21.15.92:0/1089010003 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12dffff640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d301b0 0x55dc5610bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.250+0000 7f12e4a73640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d59070 0x55dc5610c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12df7fe640 1 -- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d59070 msgr2=0x55dc5610c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12df7fe640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d59070 0x55dc5610c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12df7fe640 1 -- 172.21.15.92:0/1089010003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d301b0 msgr2=0x55dc5610bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12df7fe640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d301b0 0x55dc5610bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12df7fe640 1 -- 172.21.15.92:0/1089010003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5602ec40 con 0x55dc55d38390 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12e4a73640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d59070 0x55dc5610c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12df7fe640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc5610c1b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f12d4006010 tx=0x7f12d4012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8748ed916cdb3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12bf7fe640 1 -- 172.21.15.92:0/1089010003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d402a060 con 0x55dc55d38390 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12bf7fe640 1 -- 172.21.15.92:0/1089010003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f12d400bdf0 con 0x55dc55d38390 2024-09-21T02:14:16.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d40128a0 con 0x55dc55d38390 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12bf7fe640 1 -- 172.21.15.92:0/1089010003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12d4012c60 con 0x55dc55d38390 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12eca4a740 10 client.18231 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.251+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc5611aab0 con 0x55dc55d38390 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12bf7fe640 1 -- 172.21.15.92:0/1089010003 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d40128a0 con 0x55dc55d38390 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12bf7fe640 1 -- 172.21.15.92:0/1089010003 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f12d4004aa0 con 0x55dc55d38390 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12eca4a740 10 client.18231 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12eca4a740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12eca4a740 20 client.18231 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d179f0 0x55dc5611dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12eca4a740 4 client.18231 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc56078da0 con 0x55dc55d179f0 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.252+0000 7f12dffff640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d179f0 0x55dc5611dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.253+0000 7f12bf7fe640 10 client.18231 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:16.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.253+0000 7f12dffff640 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d179f0 0x55dc5611dec0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c3169376f25502 server_cookie=3621762fc52774b3 in_seq=0 out_seq=0 2024-09-21T02:14:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12bf7fe640 1 -- 172.21.15.92:0/1089010003 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc56078da0 con 0x55dc55d179f0 2024-09-21T02:14:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12bf7fe640 10 client.18231 handle_command_reply: tid=0 2024-09-21T02:14:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 client.18231 shutdown 2024-09-21T02:14:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 msgr2=0x55dc5610c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc5610c1b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f12d4006010 tx=0x7f12d4012620 comp rx=0 tx=0).stop 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 shutdown_connections 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d179f0 0x55dc5611dec0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d59070 0x55dc5610c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc5610c1b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12bf7fe640 0 client.18231 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1089010003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d301b0 0x55dc5610bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 >> 172.21.15.92:0/1089010003 conn(0x55dc55d4d460 msgr2=0x55dc56103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 shutdown_connections 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 1 -- 172.21.15.92:0/1089010003 wait complete. 2024-09-21T02:14:16.258 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.254+0000 7f12eca4a740 20 client.18231 trim_cache size 0 max 16384 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 Processor -- start 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 -- start start 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d179f0 0x55dc56102040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc5610d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc56108e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01af5b0 con 0x55dc55d38390 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b10f0 con 0x55dc55d179f0 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.271+0000 7f12eca4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01afc80 con 0x55dc55d301b0 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc56108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc56108e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34440/0 (socket says 172.21.15.92:34440) 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 -- 172.21.15.92:0/1887145411 learned_addr learned my addr 172.21.15.92:0/1887145411 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12dffff640 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d179f0 0x55dc56102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12df7fe640 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc5610d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 -- 172.21.15.92:0/1887145411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 msgr2=0x55dc5610d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc5610d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 -- 172.21.15.92:0/1887145411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d179f0 msgr2=0x55dc56102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d179f0 0x55dc56102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 -- 172.21.15.92:0/1887145411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5610cf30 con 0x55dc55d38390 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12dffff640 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d179f0 0x55dc56102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12e4a73640 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc56108e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f12d800caa0 tx=0x7f12d80ad060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4127d8cc3bfc93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12c5ffb640 1 -- 172.21.15.92:0/1887145411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d801d040 con 0x55dc55d38390 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12c5ffb640 1 -- 172.21.15.92:0/1887145411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d80b6e30 con 0x55dc55d38390 2024-09-21T02:14:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.272+0000 7f12c5ffb640 1 -- 172.21.15.92:0/1887145411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a8fc0 con 0x55dc55d38390 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 -- 172.21.15.92:0/1887145411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 msgr2=0x55dc56108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc56108e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f12d800caa0 tx=0x7f12d80ad060 comp rx=0 tx=0).stop 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 -- 172.21.15.92:0/1887145411 shutdown_connections 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d38390 0x55dc56108e60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d301b0 0x55dc5610d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 --2- 172.21.15.92:0/1887145411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d179f0 0x55dc56102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 -- 172.21.15.92:0/1887145411 >> 172.21.15.92:0/1887145411 conn(0x55dc55d4d460 msgr2=0x55dc5610d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 -- 172.21.15.92:0/1887145411 shutdown_connections 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.273+0000 7f12eca4a740 1 -- 172.21.15.92:0/1887145411 wait complete. 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 -- start start 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 Processor -- start 2024-09-21T02:14:16.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d179f0 0x55dc5611f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 0x55dc5611fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc56187640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b0a20 con 0x55dc55d301b0 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01b0350 con 0x55dc55d38390 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01ad590 con 0x55dc55d179f0 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12eca4a740 10 client.0 _ll_register_callbacks cb 0x7f12d400bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 0x55dc5611fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 0x55dc5611fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34456/0 (socket says 172.21.15.92:34456) 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12df7fe640 1 -- 172.21.15.92:0/3761275047 learned_addr learned my addr 172.21.15.92:0/3761275047 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12e4a73640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc56187640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.274+0000 7f12dffff640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d179f0 0x55dc5611f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12df7fe640 1 -- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d179f0 msgr2=0x55dc5611f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12df7fe640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d179f0 0x55dc5611f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12df7fe640 1 -- 172.21.15.92:0/3761275047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 msgr2=0x55dc56187640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:16.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12df7fe640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc56187640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12df7fe640 1 -- 172.21.15.92:0/3761275047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc56187c30 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12e4a73640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc56187640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12df7fe640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 0x55dc5611fbf0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f12d4011c50 tx=0x7f12d400b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a14d5e4b3a1dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12bf7fe640 1 -- 172.21.15.92:0/3761275047 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d402a040 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12bf7fe640 1 -- 172.21.15.92:0/3761275047 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f12d4004240 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d80b6e30 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12bf7fe640 1 -- 172.21.15.92:0/3761275047 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12d4004440 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12eca4a740 10 client.18237 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc56188ea0 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12bf7fe640 1 -- 172.21.15.92:0/3761275047 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d80b6e30 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.275+0000 7f12bf7fe640 1 -- 172.21.15.92:0/3761275047 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f12d40051c0 con 0x55dc55d301b0 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12eca4a740 10 client.18237 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12eca4a740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12eca4a740 20 client.18237 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:16.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12eca4a740 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d59070 0x55dc5618b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12eca4a740 4 client.18237 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:14:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc56074850 con 0x55dc55d59070 2024-09-21T02:14:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12dffff640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d59070 0x55dc5618b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12bf7fe640 10 client.18237 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:16.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.276+0000 7f12dffff640 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d59070 0x55dc5618b5b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcff4c0bc5b8ee86 server_cookie=e3b3e116de314fb4 in_seq=0 out_seq=0 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12bf7fe640 1 -- 172.21.15.92:0/3761275047 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55dc56074850 con 0x55dc55d59070 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12bf7fe640 10 client.18237 handle_command_reply: tid=0 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 client.18237 shutdown 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 msgr2=0x55dc5611fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 0x55dc5611fbf0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f12d4011c50 tx=0x7f12d400b620 comp rx=0 tx=0).stop 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 shutdown_connections 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55dc55d59070 0x55dc5618b5b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc55d38390 0x55dc56187640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dc55d301b0 0x55dc5611fbf0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 --2- 172.21.15.92:0/3761275047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc55d179f0 0x55dc5611f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 >> 172.21.15.92:0/3761275047 conn(0x55dc55d4d460 msgr2=0x55dc5610a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12bf7fe640 0 client.18237 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 shutdown_connections 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 1 -- 172.21.15.92:0/3761275047 wait complete. 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.277+0000 7f12eca4a740 20 client.18237 trim_cache size 0 max 16384 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a": { 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a", 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.282 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a": { 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:14:14.805000+0000", 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.283 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:13:30.072031+0000", 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f12c80b6510 msgr2=0x7f12c80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f12c80b6510 0x7f12c80b8a20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000c00 tx=0x7f12d0031400 comp rx=0 tx=0).stop 2024-09-21T02:14:16.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 msgr2=0x7f12e019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 0x7f12e019bdb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f12d800cad0 tx=0x7f12d800cf90 comp rx=0 tx=0).stop 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 shutdown_connections 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f12c80b6510 0x7f12c80b8a20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0108cf0 0x7f12e019bdb0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12e01083a0 0x7f12e019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 --2- 172.21.15.92:0/3547583802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e0102a60 0x7f12e019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 >> 172.21.15.92:0/3547583802 conn(0x7f12e0079730 msgr2=0x7f12e01014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 shutdown_connections 2024-09-21T02:14:16.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:16.278+0000 7f12e64d6640 1 -- 172.21.15.92:0/3547583802 wait complete. 2024-09-21T02:14:16.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:16 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:14:16.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:14:16.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:16 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:14:16.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:16 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:14:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:16 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:14:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:14:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:16 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:14:16.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:16 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:14:17.092 INFO:tasks.cephfs.filesystem:scrub status for tag:622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a - {'path': '/', 'tag': '622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a', 'options': 'recursive,force'} 2024-09-21T02:14:17.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:17 smithi099 ceph-mon[34404]: pgmap v639: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:14:17.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:17 smithi136 ceph-mon[34754]: pgmap v639: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:14:17.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:17 smithi092 ceph-mon[31190]: pgmap v639: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-09-21T02:14:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:19 smithi099 ceph-mon[34404]: pgmap v640: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:19.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:19 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:14:19.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:19 smithi136 ceph-mon[34754]: pgmap v640: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:19.534 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:19 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:14:19.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:19 smithi092 ceph-mon[31190]: pgmap v640: 97 pgs: 97 active+clean; 3.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:19.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:19 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:14:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1312091063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41034c0 msgr2=0x7f2ae4103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1312091063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41034c0 0x7f2ae4103990 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009900 tx=0x7f2ad802a410 comp rx=0 tx=0).stop 2024-09-21T02:14:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1312091063 shutdown_connections 2024-09-21T02:14:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1312091063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ae41040d0 0x7f2ae410ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1312091063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41034c0 0x7f2ae4103990 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1312091063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4102ae0 0x7f2ae4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1312091063 >> 172.21.15.92:0/1312091063 conn(0x7f2ae40fd6d0 msgr2=0x7f2ae40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1312091063 shutdown_connections 2024-09-21T02:14:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.538+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1312091063 wait complete. 2024-09-21T02:14:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 Processor -- start 2024-09-21T02:14:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 -- start start 2024-09-21T02:14:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ae4102ae0 0x7f2ae4102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae41034c0 0x7f2ae419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 0x7f2ae41a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410e210 con 0x7f2ae41040d0 2024-09-21T02:14:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410e090 con 0x7f2ae41034c0 2024-09-21T02:14:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae8e25640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410e390 con 0x7f2ae4102ae0 2024-09-21T02:14:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.539+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 0x7f2ae41a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 0x7f2ae41a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34478/0 (socket says 172.21.15.92:34478) 2024-09-21T02:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 -- 172.21.15.92:0/1924420231 learned_addr learned my addr 172.21.15.92:0/1924420231 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae259c640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ae4102ae0 0x7f2ae4102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae41034c0 0x7f2ae419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 -- 172.21.15.92:0/1924420231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ae4102ae0 msgr2=0x7f2ae4102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ae4102ae0 0x7f2ae4102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 -- 172.21.15.92:0/1924420231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae41034c0 msgr2=0x7f2ae419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae41034c0 0x7f2ae419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 -- 172.21.15.92:0/1924420231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41a19f0 con 0x7f2ae41040d0 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae2d9d640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 0x7f2ae41a11f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00eaa0 tx=0x7f2adc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ad37fe640 1 -- 172.21.15.92:0/1924420231 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00cdb0 con 0x7f2ae41040d0 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ad37fe640 1 -- 172.21.15.92:0/1924420231 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2adc005020 con 0x7f2ae41040d0 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41a1ce0 con 0x7f2ae41040d0 2024-09-21T02:14:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ad37fe640 1 -- 172.21.15.92:0/1924420231 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0116d0 con 0x7f2ae41040d0 2024-09-21T02:14:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.540+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a21d0 con 0x7f2ae41040d0 2024-09-21T02:14:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2ad37fe640 1 -- 172.21.15.92:0/1924420231 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2adc0051d0 con 0x7f2ae41040d0 2024-09-21T02:14:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 Processor -- start 2024-09-21T02:14:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 -- start start 2024-09-21T02:14:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def531b0 0x55b9df2fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df2fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df2fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9df26d6f0 con 0x55b9def324d0 2024-09-21T02:14:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b9df25cb20 con 0x55b9def2a2f0 2024-09-21T02:14:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.543+0000 7f2aefe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b9df260930 con 0x55b9def531b0 2024-09-21T02:14:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df2fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df2fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34494/0 (socket says 172.21.15.92:34494) 2024-09-21T02:14:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/211394401 learned_addr learned my addr 172.21.15.92:0/211394401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df2fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def531b0 0x55b9df2fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ad37fe640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2acc0b6590 0x7f2acc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 -- 172.21.15.92:0/211394401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def531b0 msgr2=0x55b9df2fae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def531b0 0x55b9df2fae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae2d9d640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2acc0b6590 0x7f2acc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 -- 172.21.15.92:0/211394401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 msgr2=0x55b9df2fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df2fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 -- 172.21.15.92:0/211394401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9df222530 con 0x55b9def2a2f0 2024-09-21T02:14:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ad37fe640 1 -- 172.21.15.92:0/1924420231 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2adc0e73e0 con 0x7f2ae41040d0 2024-09-21T02:14:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ae259c640 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df2fb4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400bbf0 tx=0x7f2ad40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f80c3695f38d125 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.544+0000 7f2ad0ff9640 1 -- 172.21.15.92:0/211394401 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4004030 con 0x55b9def2a2f0 2024-09-21T02:14:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2ae2d9d640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2acc0b6590 0x7f2acc0b8aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00efd0 tx=0x7f2adc07a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2ad0ff9640 1 -- 172.21.15.92:0/211394401 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad40035f0 con 0x55b9def2a2f0 2024-09-21T02:14:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2ad0ff9640 1 -- 172.21.15.92:0/211394401 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40116a0 con 0x55b9def2a2f0 2024-09-21T02:14:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 -- 172.21.15.92:0/211394401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 msgr2=0x55b9df2fb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df2fb4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400bbf0 tx=0x7f2ad40107a0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 -- 172.21.15.92:0/211394401 shutdown_connections 2024-09-21T02:14:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df2fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df2fb4c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 --2- 172.21.15.92:0/211394401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def531b0 0x55b9df2fae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.545+0000 7f2aefe73740 1 -- 172.21.15.92:0/211394401 >> 172.21.15.92:0/211394401 conn(0x55b9def47300 msgr2=0x55b9df2f68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 -- 172.21.15.92:0/211394401 shutdown_connections 2024-09-21T02:14:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 -- 172.21.15.92:0/211394401 wait complete. 2024-09-21T02:14:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 -- start start 2024-09-21T02:14:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 Processor -- start 2024-09-21T02:14:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def2a2f0 0x55b9df3045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df304c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df305800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b9df194ab0 con 0x55b9def324d0 2024-09-21T02:14:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b9df202730 con 0x55b9def531b0 2024-09-21T02:14:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b9df2fcb30 con 0x55b9def2a2f0 2024-09-21T02:14:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2aefe73740 10 client.0 _ll_register_callbacks cb 0x55b9df2566b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df304c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df304c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34502/0 (socket says 172.21.15.92:34502) 2024-09-21T02:14:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.546+0000 7f2ae259c640 1 -- 172.21.15.92:0/3554003554 learned_addr learned my addr 172.21.15.92:0/3554003554 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def2a2f0 0x55b9df3045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df305800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae259c640 1 -- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def2a2f0 msgr2=0x55b9df3045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae259c640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def2a2f0 0x55b9df3045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae259c640 1 -- 172.21.15.92:0/3554003554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 msgr2=0x55b9df305800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae259c640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df305800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae259c640 1 -- 172.21.15.92:0/3554003554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9defbe790 con 0x55b9def324d0 2024-09-21T02:14:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df305800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ae259c640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df304c90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4003fc0 tx=0x7f2ad4003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aefa3a70a04729d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/3554003554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad402a060 con 0x55b9def324d0 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/3554003554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2ad4004420 con 0x55b9def324d0 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad40035f0 con 0x55b9def324d0 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/3554003554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ad40037a0 con 0x55b9def324d0 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2aefe73740 10 client.18249 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9df037620 con 0x55b9def324d0 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.547+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/3554003554 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad40035f0 con 0x55b9def324d0 2024-09-21T02:14:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/3554003554 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2ad40047e0 con 0x55b9def324d0 2024-09-21T02:14:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2aefe73740 10 client.18249 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2aefe73740 10 client.18249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2aefe73740 20 client.18249 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2aefe73740 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9def11d80 0x55b9df3119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2aefe73740 4 client.18249 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9df26f640 con 0x55b9def11d80 2024-09-21T02:14:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9def11d80 0x55b9df3119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2ac9ffb640 10 client.18249 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.548+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9def11d80 0x55b9df3119f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e6dce837c895e4 server_cookie=f5ca48328f6e845a in_seq=0 out_seq=0 2024-09-21T02:14:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/3554003554 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9df26f640 con 0x55b9def11d80 2024-09-21T02:14:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2ac9ffb640 10 client.18249 handle_command_reply: tid=0 2024-09-21T02:14:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 client.18249 shutdown 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 msgr2=0x55b9df304c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df304c90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4003fc0 tx=0x7f2ad4003ff0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 shutdown_connections 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9def11d80 0x55b9df3119f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df305800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2ac9ffb640 0 client.18249 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def324d0 0x55b9df304c90 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 --2- 172.21.15.92:0/3554003554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def2a2f0 0x55b9df3045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 >> 172.21.15.92:0/3554003554 conn(0x55b9def47300 msgr2=0x55b9df2f4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 shutdown_connections 2024-09-21T02:14:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.549+0000 7f2aefe73740 1 -- 172.21.15.92:0/3554003554 wait complete. 2024-09-21T02:14:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.550+0000 7f2aefe73740 20 client.18249 trim_cache size 0 max 16384 2024-09-21T02:14:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 Processor -- start 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 -- start start 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 0x55b9df2fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df2fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df302ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ab0c0 con 0x55b9def2a2f0 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41acc00 con 0x55b9def531b0 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2aefe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ab790 con 0x55b9def324d0 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2ae1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 0x55b9df2fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2ae1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 0x55b9df2fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34508/0 (socket says 172.21.15.92:34508) 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.564+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/409749821 learned_addr learned my addr 172.21.15.92:0/409749821 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae259c640 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df2fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/409749821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 msgr2=0x55b9df2fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df2fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/409749821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 msgr2=0x55b9df302ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df302ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/409749821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9df302760 con 0x55b9def2a2f0 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 0x55b9df2fadf0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8005ab0 tx=0x7f2ad803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91488ce81f4023a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2acbfff640 1 -- 172.21.15.92:0/409749821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8035040 con 0x55b9def2a2f0 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2acbfff640 1 -- 172.21.15.92:0/409749821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad80044d0 con 0x55b9def2a2f0 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.565+0000 7f2acbfff640 1 -- 172.21.15.92:0/409749821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8003970 con 0x55b9def2a2f0 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- 172.21.15.92:0/409749821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 msgr2=0x55b9df2fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 0x55b9df2fadf0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8005ab0 tx=0x7f2ad803f040 comp rx=0 tx=0).stop 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- 172.21.15.92:0/409749821 shutdown_connections 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def531b0 0x55b9df302ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df2fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- 172.21.15.92:0/409749821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def2a2f0 0x55b9df2fadf0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- 172.21.15.92:0/409749821 >> 172.21.15.92:0/409749821 conn(0x55b9def47300 msgr2=0x55b9df3037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- 172.21.15.92:0/409749821 shutdown_connections 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- 172.21.15.92:0/409749821 wait complete. 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- start start 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 Processor -- start 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df377630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df377cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 0x55b9df378860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ac530 con 0x55b9def531b0 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41abe60 con 0x55b9def2a2f0 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41a90a0 con 0x55b9def324d0 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2aefe73740 10 client.0 _ll_register_callbacks cb 0x7f2ad8005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2ae1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 0x55b9df378860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2ae1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 0x55b9df378860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34520/0 (socket says 172.21.15.92:34520) 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.566+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/2109752918 learned_addr learned my addr 172.21.15.92:0/2109752918 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 msgr2=0x55b9df377cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae259c640 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df377cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df377cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/2109752918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 msgr2=0x55b9df377630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df377630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae1d9b640 1 -- 172.21.15.92:0/2109752918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9df378fc0 con 0x55b9def531b0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ae1d9b640 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 0x55b9df378860 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009570 tx=0x7f2ad8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8038ca982053608 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/2109752918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8062060 con 0x55b9def531b0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/2109752918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2ad8031b50 con 0x55b9def531b0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad80044d0 con 0x55b9def531b0 2024-09-21T02:14:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/2109752918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ad8035030 con 0x55b9def531b0 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2aefe73740 10 client.18255 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9df380320 con 0x55b9def531b0 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.567+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/2109752918 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad80044d0 con 0x55b9def531b0 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/2109752918 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2ad805c030 con 0x55b9def531b0 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2aefe73740 10 client.18255 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2aefe73740 10 client.18255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2aefe73740 20 client.18255 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2aefe73740 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9deee71c0 0x55b9df382cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2aefe73740 4 client.18255 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9df2fc420 con 0x55b9deee71c0 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2ae259c640 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9deee71c0 0x55b9df382cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2ac9ffb640 10 client.18255 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.568+0000 7f2ae259c640 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9deee71c0 0x55b9df382cd0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ff0bcae6c5262 server_cookie=5b6426d1aad2130c in_seq=0 out_seq=0 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.582+0000 7f2ac9ffb640 1 -- 172.21.15.92:0/2109752918 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9df2fc420 con 0x55b9deee71c0 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.582+0000 7f2ac9ffb640 10 client.18255 handle_command_reply: tid=0 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.582+0000 7f2aefe73740 1 client.18255 shutdown 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 msgr2=0x55b9df378860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 0x55b9df378860 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009570 tx=0x7f2ad8000bc0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 shutdown_connections 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b9deee71c0 0x55b9df382cd0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9def531b0 0x55b9df378860 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ac9ffb640 0 client.18255 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b9def324d0 0x55b9df377cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 --2- 172.21.15.92:0/2109752918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b9def2a2f0 0x55b9df377630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 >> 172.21.15.92:0/2109752918 conn(0x55b9def47300 msgr2=0x55b9df2f55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 shutdown_connections 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 1 -- 172.21.15.92:0/2109752918 wait complete. 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2aefe73740 20 client.18255 trim_cache size 0 max 16384 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:14:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2acc0b6590 msgr2=0x7f2acc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2acc0b6590 0x7f2acc0b8aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00efd0 tx=0x7f2adc07a8d0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 msgr2=0x7f2ae41a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 0x7f2ae41a11f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00eaa0 tx=0x7f2adc00ef60 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 shutdown_connections 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2acc0b6590 0x7f2acc0b8aa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ae41040d0 0x7f2ae41a11f0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae41034c0 0x7f2ae419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 --2- 172.21.15.92:0/1924420231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ae4102ae0 0x7f2ae4102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.583+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 >> 172.21.15.92:0/1924420231 conn(0x7f2ae40fd6d0 msgr2=0x7f2ae4106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.584+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 shutdown_connections 2024-09-21T02:14:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:20.584+0000 7f2ae8e25640 1 -- 172.21.15.92:0/1924420231 wait complete. 2024-09-21T02:14:21.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:21 smithi099 ceph-mon[34404]: pgmap v641: 97 pgs: 97 active+clean; 3.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:14:21.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:21 smithi136 ceph-mon[34754]: pgmap v641: 97 pgs: 97 active+clean; 3.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:14:21.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:21 smithi092 ceph-mon[31190]: pgmap v641: 97 pgs: 97 active+clean; 3.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:14:23.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:23 smithi099 ceph-mon[34404]: pgmap v642: 97 pgs: 97 active+clean; 3.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:14:23.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:23 smithi136 ceph-mon[34754]: pgmap v642: 97 pgs: 97 active+clean; 3.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:14:23.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:23 smithi092 ceph-mon[31190]: pgmap v642: 97 pgs: 97 active+clean; 3.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:14:25.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:25 smithi099 ceph-mon[34404]: pgmap v643: 97 pgs: 97 active+clean; 3.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-09-21T02:14:25.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:25 smithi136 ceph-mon[34754]: pgmap v643: 97 pgs: 97 active+clean; 3.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-09-21T02:14:25.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:25 smithi092 ceph-mon[31190]: pgmap v643: 97 pgs: 97 active+clean; 3.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-09-21T02:14:27.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:27 smithi099 ceph-mon[34404]: pgmap v644: 97 pgs: 97 active+clean; 3.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-09-21T02:14:27.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:27 smithi136 ceph-mon[34754]: pgmap v644: 97 pgs: 97 active+clean; 3.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-09-21T02:14:27.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:27 smithi092 ceph-mon[31190]: pgmap v644: 97 pgs: 97 active+clean; 3.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-09-21T02:14:29.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:29 smithi099 ceph-mon[34404]: pgmap v645: 97 pgs: 97 active+clean; 3.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:14:29.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:29 smithi136 ceph-mon[34754]: pgmap v645: 97 pgs: 97 active+clean; 3.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:14:29.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:29 smithi092 ceph-mon[31190]: pgmap v645: 97 pgs: 97 active+clean; 3.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:14:31.331 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:31 smithi099 ceph-mon[34404]: pgmap v646: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-09-21T02:14:31.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:31 smithi136 ceph-mon[34754]: pgmap v646: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-09-21T02:14:31.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:31 smithi092 ceph-mon[31190]: pgmap v646: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-09-21T02:14:33.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:33 smithi136 ceph-mon[34754]: pgmap v647: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:33.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:33 smithi092 ceph-mon[31190]: pgmap v647: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:33.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:33 smithi099 ceph-mon[34404]: pgmap v647: 97 pgs: 97 active+clean; 3.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 -- 172.21.15.92:0/1931367503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 msgr2=0x7fdc0c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 --2- 172.21.15.92:0/1931367503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c103a50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009900 tx=0x7fdbfc02a410 comp rx=0 tx=0).stop 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 -- 172.21.15.92:0/1931367503 shutdown_connections 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 --2- 172.21.15.92:0/1931367503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0c103fe0 0x7fdc0c10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 --2- 172.21.15.92:0/1931367503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c103a50 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 --2- 172.21.15.92:0/1931367503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0c1025c0 0x7fdc0c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.689+0000 7fdc13470640 1 -- 172.21.15.92:0/1931367503 >> 172.21.15.92:0/1931367503 conn(0x7fdc0c0fdf00 msgr2=0x7fdc0c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 -- 172.21.15.92:0/1931367503 shutdown_connections 2024-09-21T02:14:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 -- 172.21.15.92:0/1931367503 wait complete. 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 Processor -- start 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 -- start start 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0c1025c0 0x7fdc0c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0c103fe0 0x7fdc0c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10e210 con 0x7fdc0c103640 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10e090 con 0x7fdc0c103fe0 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.690+0000 7fdc13470640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10e390 con 0x7fdc0c1025c0 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59040/0 (socket says 172.21.15.92:59040) 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 -- 172.21.15.92:0/3967905267 learned_addr learned my addr 172.21.15.92:0/3967905267 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc11a0d640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0c103fe0 0x7fdc0c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0c1025c0 0x7fdc0c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 -- 172.21.15.92:0/3967905267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0c1025c0 msgr2=0x7fdc0c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0c1025c0 0x7fdc0c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 -- 172.21.15.92:0/3967905267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0c103fe0 msgr2=0x7fdc0c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0c103fe0 0x7fdc0c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 -- 172.21.15.92:0/3967905267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c19a8d0 con 0x7fdc0c103640 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c199500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006010 tx=0x7fdbfc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc027fc640 1 -- 172.21.15.92:0/3967905267 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc03f030 con 0x7fdc0c103640 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc027fc640 1 -- 172.21.15.92:0/3967905267 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbfc003cb0 con 0x7fdc0c103640 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c19ab60 con 0x7fdc0c103640 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc027fc640 1 -- 172.21.15.92:0/3967905267 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc035070 con 0x7fdc0c103640 2024-09-21T02:14:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.691+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c19ae70 con 0x7fdc0c103640 2024-09-21T02:14:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 Processor -- start 2024-09-21T02:14:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 -- start start 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7839060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc783a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55adc77a96f0 con 0x55adc746e4d0 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55adc7798b20 con 0x55adc74662f0 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.694+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55adc779c930 con 0x55adc748f1b0 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc027fc640 1 -- 172.21.15.92:0/3967905267 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fdbfc042030 con 0x7fdc0c103640 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:54766/0 (socket says 172.21.15.92:54766) 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc11a0d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7839060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 -- 172.21.15.92:0/1314240514 learned_addr learned my addr 172.21.15.92:0/1314240514 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc783a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 -- 172.21.15.92:0/1314240514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 msgr2=0x55adc7839060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7839060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 -- 172.21.15.92:0/1314240514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 msgr2=0x55adc783a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc783a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 -- 172.21.15.92:0/1314240514 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc775e530 con 0x55adc74662f0 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc1120c640 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78396d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800bd10 tx=0x7fdc080107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac65ec93fdbf3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.695+0000 7fdc027fc640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdbf80b6510 0x7fdbf80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.696+0000 7fdbf77fe640 1 -- 172.21.15.92:0/1314240514 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08004030 con 0x55adc74662f0 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.696+0000 7fdbf77fe640 1 -- 172.21.15.92:0/1314240514 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc080035f0 con 0x55adc74662f0 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.696+0000 7fdbf77fe640 1 -- 172.21.15.92:0/1314240514 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08011770 con 0x55adc74662f0 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.696+0000 7fdc11a0d640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdbf80b6510 0x7fdbf80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.696+0000 7fdc027fc640 1 -- 172.21.15.92:0/3967905267 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fdbfc107920 con 0x7fdc0c103640 2024-09-21T02:14:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/1314240514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 msgr2=0x55adc78396d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc11a0d640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdbf80b6510 0x7fdbf80b8a20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400af10 tx=0x7fdc0400a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78396d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800bd10 tx=0x7fdc080107a0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/1314240514 shutdown_connections 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc783a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78396d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/1314240514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7839060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/1314240514 >> 172.21.15.92:0/1314240514 conn(0x55adc7483300 msgr2=0x55adc78328e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/1314240514 shutdown_connections 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/1314240514 wait complete. 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 -- start start 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.697+0000 7fdc1a3dc740 1 Processor -- start 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc7842eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7843a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55adc76d0ab0 con 0x55adc746e4d0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55adc773e730 con 0x55adc74662f0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55adc783ad40 con 0x55adc748f1b0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 10 client.0 _ll_register_callbacks cb 0x55adc77926b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc7842eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc7842eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59058/0 (socket says 172.21.15.92:59058) 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 -- 172.21.15.92:0/477360231 learned_addr learned my addr 172.21.15.92:0/477360231 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1120c640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc11a0d640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7843a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 -- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 msgr2=0x55adc7843a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7843a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 -- 172.21.15.92:0/477360231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 msgr2=0x55adc78427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 -- 172.21.15.92:0/477360231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc74fa790 con 0x55adc746e4d0 2024-09-21T02:14:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1120c640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc7842eb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0cadb0 tx=0x7fdbfc0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc90042473c352c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdbf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/477360231 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc049070 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/477360231 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fdbfc03bc20 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc080035f0 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/477360231 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdbfc0c99e0 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 10 client.18267 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.698+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adc7573620 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/477360231 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc080035f0 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/477360231 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdbfc0d7020 con 0x55adc746e4d0 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1a3dc740 10 client.18267 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1a3dc740 10 client.18267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1a3dc740 20 client.18267 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc744dd80 0x55adc784ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1a3dc740 4 client.18267 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc77ab640 con 0x55adc744dd80 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.699+0000 7fdc1120c640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc744dd80 0x55adc784ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.700+0000 7fdc1120c640 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc744dd80 0x55adc784ff30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c55128f994db97 server_cookie=6fe1f7a2790b777b in_seq=0 out_seq=0 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.700+0000 7fdbf4ff9640 10 client.18267 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/477360231 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55adc77ab640 con 0x55adc744dd80 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdbf4ff9640 10 client.18267 handle_command_reply: tid=0 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 client.18267 shutdown 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 msgr2=0x55adc7842eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc7842eb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0cadb0 tx=0x7fdbfc0045f0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 shutdown_connections 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc744dd80 0x55adc784ff30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc748f1b0 0x55adc7843a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc746e4d0 0x55adc7842eb0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdbf4ff9640 0 client.18267 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/477360231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc74662f0 0x55adc78427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 >> 172.21.15.92:0/477360231 conn(0x55adc7483300 msgr2=0x55adc7830ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 shutdown_connections 2024-09-21T02:14:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/477360231 wait complete. 2024-09-21T02:14:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.701+0000 7fdc1a3dc740 20 client.18267 trim_cache size 0 max 16384 2024-09-21T02:14:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.715+0000 7fdc1a3dc740 1 Processor -- start 2024-09-21T02:14:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 -- start start 2024-09-21T02:14:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc7840d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7841240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7839000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1a8eb0 con 0x55adc744dd80 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1aa9f0 con 0x55adc746e4d0 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1a9580 con 0x55adc74662f0 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc7840d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc7840d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59066/0 (socket says 172.21.15.92:59066) 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 -- 172.21.15.92:0/3466507953 learned_addr learned my addr 172.21.15.92:0/3466507953 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7841240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 -- 172.21.15.92:0/3466507953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 msgr2=0x55adc7841240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7841240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 -- 172.21.15.92:0/3466507953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 msgr2=0x55adc7839000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7839000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 -- 172.21.15.92:0/3466507953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc7842120 con 0x55adc744dd80 2024-09-21T02:14:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7841240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc7840d00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdc08000c00 tx=0x7fdc08005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef3269ddf0c6512 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3466507953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08003a20 con 0x55adc744dd80 2024-09-21T02:14:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.716+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3466507953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0800b8e0 con 0x55adc744dd80 2024-09-21T02:14:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3466507953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08011710 con 0x55adc744dd80 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3466507953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 msgr2=0x55adc7840d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc7840d00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdc08000c00 tx=0x7fdc08005850 comp rx=0 tx=0).stop 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3466507953 shutdown_connections 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7839000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7841240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3466507953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc7840d00 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3466507953 >> 172.21.15.92:0/3466507953 conn(0x55adc7483300 msgr2=0x55adc7841a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3466507953 shutdown_connections 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.717+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3466507953 wait complete. 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 -- start start 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 Processor -- start 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc784a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7846510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7846c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1aa320 con 0x55adc744dd80 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1a9c50 con 0x55adc746e4d0 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1a6e90 con 0x55adc74662f0 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1a3dc740 10 client.0 _ll_register_callbacks cb 0x7fdbfc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc784a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc784a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59072/0 (socket says 172.21.15.92:59072) 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 -- 172.21.15.92:0/3116845176 learned_addr learned my addr 172.21.15.92:0/3116845176 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 -- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 msgr2=0x55adc7846510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7846510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc11a0d640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7846c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7846510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 -- 172.21.15.92:0/3116845176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 msgr2=0x55adc7846c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7846c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.718+0000 7fdc1120c640 1 -- 172.21.15.92:0/3116845176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc7847820 con 0x55adc744dd80 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7846510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc11a0d640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7846c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1120c640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc784a6a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fdc080107e0 tx=0x7fdc08005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4156fd90e6e786e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdbf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/3116845176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08032070 con 0x55adc744dd80 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/3116845176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fdc080052b0 con 0x55adc744dd80 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0800b8e0 con 0x55adc744dd80 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/3116845176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc0803d070 con 0x55adc744dd80 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 10 client.18273 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adc78bac80 con 0x55adc744dd80 2024-09-21T02:14:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/3116845176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0800b8e0 con 0x55adc744dd80 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/3116845176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc08037460 con 0x55adc744dd80 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 10 client.18273 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 10 client.18273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 20 client.18273 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc748f1b0 0x55adc78bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 4 client.18273 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.719+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adc77a70f0 con 0x55adc748f1b0 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.720+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc748f1b0 0x55adc78bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.720+0000 7fdbf4ff9640 10 client.18273 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.720+0000 7fdc10a0b640 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc748f1b0 0x55adc78bd4a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8f7519a067fbb1 server_cookie=e46accb7ada4cffa in_seq=0 out_seq=0 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.735+0000 7fdbf4ff9640 1 -- 172.21.15.92:0/3116845176 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55adc77a70f0 con 0x55adc748f1b0 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.735+0000 7fdbf4ff9640 10 client.18273 handle_command_reply: tid=0 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.735+0000 7fdc1a3dc740 1 client.18273 shutdown 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 msgr2=0x55adc784a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc784a6a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fdc080107e0 tx=0x7fdc08005c10 comp rx=0 tx=0).stop 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 shutdown_connections 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55adc748f1b0 0x55adc78bd4a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc746e4d0 0x55adc7846c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55adc74662f0 0x55adc7846510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdbf4ff9640 0 client.18273 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 --2- 172.21.15.92:0/3116845176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55adc744dd80 0x55adc784a6a0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 >> 172.21.15.92:0/3116845176 conn(0x55adc7483300 msgr2=0x55adc7841a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 shutdown_connections 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 1 -- 172.21.15.92:0/3116845176 wait complete. 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc1a3dc740 20 client.18273 trim_cache size 0 max 16384 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:14:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdbf80b6510 msgr2=0x7fdbf80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdbf80b6510 0x7fdbf80b8a20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400af10 tx=0x7fdc0400a4d0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 msgr2=0x7fdc0c199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c199500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006010 tx=0x7fdbfc004340 comp rx=0 tx=0).stop 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 shutdown_connections 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fdbf80b6510 0x7fdbf80b8a20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0c103fe0 0x7fdc0c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc0c103640 0x7fdc0c199500 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 --2- 172.21.15.92:0/3967905267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0c1025c0 0x7fdc0c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.736+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 >> 172.21.15.92:0/3967905267 conn(0x7fdc0c0fdf00 msgr2=0x7fdc0c106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.737+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 shutdown_connections 2024-09-21T02:14:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:33.737+0000 7fdc13470640 1 -- 172.21.15.92:0/3967905267 wait complete. 2024-09-21T02:14:35.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:35 smithi136 ceph-mon[34754]: pgmap v648: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:35.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:35 smithi092 ceph-mon[31190]: pgmap v648: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:35.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:35 smithi099 ceph-mon[34404]: pgmap v648: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-09-21T02:14:37.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:37 smithi136 ceph-mon[34754]: pgmap v649: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:14:37.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:37 smithi092 ceph-mon[31190]: pgmap v649: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:14:37.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:37 smithi099 ceph-mon[34404]: pgmap v649: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:14:39.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:39 smithi136 ceph-mon[34754]: pgmap v650: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:14:39.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:39 smithi092 ceph-mon[31190]: pgmap v650: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:14:39.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:39 smithi099 ceph-mon[34404]: pgmap v650: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:14:41.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:41 smithi136 ceph-mon[34754]: pgmap v651: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:14:41.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:41 smithi092 ceph-mon[31190]: pgmap v651: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:14:41.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:41 smithi099 ceph-mon[34404]: pgmap v651: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:14:43.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:43 smithi136 ceph-mon[34754]: pgmap v652: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:43.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:43 smithi092 ceph-mon[31190]: pgmap v652: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:43.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:43 smithi099 ceph-mon[34404]: pgmap v652: 97 pgs: 97 active+clean; 3.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:14:45.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:45 smithi136 ceph-mon[34754]: pgmap v653: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:14:45.533 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:45 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:14:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:45 smithi092 ceph-mon[31190]: pgmap v653: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:14:45.554 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:45 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:14:45.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:45 smithi099 ceph-mon[34404]: pgmap v653: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:14:45.581 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:45 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/50668497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0103ff0 msgr2=0x7f8ff010a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/50668497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0103ff0 0x7f8ff010a990 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009ca0 tx=0x7f8fe802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/50668497 shutdown_connections 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/50668497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0103ff0 0x7f8ff010a990 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/50668497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff0103650 0x7f8ff0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/50668497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff01025d0 0x7f8ff0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/50668497 >> 172.21.15.92:0/50668497 conn(0x7f8ff00fdf30 msgr2=0x7f8ff01003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/50668497 shutdown_connections 2024-09-21T02:14:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/50668497 wait complete. 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.840+0000 7f8ff4b4a640 1 Processor -- start 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 -- start start 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 0x7f8ff006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff0103650 0x7f8ff019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff0103ff0 0x7f8ff01a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff010e220 con 0x7f8ff01025d0 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff010e0a0 con 0x7f8ff0103650 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8ff4b4a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff010e3a0 con 0x7f8ff0103ff0 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 0x7f8ff006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 0x7f8ff006c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49630/0 (socket says 172.21.15.92:49630) 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 -- 172.21.15.92:0/382153062 learned_addr learned my addr 172.21.15.92:0/382153062 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff0103650 0x7f8ff019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 -- 172.21.15.92:0/382153062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff0103ff0 msgr2=0x7f8ff01a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff0103ff0 0x7f8ff01a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 -- 172.21.15.92:0/382153062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff0103650 msgr2=0x7f8ff019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff0103650 0x7f8ff019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 -- 172.21.15.92:0/382153062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff01a1a30 con 0x7f8ff01025d0 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.841+0000 7f8fee59c640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 0x7f8ff006c1f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9f0 tx=0x7f8fe000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.842+0000 7f8fdf7fe640 1 -- 172.21.15.92:0/382153062 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000cd80 con 0x7f8ff01025d0 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.842+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff01a1d20 con 0x7f8ff01025d0 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.842+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a2200 con 0x7f8ff01025d0 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.842+0000 7f8fdf7fe640 1 -- 172.21.15.92:0/382153062 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe00108e0 con 0x7f8ff01025d0 2024-09-21T02:14:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.842+0000 7f8fdf7fe640 1 -- 172.21.15.92:0/382153062 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00188e0 con 0x7f8ff01025d0 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.844+0000 7f8ffbab4740 1 Processor -- start 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.844+0000 7f8ffbab4740 1 -- start start 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8fdf7fe640 1 -- 172.21.15.92:0/382153062 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8fe0018a80 con 0x7f8ff01025d0 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 0x55c01bd38e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd3a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c01bcab6f0 con 0x55c01b9682f0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c01bc9ab20 con 0x55c01b9911b0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c01bc9e930 con 0x55c01b9704d0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8fee59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 0x55c01bd38e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 0x55c01bd38e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.92:37820/0 (socket says 172.21.15.92:37820) 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 -- 172.21.15.92:0/435649776 learned_addr learned my addr 172.21.15.92:0/435649776 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd3a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 -- 172.21.15.92:0/435649776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 msgr2=0x55c01bd3a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd3a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 -- 172.21.15.92:0/435649776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 msgr2=0x55c01bd394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 -- 172.21.15.92:0/435649776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01bc60530 con 0x55c01b9911b0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8fdf7fe640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8fd80b6510 0x7f8fd80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.845+0000 7f8feed9d640 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 0x55c01bd38e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80097b0 tx=0x7f8fe802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c4845db28d832f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.846+0000 7f8fee59c640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8fd80b6510 0x7f8fd80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.846+0000 7f8fdcff9640 1 -- 172.21.15.92:0/435649776 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8031650 con 0x55c01b9911b0 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.846+0000 7f8fdf7fe640 1 -- 172.21.15.92:0/382153062 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8fe00e7690 con 0x7f8ff01025d0 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.846+0000 7f8fdcff9640 1 -- 172.21.15.92:0/435649776 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe803f070 con 0x55c01b9911b0 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.846+0000 7f8fdcff9640 1 -- 172.21.15.92:0/435649776 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80044e0 con 0x55c01b9911b0 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- 172.21.15.92:0/435649776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 msgr2=0x55c01bd38e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8fee59c640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8fd80b6510 0x7f8fd80b8a20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00aaa70 tx=0x7f8fe00ed550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 0x55c01bd38e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80097b0 tx=0x7f8fe802f3a0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- 172.21.15.92:0/435649776 shutdown_connections 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd3a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/435649776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9911b0 0x55c01bd38e50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- 172.21.15.92:0/435649776 >> 172.21.15.92:0/435649776 conn(0x55c01b985300 msgr2=0x55c01bd348e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- 172.21.15.92:0/435649776 shutdown_connections 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- 172.21.15.92:0/435649776 wait complete. 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- start start 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 Processor -- start 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9682f0 0x55c01bd425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bd42c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9911b0 0x55c01bd437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c01bbd2ab0 con 0x55c01b9911b0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c01bc40730 con 0x55c01b9704d0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.847+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c01bd3aad0 con 0x55c01b9682f0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8ffbab4740 10 client.0 _ll_register_callbacks cb 0x55c01bc946b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9911b0 0x55c01bd437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9682f0 0x55c01bd425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bd42c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8feed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9682f0 0x55c01bd425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:37844/0 (socket says 172.21.15.92:37844) 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8feed9d640 1 -- 172.21.15.92:0/1198781547 learned_addr learned my addr 172.21.15.92:0/1198781547 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 -- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9682f0 msgr2=0x55c01bd425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9682f0 0x55c01bd425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 -- 172.21.15.92:0/1198781547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9911b0 msgr2=0x55c01bd437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9911b0 0x55c01bd437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 -- 172.21.15.92:0/1198781547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01b9fc790 con 0x55c01b9704d0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bd42c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400b960 tx=0x7f8fe400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=225ef4c406b3b336 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/1198781547 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4005020 con 0x55c01b9704d0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/1198781547 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8fe40051c0 con 0x55c01b9704d0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe803f070 con 0x55c01b9704d0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/1198781547 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fe4021070 con 0x55c01b9704d0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8ffbab4740 10 client.27050 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.848+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c01ba75620 con 0x55c01b9704d0 2024-09-21T02:14:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/1198781547 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe803f070 con 0x55c01b9704d0 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/1198781547 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8fe401b470 con 0x55c01b9704d0 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8ffbab4740 10 client.27050 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8ffbab4740 10 client.27050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8ffbab4740 20 client.27050 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b94fd80 0x55c01bd4fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8ffbab4740 4 client.27050 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c01bcad640 con 0x55c01b94fd80 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.849+0000 7f8feed9d640 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b94fd80 0x55c01bd4fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8fd5ffb640 10 client.27050 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8feed9d640 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b94fd80 0x55c01bd4fe50 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89baccbd66c68019 server_cookie=47c966e54fe03d6d in_seq=0 out_seq=0 2024-09-21T02:14:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/1198781547 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c01bcad640 con 0x55c01b94fd80 2024-09-21T02:14:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8fd5ffb640 10 client.27050 handle_command_reply: tid=0 2024-09-21T02:14:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 client.27050 shutdown 2024-09-21T02:14:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 msgr2=0x55c01bd42c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bd42c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400b960 tx=0x7f8fe400be20 comp rx=0 tx=0).stop 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 shutdown_connections 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b94fd80 0x55c01bd4fe50 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9911b0 0x55c01bd437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bd42c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8fd5ffb640 0 client.27050 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.850+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/1198781547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9682f0 0x55c01bd425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.851+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 >> 172.21.15.92:0/1198781547 conn(0x55c01b985300 msgr2=0x55c01bd39ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.851+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 shutdown_connections 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.851+0000 7f8ffbab4740 1 -- 172.21.15.92:0/1198781547 wait complete. 2024-09-21T02:14:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.851+0000 7f8ffbab4740 20 client.27050 trim_cache size 0 max 16384 2024-09-21T02:14:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 Processor -- start 2024-09-21T02:14:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 -- start start 2024-09-21T02:14:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b94fd80 0x55c01bd38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd39330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd000e530 con 0x55c01b9682f0 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0010070 con 0x55c01b94fd80 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd000ec00 con 0x55c01b9704d0 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd39330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd39330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49670/0 (socket says 172.21.15.92:49670) 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 -- 172.21.15.92:0/3234845129 learned_addr learned my addr 172.21.15.92:0/3234845129 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8feed9d640 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 -- 172.21.15.92:0/3234845129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 msgr2=0x55c01bd425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 -- 172.21.15.92:0/3234845129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b94fd80 msgr2=0x55c01bd38df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b94fd80 0x55c01bd38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8fedd9b640 1 -- 172.21.15.92:0/3234845129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01bd3dd50 con 0x55c01b9682f0 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.865+0000 7f8feed9d640 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.866+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd39330 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4001eb0 tx=0x7f8fe401cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c398a38ba718f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.866+0000 7f8fd7fff640 1 -- 172.21.15.92:0/3234845129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401bb00 con 0x55c01b9682f0 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.866+0000 7f8fd7fff640 1 -- 172.21.15.92:0/3234845129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe401bca0 con 0x55c01b9682f0 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.866+0000 7f8fd7fff640 1 -- 172.21.15.92:0/3234845129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40049f0 con 0x55c01b9682f0 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.866+0000 7f8ffbab4740 1 -- 172.21.15.92:0/3234845129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 msgr2=0x55c01bd39330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.866+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd39330 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4001eb0 tx=0x7f8fe401cf60 comp rx=0 tx=0).stop 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- 172.21.15.92:0/3234845129 shutdown_connections 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b9704d0 0x55c01bd425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bd39330 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/3234845129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b94fd80 0x55c01bd38df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- 172.21.15.92:0/3234845129 >> 172.21.15.92:0/3234845129 conn(0x55c01b985300 msgr2=0x55c01bd2f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- 172.21.15.92:0/3234845129 shutdown_connections 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- 172.21.15.92:0/3234845129 wait complete. 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- start start 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 Processor -- start 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b94fd80 0x55c01bdb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bdb5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bdb67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd000f9a0 con 0x55c01b9682f0 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd000f2d0 con 0x55c01b9704d0 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd000c510 con 0x55c01b94fd80 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8ffbab4740 10 client.0 _ll_register_callbacks cb 0x7f8fe40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8fedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bdb5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8fedd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bdb5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49676/0 (socket says 172.21.15.92:49676) 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.867+0000 7f8fedd9b640 1 -- 172.21.15.92:0/4137465560 learned_addr learned my addr 172.21.15.92:0/4137465560 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8feed9d640 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b94fd80 0x55c01bdb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fedd9b640 1 -- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b94fd80 msgr2=0x55c01bdb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b94fd80 0x55c01bdb55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fedd9b640 1 -- 172.21.15.92:0/4137465560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 msgr2=0x55c01bdb67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bdb67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fedd9b640 1 -- 172.21.15.92:0/4137465560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01bdb6f50 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fedd9b640 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bdb5c80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400be60 tx=0x7f8fe401b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562bc3cc4dd99d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/4137465560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4026690 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/4137465560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8fe4026830 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe401bca0 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/4137465560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fe4026a30 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 10 client.18291 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c01bdbe1f0 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/4137465560 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe401bca0 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/4137465560 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8fe4037070 con 0x55c01b9682f0 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 10 client.18291 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 20 client.18291 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b9911b0 0x55c01bdc0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 4 client.18291 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:14:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.868+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c01bca90f0 con 0x55c01b9911b0 2024-09-21T02:14:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.869+0000 7f8feed9d640 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b9911b0 0x55c01bdc0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.869+0000 7f8feed9d640 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b9911b0 0x55c01bdc0b80 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f1f83693f64e80 server_cookie=e14b42f62308f657 in_seq=0 out_seq=0 2024-09-21T02:14:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.869+0000 7f8fd5ffb640 10 client.18291 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.883+0000 7f8fd5ffb640 1 -- 172.21.15.92:0/4137465560 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c01bca90f0 con 0x55c01b9911b0 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.883+0000 7f8fd5ffb640 10 client.18291 handle_command_reply: tid=0 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 client.18291 shutdown 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 msgr2=0x55c01bdb5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bdb5c80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400be60 tx=0x7f8fe401b470 comp rx=0 tx=0).stop 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 shutdown_connections 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55c01b9911b0 0x55c01bdc0b80 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c01b9704d0 0x55c01bdb67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8fd5ffb640 0 client.18291 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c01b9682f0 0x55c01bdb5c80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 --2- 172.21.15.92:0/4137465560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c01b94fd80 0x55c01bdb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 >> 172.21.15.92:0/4137465560 conn(0x55c01b985300 msgr2=0x55c01bd408f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 shutdown_connections 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 1 -- 172.21.15.92:0/4137465560 wait complete. 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ffbab4740 20 client.18291 trim_cache size 0 max 16384 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:14:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8fd80b6510 msgr2=0x7f8fd80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.884+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8fd80b6510 0x7f8fd80b8a20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00aaa70 tx=0x7f8fe00ed550 comp rx=0 tx=0).stop 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 msgr2=0x7f8ff006c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 0x7f8ff006c1f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9f0 tx=0x7f8fe000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 shutdown_connections 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8fd80b6510 0x7f8fd80b8a20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff0103ff0 0x7f8ff01a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff0103650 0x7f8ff019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 --2- 172.21.15.92:0/382153062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01025d0 0x7f8ff006c1f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 >> 172.21.15.92:0/382153062 conn(0x7f8ff00fdf30 msgr2=0x7f8ff0106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 shutdown_connections 2024-09-21T02:14:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:46.885+0000 7f8ff4b4a640 1 -- 172.21.15.92:0/382153062 wait complete. 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: pgmap v654: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:14:47.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:14:47.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: pgmap v654: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:14:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:14:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:14:47.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: pgmap v654: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:14:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:14:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:14:47.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:14:47.339 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:47.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 -- 172.21.15.92:0/3720393351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 msgr2=0x7fb090103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 --2- 172.21.15.92:0/3720393351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb090103f70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb084009900 tx=0x7fb08402a410 comp rx=0 tx=0).stop 2024-09-21T02:14:47.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 -- 172.21.15.92:0/3720393351 shutdown_connections 2024-09-21T02:14:47.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 --2- 172.21.15.92:0/3720393351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb090109df0 0x7fb090110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 --2- 172.21.15.92:0/3720393351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 0x7fb0901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 --2- 172.21.15.92:0/3720393351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb090103f70 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 -- 172.21.15.92:0/3720393351 >> 172.21.15.92:0/3720393351 conn(0x7fb09007b850 msgr2=0x7fb09007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 -- 172.21.15.92:0/3720393351 shutdown_connections 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.803+0000 7fb096e22640 1 -- 172.21.15.92:0/3720393351 wait complete. 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 Processor -- start 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 -- start start 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb09019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 0x7fb09019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb090109df0 0x7fb09019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb090113b70 con 0x7fb090103b60 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901139f0 con 0x7fb090109df0 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb096e22640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb090113cf0 con 0x7fb0901094a0 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb094bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb09019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb094bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb09019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49692/0 (socket says 172.21.15.92:49692) 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.804+0000 7fb094bbe640 1 -- 172.21.15.92:0/827591002 learned_addr learned my addr 172.21.15.92:0/827591002 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb0953bf640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb090109df0 0x7fb09019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb08ffff640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 0x7fb09019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb094bbe640 1 -- 172.21.15.92:0/827591002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 msgr2=0x7fb09019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb094bbe640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 0x7fb09019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb094bbe640 1 -- 172.21.15.92:0/827591002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb090109df0 msgr2=0x7fb09019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb094bbe640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb090109df0 0x7fb09019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb094bbe640 1 -- 172.21.15.92:0/827591002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0901a0530 con 0x7fb090103b60 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb08ffff640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 0x7fb09019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb094bbe640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb09019a5c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb084006010 tx=0x7fb0840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb08dffb640 1 -- 172.21.15.92:0/827591002 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08403f030 con 0x7fb090103b60 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb08dffb640 1 -- 172.21.15.92:0/827591002 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb084003800 con 0x7fb090103b60 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb08dffb640 1 -- 172.21.15.92:0/827591002 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084035070 con 0x7fb090103b60 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.805+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0901a07c0 con 0x7fb090103b60 2024-09-21T02:14:47.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.806+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0901a6f80 con 0x7fb090103b60 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 Processor -- start 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb08dffb640 1 -- 172.21.15.92:0/827591002 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb0840039b0 con 0x7fb090103b60 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 -- start start 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f2200070 0x55b5f25a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b5f251de50 con 0x55b5f21d71b0 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b5f250d280 con 0x55b5f21df390 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb09d396740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b5f2511090 con 0x55b5f2200070 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.808+0000 7fb094bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25a77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49694/0 (socket says 172.21.15.92:49694) 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 -- 172.21.15.92:0/1044488429 learned_addr learned my addr 172.21.15.92:0/1044488429 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:47.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb0953bf640 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f2200070 0x55b5f25a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb08ffff640 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 -- 172.21.15.92:0/1044488429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f2200070 msgr2=0x55b5f25a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f2200070 0x55b5f25a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 -- 172.21.15.92:0/1044488429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 msgr2=0x55b5f25a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 -- 172.21.15.92:0/1044488429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f226b650 con 0x55b5f21d71b0 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb08dffb640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0800b6590 0x7fb0800b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb0953bf640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0800b6590 0x7fb0800b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb08dffb640 1 -- 172.21.15.92:0/827591002 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb084107be0 con 0x7fb090103b60 2024-09-21T02:14:47.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb094bbe640 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25a77b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb08410bd80 tx=0x7fb08410be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b5b90e923a9ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.809+0000 7fb07affd640 1 -- 172.21.15.92:0/1044488429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084049070 con 0x55b5f21d71b0 2024-09-21T02:14:47.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb07affd640 1 -- 172.21.15.92:0/1044488429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0840cd860 con 0x55b5f21d71b0 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb07affd640 1 -- 172.21.15.92:0/1044488429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840cdb20 con 0x55b5f21d71b0 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb09d396740 1 -- 172.21.15.92:0/1044488429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 msgr2=0x55b5f25a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb0953bf640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0800b6590 0x7fb0800b8aa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb08800a400 tx=0x7fb08800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb09d396740 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25a77b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb08410bd80 tx=0x7fb08410be70 comp rx=0 tx=0).stop 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb09d396740 1 -- 172.21.15.92:0/1044488429 shutdown_connections 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb09d396740 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb09d396740 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25a77b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.810+0000 7fb09d396740 1 --2- 172.21.15.92:0/1044488429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f2200070 0x55b5f25a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.811+0000 7fb09d396740 1 -- 172.21.15.92:0/1044488429 >> 172.21.15.92:0/1044488429 conn(0x55b5f21f4460 msgr2=0x55b5f25a5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.811+0000 7fb09d396740 1 -- 172.21.15.92:0/1044488429 shutdown_connections 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.811+0000 7fb09d396740 1 -- 172.21.15.92:0/1044488429 wait complete. 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.811+0000 7fb09d396740 1 -- start start 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.811+0000 7fb09d396740 1 Processor -- start 2024-09-21T02:14:47.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.811+0000 7fb09d396740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21df390 0x55b5f25b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f2200070 0x55b5f25beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b5f236e700 con 0x55b5f21d71b0 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b5f24a8110 con 0x55b5f2200070 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b5f25b5810 con 0x55b5f21df390 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49698/0 (socket says 172.21.15.92:49698) 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 -- 172.21.15.92:0/2459947849 learned_addr learned my addr 172.21.15.92:0/2459947849 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 10 client.0 _ll_register_callbacks cb 0x55b5f24c1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb0953bf640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f2200070 0x55b5f25beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb08ffff640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21df390 0x55b5f25b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 -- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21df390 msgr2=0x55b5f25b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21df390 0x55b5f25b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 -- 172.21.15.92:0/2459947849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f2200070 msgr2=0x55b5f25beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f2200070 0x55b5f25beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 -- 172.21.15.92:0/2459947849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f24d5c40 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb0953bf640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f2200070 0x55b5f25beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb094bbe640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b2920 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb084105710 tx=0x7fb084005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad824428d08b6e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb06ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb06ffff640 1 -- 172.21.15.92:0/2459947849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084049070 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb06ffff640 1 -- 172.21.15.92:0/2459947849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb0840b5030 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.812+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0840cd860 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb06ffff640 1 -- 172.21.15.92:0/2459947849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0840cd4c0 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 10 client.18303 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5f25c1460 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb06ffff640 1 -- 172.21.15.92:0/2459947849 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0840cd860 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb06ffff640 1 -- 172.21.15.92:0/2459947849 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb0840d72e0 con 0x55b5f21d71b0 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 10 client.18303 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 20 client.18303 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f21be9f0 0x55b5f25c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 4 client.18303 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:47.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5f251fda0 con 0x55b5f21be9f0 2024-09-21T02:14:47.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.813+0000 7fb08ffff640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f21be9f0 0x55b5f25c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.814+0000 7fb08ffff640 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f21be9f0 0x55b5f25c47c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df154823e014bbb9 server_cookie=952ad4af8939adc8 in_seq=0 out_seq=0 2024-09-21T02:14:47.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.814+0000 7fb06ffff640 10 client.18303 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb06ffff640 1 -- 172.21.15.92:0/2459947849 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b5f251fda0 con 0x55b5f21be9f0 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb06ffff640 10 client.18303 handle_command_reply: tid=0 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 client.18303 shutdown 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 msgr2=0x55b5f25b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b2920 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb084105710 tx=0x7fb084005f30 comp rx=0 tx=0).stop 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 shutdown_connections 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f21be9f0 0x55b5f25c47c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb06ffff640 0 client.18303 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f2200070 0x55b5f25beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21df390 0x55b5f25b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 --2- 172.21.15.92:0/2459947849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b2920 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 >> 172.21.15.92:0/2459947849 conn(0x55b5f21f4460 msgr2=0x55b5f25aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 shutdown_connections 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 1 -- 172.21.15.92:0/2459947849 wait complete. 2024-09-21T02:14:47.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.815+0000 7fb09d396740 20 client.18303 trim_cache size 0 max 16384 2024-09-21T02:14:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.832+0000 7fb09d396740 1 Processor -- start 2024-09-21T02:14:47.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.832+0000 7fb09d396740 1 -- start start 2024-09-21T02:14:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb09d396740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb09d396740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb09d396740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb09d396740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901af500 con 0x55b5f21d71b0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb09d396740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901b1040 con 0x55b5f21df390 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb09d396740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901afbd0 con 0x55b5f21be9f0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49700/0 (socket says 172.21.15.92:49700) 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 -- 172.21.15.92:0/962393589 learned_addr learned my addr 172.21.15.92:0/962393589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb0953bf640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb094bbe640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 -- 172.21.15.92:0/962393589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 msgr2=0x55b5f25a8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25a8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 -- 172.21.15.92:0/962393589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 msgr2=0x55b5f25bea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25bea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 -- 172.21.15.92:0/962393589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f25aa0d0 con 0x55b5f21d71b0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb094bbe640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb0953bf640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb08ffff640 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b4550 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb09006c9f0 tx=0x7fb07c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e1e4174b070f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.833+0000 7fb07a7fc640 1 -- 172.21.15.92:0/962393589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c020070 con 0x55b5f21d71b0 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb07a7fc640 1 -- 172.21.15.92:0/962393589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07c0143b0 con 0x55b5f21d71b0 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb07a7fc640 1 -- 172.21.15.92:0/962393589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0235c0 con 0x55b5f21d71b0 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 -- 172.21.15.92:0/962393589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 msgr2=0x55b5f25b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b4550 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb09006c9f0 tx=0x7fb07c008040 comp rx=0 tx=0).stop 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 -- 172.21.15.92:0/962393589 shutdown_connections 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21df390 0x55b5f25bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21d71b0 0x55b5f25b4550 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 --2- 172.21.15.92:0/962393589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 -- 172.21.15.92:0/962393589 >> 172.21.15.92:0/962393589 conn(0x55b5f21f4460 msgr2=0x55b5f25b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 -- 172.21.15.92:0/962393589 shutdown_connections 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.834+0000 7fb09d396740 1 -- 172.21.15.92:0/962393589 wait complete. 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 -- start start 2024-09-21T02:14:47.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 Processor -- start 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21d71b0 0x55b5f25b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 0x55b5f2627130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901b0970 con 0x55b5f21df390 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901b02a0 con 0x55b5f21d71b0 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0901ad4e0 con 0x55b5f21be9f0 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb09d396740 10 client.0 _ll_register_callbacks cb 0x7fb07c0071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb0953bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 0x55b5f2627130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb0953bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 0x55b5f2627130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49706/0 (socket says 172.21.15.92:49706) 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.835+0000 7fb0953bf640 1 -- 172.21.15.92:0/3260114659 learned_addr learned my addr 172.21.15.92:0/3260114659 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb094bbe640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb08ffff640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21d71b0 0x55b5f25b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb0953bf640 1 -- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 msgr2=0x55b5f25b95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb0953bf640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25b95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb0953bf640 1 -- 172.21.15.92:0/3260114659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21d71b0 msgr2=0x55b5f25b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb0953bf640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21d71b0 0x55b5f25b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb0953bf640 1 -- 172.21.15.92:0/3260114659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f26277a0 con 0x55b5f21df390 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb08ffff640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21d71b0 0x55b5f25b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb0953bf640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 0x55b5f2627130 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb0880164e0 tx=0x7fb088004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6a3df3e06b1776 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb06ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb06ffff640 1 -- 172.21.15.92:0/3260114659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088027020 con 0x55b5f21df390 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb06ffff640 1 -- 172.21.15.92:0/3260114659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb088023820 con 0x55b5f21df390 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07c0143b0 con 0x55b5f21df390 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb06ffff640 1 -- 172.21.15.92:0/3260114659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb088032050 con 0x55b5f21df390 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb09d396740 10 client.18309 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.836+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5f262fcd0 con 0x55b5f21df390 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb06ffff640 1 -- 172.21.15.92:0/3260114659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07c0143b0 con 0x55b5f21df390 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb06ffff640 1 -- 172.21.15.92:0/3260114659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb088017a90 con 0x55b5f21df390 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb09d396740 10 client.18309 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb09d396740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb09d396740 20 client.18309 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb09d396740 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f2200070 0x55b5f2632540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb09d396740 4 client.18309 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5f251b850 con 0x55b5f2200070 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.837+0000 7fb094bbe640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f2200070 0x55b5f2632540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb06ffff640 10 client.18309 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb094bbe640 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f2200070 0x55b5f2632540 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f062b06dfd32ba server_cookie=d4e1bbe60c6a6d56 in_seq=0 out_seq=0 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb06ffff640 1 -- 172.21.15.92:0/3260114659 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x55b5f251b850 con 0x55b5f2200070 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb06ffff640 10 client.18309 handle_command_reply: tid=0 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 client.18309 shutdown 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 msgr2=0x55b5f2627130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 0x55b5f2627130 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb0880164e0 tx=0x7fb088004a40 comp rx=0 tx=0).stop 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 shutdown_connections 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55b5f2200070 0x55b5f2632540 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5f21df390 0x55b5f2627130 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb06ffff640 0 client.18309 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5f21d71b0 0x55b5f25b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 --2- 172.21.15.92:0/3260114659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5f21be9f0 0x55b5f25b95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 >> 172.21.15.92:0/3260114659 conn(0x55b5f21f4460 msgr2=0x55b5f25b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 shutdown_connections 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 1 -- 172.21.15.92:0/3260114659 wait complete. 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.838+0000 7fb09d396740 20 client.18309 trim_cache size 0 max 16384 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:14:47.843 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a": { 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:14:14.805000+0000", 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:14:47.844 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:14:47.845 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:14:47.846 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:14:47.847 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:13:30.072031+0000", 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.848 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.849 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:14:47.850 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:14:47.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0800b6590 msgr2=0x7fb0800b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0800b6590 0x7fb0800b8aa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb08800a400 tx=0x7fb08800d040 comp rx=0 tx=0).stop 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 msgr2=0x7fb09019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb09019a5c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb084006010 tx=0x7fb0840042f0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 shutdown_connections 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb0800b6590 0x7fb0800b8aa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb090109df0 0x7fb09019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0901094a0 0x7fb09019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 --2- 172.21.15.92:0/827591002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090103b60 0x7fb09019a5c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 >> 172.21.15.92:0/827591002 conn(0x7fb09007b850 msgr2=0x7fb0901075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 shutdown_connections 2024-09-21T02:14:47.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:47.839+0000 7fb096e22640 1 -- 172.21.15.92:0/827591002 wait complete. 2024-09-21T02:14:48.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:14:48.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:14:48.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:48 smithi092 ceph-mon[31190]: pgmap v655: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:14:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:49.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:48 smithi136 ceph-mon[34754]: pgmap v655: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:14:49.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:48 smithi099 ceph-mon[34404]: pgmap v655: 97 pgs: 97 active+clean; 3.4 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:14:49.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 -- 172.21.15.92:0/1794236629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96441048e0 msgr2=0x7f9644104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 --2- 172.21.15.92:0/1794236629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96441048e0 0x7f9644104cf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9634009ca0 tx=0x7f963402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 -- 172.21.15.92:0/1794236629 shutdown_connections 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 --2- 172.21.15.92:0/1794236629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f964410de90 0x7f9644110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 --2- 172.21.15.92:0/1794236629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644105310 0x7f964410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 --2- 172.21.15.92:0/1794236629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96441048e0 0x7f9644104cf0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 -- 172.21.15.92:0/1794236629 >> 172.21.15.92:0/1794236629 conn(0x7f9644079760 msgr2=0x7f9644079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 -- 172.21.15.92:0/1794236629 shutdown_connections 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.393+0000 7f9649a18640 1 -- 172.21.15.92:0/1794236629 wait complete. 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 Processor -- start 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 -- start start 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441048e0 0x7f96441027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9644105310 0x7f9644102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 0x7f9644103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9644113ac0 con 0x7f964410de90 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9644113940 con 0x7f96441048e0 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9649a18640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9644113c40 con 0x7f9644105310 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f96437fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 0x7f9644103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f96437fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 0x7f9644103270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49734/0 (socket says 172.21.15.92:49734) 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f96437fe640 1 -- 172.21.15.92:0/3374128008 learned_addr learned my addr 172.21.15.92:0/3374128008 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f9642ffd640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441048e0 0x7f96441027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.394+0000 7f96427fc640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9644105310 0x7f9644102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f96437fe640 1 -- 172.21.15.92:0/3374128008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9644105310 msgr2=0x7f9644102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f96437fe640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9644105310 0x7f9644102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f96437fe640 1 -- 172.21.15.92:0/3374128008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441048e0 msgr2=0x7f96441027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f96437fe640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441048e0 0x7f96441027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f96437fe640 1 -- 172.21.15.92:0/3374128008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a8850 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f96437fe640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 0x7f9644103270 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f963c00d750 tx=0x7f963c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c017070 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963c0054d0 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a8b40 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c004c60 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.395+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441a9080 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.396+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9644104cf0 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.398+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f963c009bd0 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.398+0000 7f9648a16640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96300b6510 0x7f96300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.398+0000 7f9642ffd640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96300b6510 0x7f96300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.399+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f963c0e6ba0 con 0x7f964410de90 2024-09-21T02:14:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.399+0000 7f9642ffd640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96300b6510 0x7f96300b8a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9634000f80 tx=0x7f9634009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.403+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f963c0b17f0 con 0x7f964410de90 2024-09-21T02:14:49.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.617+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96441a9440 con 0x7f964410de90 2024-09-21T02:14:49.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.617+0000 7f9648a16640 1 -- 172.21.15.92:0/3374128008 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f963c0b6730 con 0x7f964410de90 2024-09-21T02:14:49.619 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:14:49.619 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:14:49.619 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96300b6510 msgr2=0x7f96300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96300b6510 0x7f96300b8a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9634000f80 tx=0x7f9634009360 comp rx=0 tx=0).stop 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 msgr2=0x7f9644103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 0x7f9644103270 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f963c00d750 tx=0x7f963c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 shutdown_connections 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f96300b6510 0x7f96300b8a20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f964410de90 0x7f9644103270 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9644105310 0x7f9644102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 --2- 172.21.15.92:0/3374128008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441048e0 0x7f96441027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 >> 172.21.15.92:0/3374128008 conn(0x7f9644079760 msgr2=0x7f96441068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 shutdown_connections 2024-09-21T02:14:49.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:49.620+0000 7f9649a18640 1 -- 172.21.15.92:0/3374128008 wait complete. 2024-09-21T02:14:50.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:14:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:50.928 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:50 smithi092 ceph-mon[31190]: pgmap v656: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:14:50.929 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:50 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/3374128008' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/4184027146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8103190 msgr2=0x7fd9e81035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/4184027146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8103190 0x7fd9e81035a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009900 tx=0x7fd9d802a410 comp rx=0 tx=0).stop 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/4184027146 shutdown_connections 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/4184027146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e8108d50 0x7fd9e810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/4184027146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9e8108400 0x7fd9e8108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/4184027146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8103190 0x7fd9e81035a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/4184027146 >> 172.21.15.92:0/4184027146 conn(0x7fd9e8079f90 msgr2=0x7fd9e807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/4184027146 shutdown_connections 2024-09-21T02:14:51.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.019+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/4184027146 wait complete. 2024-09-21T02:14:51.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 Processor -- start 2024-09-21T02:14:51.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 -- start start 2024-09-21T02:14:51.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9e8103190 0x7fd9e819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e8108400 0x7fd9e819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 0x7fd9e819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e8112a90 con 0x7fd9e8108d50 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e8112910 con 0x7fd9e8103190 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ee2ab640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e8112c10 con 0x7fd9e8108400 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ec848640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 0x7fd9e819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ec848640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 0x7fd9e819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49744/0 (socket says 172.21.15.92:49744) 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9ec848640 1 -- 172.21.15.92:0/986294934 learned_addr learned my addr 172.21.15.92:0/986294934 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9e8103190 0x7fd9e819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.020+0000 7fd9dffff640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e8108400 0x7fd9e819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ec848640 1 -- 172.21.15.92:0/986294934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e8108400 msgr2=0x7fd9e819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ec848640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e8108400 0x7fd9e819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ec848640 1 -- 172.21.15.92:0/986294934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9e8103190 msgr2=0x7fd9e819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ec848640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9e8103190 0x7fd9e819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ec848640 1 -- 172.21.15.92:0/986294934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e81a0520 con 0x7fd9e8108d50 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ec848640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 0x7fd9e819fdf0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000b970 tx=0x7fd9e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9e5ffb640 1 -- 172.21.15.92:0/986294934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0019070 con 0x7fd9e8108d50 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9e5ffb640 1 -- 172.21.15.92:0/986294934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e0014440 con 0x7fd9e8108d50 2024-09-21T02:14:51.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e81a0810 con 0x7fd9e8108d50 2024-09-21T02:14:51.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9e5ffb640 1 -- 172.21.15.92:0/986294934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e00075d0 con 0x7fd9e8108d50 2024-09-21T02:14:51.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.021+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e81a6f70 con 0x7fd9e8108d50 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.023+0000 7fd9f481f740 1 Processor -- start 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.023+0000 7fd9e5ffb640 1 -- 172.21.15.92:0/986294934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd9e001c070 con 0x7fd9e8108d50 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 -- start start 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190631be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190632120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571905a7ea0 con 0x557190269390 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5571905972d0 con 0x5571902611b0 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9f481f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55719059b0e0 con 0x55719028a070 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190632120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190632120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49748/0 (socket says 172.21.15.92:49748) 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9dffff640 1 -- 172.21.15.92:0/3590268017 learned_addr learned my addr 172.21.15.92:0/3590268017 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190631be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9dffff640 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9e5ffb640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9d40b6590 0x7fd9d40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9e7fff640 1 -- 172.21.15.92:0/3590268017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 msgr2=0x5571906316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9e7fff640 1 -- 172.21.15.92:0/3590268017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 msgr2=0x557190632120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.024+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190632120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9e7fff640 1 -- 172.21.15.92:0/3590268017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571902f5650 con 0x5571902611b0 2024-09-21T02:14:51.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9dffff640 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190632120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9e5ffb640 1 -- 172.21.15.92:0/986294934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd9e00e6710 con 0x7fd9e8108d50 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9ec848640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9d40b6590 0x7fd9d40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190631be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009230 tx=0x7fd9d8031e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13853d9cd34321d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9de7fc640 1 -- 172.21.15.92:0/3590268017 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8040070 con 0x5571902611b0 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9de7fc640 1 -- 172.21.15.92:0/3590268017 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80316a0 con 0x5571902611b0 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.025+0000 7fd9de7fc640 1 -- 172.21.15.92:0/3590268017 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8035040 con 0x5571902611b0 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 -- 172.21.15.92:0/3590268017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 msgr2=0x557190631be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190631be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009230 tx=0x7fd9d8031e50 comp rx=0 tx=0).stop 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9ec848640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9d40b6590 0x7fd9d40b8aa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00e6a70 tx=0x7fd9e0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 -- 172.21.15.92:0/3590268017 shutdown_connections 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190632120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190631be0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 --2- 172.21.15.92:0/3590268017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 -- 172.21.15.92:0/3590268017 >> 172.21.15.92:0/3590268017 conn(0x55719027e460 msgr2=0x55719062f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 -- 172.21.15.92:0/3590268017 shutdown_connections 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 -- 172.21.15.92:0/3590268017 wait complete. 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.026+0000 7fd9f481f740 1 -- start start 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 Processor -- start 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571902611b0 0x55719063cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55719028a070 0x557190649110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571903f8700 con 0x557190269390 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557190532160 con 0x55719028a070 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55719063fe20 con 0x5571902611b0 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9f481f740 10 client.0 _ll_register_callbacks cb 0x55719054b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49762/0 (socket says 172.21.15.92:49762) 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 -- 172.21.15.92:0/2567863903 learned_addr learned my addr 172.21.15.92:0/2567863903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9dffff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571902611b0 0x55719063cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9dffff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55719028a070 0x557190649110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 -- 172.21.15.92:0/2567863903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571902611b0 msgr2=0x55719063cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571902611b0 0x55719063cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 -- 172.21.15.92:0/2567863903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55719028a070 msgr2=0x557190649110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55719028a070 0x557190649110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9e7fff640 1 -- 172.21.15.92:0/2567863903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719055fc90 con 0x557190269390 2024-09-21T02:14:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.027+0000 7fd9dffff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55719028a070 0x557190649110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063d280 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8031a80 tx=0x7fd9d802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5945b19e7b5efc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9c77fe640 1 -- 172.21.15.92:0/2567863903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d804c070 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9c77fe640 1 -- 172.21.15.92:0/2567863903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd9d8004040 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d80316a0 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9c77fe640 1 -- 172.21.15.92:0/2567863903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9d8058050 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 10 client.18327 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55719064bb10 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9c77fe640 1 -- 172.21.15.92:0/2567863903 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80316a0 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9c77fe640 1 -- 172.21.15.92:0/2567863903 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9d803c440 con 0x557190269390 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 10 client.18327 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 10 client.18327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:14:51.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 20 client.18327 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:51.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.028+0000 7fd9f481f740 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5571902489f0 0x55719064ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.029+0000 7fd9f481f740 4 client.18327 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:51.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.029+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571905a9df0 con 0x5571902489f0 2024-09-21T02:14:51.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.029+0000 7fd9dffff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5571902489f0 0x55719064ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.029+0000 7fd9c77fe640 10 client.18327 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:51.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.029+0000 7fd9dffff640 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5571902489f0 0x55719064ee70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aeda656dcd0b5c58 server_cookie=f6cf92a6100f2549 in_seq=0 out_seq=0 2024-09-21T02:14:51.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:50 smithi136 ceph-mon[34754]: pgmap v656: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:14:51.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:50 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/3374128008' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:14:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9c77fe640 1 -- 172.21.15.92:0/2567863903 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571905a9df0 con 0x5571902489f0 2024-09-21T02:14:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9c77fe640 10 client.18327 handle_command_reply: tid=0 2024-09-21T02:14:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 client.18327 shutdown 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 msgr2=0x55719063d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063d280 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8031a80 tx=0x7fd9d802a410 comp rx=0 tx=0).stop 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 shutdown_connections 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x5571902489f0 0x55719064ee70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55719028a070 0x557190649110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063d280 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 --2- 172.21.15.92:0/2567863903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5571902611b0 0x55719063cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 >> 172.21.15.92:0/2567863903 conn(0x55719027e460 msgr2=0x557190634880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9c77fe640 0 client.18327 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 shutdown_connections 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 1 -- 172.21.15.92:0/2567863903 wait complete. 2024-09-21T02:14:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.030+0000 7fd9f481f740 20 client.18327 trim_cache size 0 max 16384 2024-09-21T02:14:51.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.043+0000 7fd9f481f740 1 Processor -- start 2024-09-21T02:14:51.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.043+0000 7fd9f481f740 1 -- start start 2024-09-21T02:14:51.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190649080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x557190639f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9f481f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81af4f0 con 0x557190269390 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9f481f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81b1030 con 0x5571902611b0 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9f481f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81afbc0 con 0x55719028a070 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49778/0 (socket says 172.21.15.92:49778) 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 -- 172.21.15.92:0/1100573747 learned_addr learned my addr 172.21.15.92:0/1100573747 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9dffff640 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x557190639f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9dffff640 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190649080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 -- 172.21.15.92:0/1100573747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 msgr2=0x557190639f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x557190639f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 -- 172.21.15.92:0/1100573747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 msgr2=0x557190649080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190649080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 -- 172.21.15.92:0/1100573747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557190634500 con 0x557190269390 2024-09-21T02:14:51.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063e970 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009540 tx=0x7fd9d8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3298208278d06fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.044+0000 7fd9ddffb640 1 -- 172.21.15.92:0/1100573747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8055070 con 0x557190269390 2024-09-21T02:14:51.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9ddffb640 1 -- 172.21.15.92:0/1100573747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80314a0 con 0x557190269390 2024-09-21T02:14:51.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9ddffb640 1 -- 172.21.15.92:0/1100573747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d804c040 con 0x557190269390 2024-09-21T02:14:51.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 -- 172.21.15.92:0/1100573747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 msgr2=0x55719063e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063e970 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009540 tx=0x7fd9d8005780 comp rx=0 tx=0).stop 2024-09-21T02:14:51.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 -- 172.21.15.92:0/1100573747 shutdown_connections 2024-09-21T02:14:51.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x557190639f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x55719063e970 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 --2- 172.21.15.92:0/1100573747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x557190649080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.045+0000 7fd9f481f740 1 -- 172.21.15.92:0/1100573747 >> 172.21.15.92:0/1100573747 conn(0x55719027e460 msgr2=0x55719063e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 -- 172.21.15.92:0/1100573747 shutdown_connections 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 -- 172.21.15.92:0/1100573747 wait complete. 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 -- start start 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 Processor -- start 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x55719064ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190650460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.046+0000 7fd9f481f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9f481f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81b0960 con 0x557190269390 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9f481f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81b0290 con 0x5571902611b0 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9f481f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81ad4d0 con 0x55719028a070 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9f481f740 10 client.0 _ll_register_callbacks cb 0x7fd9d000a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190650460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190650460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49788/0 (socket says 172.21.15.92:49788) 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 -- 172.21.15.92:0/311979479 learned_addr learned my addr 172.21.15.92:0/311979479 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9dffff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9dffff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x55719064ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 -- 172.21.15.92:0/311979479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 msgr2=0x5571906509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 -- 172.21.15.92:0/311979479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 msgr2=0x55719064ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x55719064ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 -- 172.21.15.92:0/311979479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571906b8670 con 0x557190269390 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9dffff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x55719064ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9e7fff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190650460 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd9d802a930 tx=0x7fd9d802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3436341f5457e876 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9c77fe640 1 -- 172.21.15.92:0/311979479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8055040 con 0x557190269390 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9c77fe640 1 -- 172.21.15.92:0/311979479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd9d805d440 con 0x557190269390 2024-09-21T02:14:51.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.047+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d80314a0 con 0x557190269390 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9c77fe640 1 -- 172.21.15.92:0/311979479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9d804c040 con 0x557190269390 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 10 client.18333 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5571906b98f0 con 0x557190269390 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9c77fe640 1 -- 172.21.15.92:0/311979479 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d80314a0 con 0x557190269390 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9c77fe640 1 -- 172.21.15.92:0/311979479 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9d8051440 con 0x557190269390 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 10 client.18333 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 10 client.18333 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 20 client.18333 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55719021dda0 0x5571906bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 4 client.18333 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571905a58a0 con 0x55719021dda0 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9dffff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55719021dda0 0x5571906bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.048+0000 7fd9c77fe640 10 client.18333 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9dffff640 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55719021dda0 0x5571906bbff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ac2647e159fe04c server_cookie=4f94e56c3271ed72 in_seq=0 out_seq=0 2024-09-21T02:14:51.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9c77fe640 1 -- 172.21.15.92:0/311979479 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571905a58a0 con 0x55719021dda0 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9c77fe640 10 client.18333 handle_command_reply: tid=0 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 client.18333 shutdown 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 msgr2=0x557190650460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190650460 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd9d802a930 tx=0x7fd9d802a410 comp rx=0 tx=0).stop 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 shutdown_connections 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55719021dda0 0x5571906bbff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719028a070 0x5571906509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557190269390 0x557190650460 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 --2- 172.21.15.92:0/311979479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571902611b0 0x55719064ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 >> 172.21.15.92:0/311979479 conn(0x55719027e460 msgr2=0x557190634760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9c77fe640 0 client.18333 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 shutdown_connections 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 1 -- 172.21.15.92:0/311979479 wait complete. 2024-09-21T02:14:51.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.049+0000 7fd9f481f740 20 client.18333 trim_cache size 0 max 16384 2024-09-21T02:14:51.055 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:51.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9d40b6590 msgr2=0x7fd9d40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9d40b6590 0x7fd9d40b8aa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00e6a70 tx=0x7fd9e0004a40 comp rx=0 tx=0).stop 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 msgr2=0x7fd9e819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 0x7fd9e819fdf0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000b970 tx=0x7fd9e000be30 comp rx=0 tx=0).stop 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 shutdown_connections 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd9d40b6590 0x7fd9d40b8aa0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9e8108d50 0x7fd9e819fdf0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9e8108400 0x7fd9e819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 --2- 172.21.15.92:0/986294934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9e8103190 0x7fd9e819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 >> 172.21.15.92:0/986294934 conn(0x7fd9e8079f90 msgr2=0x7fd9e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 shutdown_connections 2024-09-21T02:14:51.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:51.050+0000 7fd9ee2ab640 1 -- 172.21.15.92:0/986294934 wait complete. 2024-09-21T02:14:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:50 smithi099 ceph-mon[34404]: pgmap v656: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:14:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:50 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/3374128008' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:14:51.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:14:52.000 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 -- 172.21.15.92:0/3058516370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 msgr2=0x7f8e4810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 --2- 172.21.15.92:0/3058516370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4810d870 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009900 tx=0x7f8e3c02a410 comp rx=0 tx=0).stop 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 -- 172.21.15.92:0/3058516370 shutdown_connections 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 --2- 172.21.15.92:0/3058516370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e4810de30 0x7f8e48110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 --2- 172.21.15.92:0/3058516370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4810d870 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 --2- 172.21.15.92:0/3058516370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e481049d0 0x7f8e48104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 -- 172.21.15.92:0/3058516370 >> 172.21.15.92:0/3058516370 conn(0x7f8e48079740 msgr2=0x7f8e48079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 -- 172.21.15.92:0/3058516370 shutdown_connections 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.495+0000 7f8e5077c640 1 -- 172.21.15.92:0/3058516370 wait complete. 2024-09-21T02:14:52.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 Processor -- start 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 -- start start 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e481049d0 0x7f8e4819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e4810de30 0x7f8e4819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48113ae0 con 0x7f8e48105320 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48113960 con 0x7f8e4810de30 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e5077c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48113c60 con 0x7f8e481049d0 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e4dd17640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e4dd17640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49800/0 (socket says 172.21.15.92:49800) 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e4dd17640 1 -- 172.21.15.92:0/1542503183 learned_addr learned my addr 172.21.15.92:0/1542503183 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e4810de30 0x7f8e4819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.496+0000 7f8e4e518640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e481049d0 0x7f8e4819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e4dd17640 1 -- 172.21.15.92:0/1542503183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e481049d0 msgr2=0x7f8e4819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e481049d0 0x7f8e4819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e4dd17640 1 -- 172.21.15.92:0/1542503183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e4810de30 msgr2=0x7f8e4819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e4810de30 0x7f8e4819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e4dd17640 1 -- 172.21.15.92:0/1542503183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e481a03a0 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4819aa90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c031440 tx=0x7f8e3c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e477fe640 1 -- 172.21.15.92:0/1542503183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c03f070 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e477fe640 1 -- 172.21.15.92:0/1542503183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e3c031d60 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e481a0630 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e477fe640 1 -- 172.21.15.92:0/1542503183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c035030 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.497+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e481a0b10 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.499+0000 7f8e56cf0740 1 Processor -- start 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e477fe640 1 -- 172.21.15.92:0/1542503183 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8e3c042070 con 0x7f8e48105320 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 -- start start 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc836070 0x5607ccbdd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 0x5607ccbdd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5607ccb53ea0 con 0x5607cc80d1b0 2024-09-21T02:14:52.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5607ccb432d0 con 0x5607cc836070 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5607ccb470e0 con 0x5607cc815390 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e4e518640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 0x5607ccbdd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e4e518640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 0x5607ccbdd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49804/0 (socket says 172.21.15.92:49804) 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e4e518640 1 -- 172.21.15.92:0/2708692110 learned_addr learned my addr 172.21.15.92:0/2708692110 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc836070 0x5607ccbdd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.500+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4e518640 1 -- 172.21.15.92:0/2708692110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 msgr2=0x5607ccbddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4e518640 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbddd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4e518640 1 -- 172.21.15.92:0/2708692110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc836070 msgr2=0x5607ccbdd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4e518640 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc836070 0x5607ccbdd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4e518640 1 -- 172.21.15.92:0/2708692110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607cc8a1650 con 0x5607cc80d1b0 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e477fe640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8e300b6540 0x7f8e300b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4e518640 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 0x5607ccbdd800 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800af10 tx=0x7f8e3800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9733ee7fc810d7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8e300b6540 0x7f8e300b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e477fe640 1 -- 172.21.15.92:0/1542503183 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8e3c107b40 con 0x7f8e48105320 2024-09-21T02:14:52.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e44ff9640 1 -- 172.21.15.92:0/2708692110 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004d20 con 0x5607cc80d1b0 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e44ff9640 1 -- 172.21.15.92:0/2708692110 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e38010af0 con 0x5607cc80d1b0 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.501+0000 7f8e44ff9640 1 -- 172.21.15.92:0/2708692110 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38005720 con 0x5607cc80d1b0 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2708692110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 msgr2=0x5607ccbdd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8e300b6540 0x7f8e300b8a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000ac90 tx=0x7f8e4000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 0x5607ccbdd800 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800af10 tx=0x7f8e3800f790 comp rx=0 tx=0).stop 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2708692110 shutdown_connections 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc80d1b0 0x5607ccbdd800 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2708692110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc836070 0x5607ccbdd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2708692110 >> 172.21.15.92:0/2708692110 conn(0x5607cc82a460 msgr2=0x5607ccbdb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2708692110 shutdown_connections 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2708692110 wait complete. 2024-09-21T02:14:52.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.502+0000 7f8e56cf0740 1 -- start start 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 Processor -- start 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc80d1b0 0x5607ccbe88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc815390 0x5607ccbe8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbe9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5607cc9a4700 con 0x5607cc836070 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5607ccade160 con 0x5607cc815390 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5607ccbebaa0 con 0x5607cc80d1b0 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e56cf0740 10 client.0 _ll_register_callbacks cb 0x5607ccaf79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbe9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbe9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49808/0 (socket says 172.21.15.92:49808) 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 -- 172.21.15.92:0/2608026830 learned_addr learned my addr 172.21.15.92:0/2608026830 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc815390 0x5607ccbe8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4e518640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc80d1b0 0x5607ccbe88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 -- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc80d1b0 msgr2=0x5607ccbe88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc80d1b0 0x5607ccbe88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 -- 172.21.15.92:0/2608026830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc815390 msgr2=0x5607ccbe8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc815390 0x5607ccbe8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.503+0000 7f8e4ed19640 1 -- 172.21.15.92:0/2608026830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ccb0bc90 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbe9370 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8e4001cbe0 tx=0x7f8e40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943333ad8a411c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e2dffb640 1 -- 172.21.15.92:0/2608026830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000ada0 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e2dffb640 1 -- 172.21.15.92:0/2608026830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8e40016030 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e38010af0 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e2dffb640 1 -- 172.21.15.92:0/2608026830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e4001ccd0 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 10 client.18345 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607ccbf7760 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e2dffb640 1 -- 172.21.15.92:0/2608026830 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e38010af0 con 0x5607cc836070 2024-09-21T02:14:52.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e2dffb640 1 -- 172.21.15.92:0/2608026830 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e40005430 con 0x5607cc836070 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 10 client.18345 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 10 client.18345 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 20 client.18345 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7f49f0 0x5607ccbfad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 4 client.18345 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.504+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607ccb55df0 con 0x5607cc7f49f0 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.505+0000 7f8e4e518640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7f49f0 0x5607ccbfad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.505+0000 7f8e4e518640 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7f49f0 0x5607ccbfad40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73dec6c57e1e44b0 server_cookie=a2df8cbb79a554f5 in_seq=0 out_seq=0 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.505+0000 7f8e2dffb640 10 client.18345 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e2dffb640 1 -- 172.21.15.92:0/2608026830 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607ccb55df0 con 0x5607cc7f49f0 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e2dffb640 10 client.18345 handle_command_reply: tid=0 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 client.18345 shutdown 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 msgr2=0x5607ccbe9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbe9370 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8e4001cbe0 tx=0x7f8e40004730 comp rx=0 tx=0).stop 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 shutdown_connections 2024-09-21T02:14:52.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbe9370 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7f49f0 0x5607ccbfad40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc815390 0x5607ccbe8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/2608026830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc80d1b0 0x5607ccbe88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 >> 172.21.15.92:0/2608026830 conn(0x5607cc82a460 msgr2=0x5607ccbe0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e2dffb640 0 client.18345 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 shutdown_connections 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 1 -- 172.21.15.92:0/2608026830 wait complete. 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.506+0000 7f8e56cf0740 20 client.18345 trim_cache size 0 max 16384 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.519+0000 7f8e56cf0740 1 Processor -- start 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.519+0000 7f8e56cf0740 1 -- start start 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.519+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbdedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.546 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.519+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbf43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481af420 con 0x5607cc836070 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481b0f60 con 0x5607cc80d1b0 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481afaf0 con 0x5607cc815390 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbf43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbf43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49818/0 (socket says 172.21.15.92:49818) 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 -- 172.21.15.92:0/4050592024 learned_addr learned my addr 172.21.15.92:0/4050592024 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 -- 172.21.15.92:0/4050592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 msgr2=0x5607ccbea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 -- 172.21.15.92:0/4050592024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 msgr2=0x5607ccbdedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbdedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 -- 172.21.15.92:0/4050592024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ccbd5440 con 0x5607cc836070 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbf43c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005870 tx=0x7f8e40004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8f188078e997ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e2ffff640 1 -- 172.21.15.92:0/4050592024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40041070 con 0x5607cc836070 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e2ffff640 1 -- 172.21.15.92:0/4050592024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e4000ada0 con 0x5607cc836070 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.520+0000 7f8e2ffff640 1 -- 172.21.15.92:0/4050592024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40036070 con 0x5607cc836070 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 -- 172.21.15.92:0/4050592024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 msgr2=0x5607ccbf43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbf43c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005870 tx=0x7f8e40004640 comp rx=0 tx=0).stop 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 -- 172.21.15.92:0/4050592024 shutdown_connections 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc836070 0x5607ccbf43c0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc815390 0x5607ccbea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/4050592024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbdedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 -- 172.21.15.92:0/4050592024 >> 172.21.15.92:0/4050592024 conn(0x5607cc82a460 msgr2=0x5607ccbf3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 -- 172.21.15.92:0/4050592024 shutdown_connections 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 -- 172.21.15.92:0/4050592024 wait complete. 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 -- start start 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.521+0000 7f8e56cf0740 1 Processor -- start 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbf1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 0x5607ccbf1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc836070 0x5607ccbec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481b0890 con 0x5607cc815390 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481b01c0 con 0x5607cc80d1b0 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481ad400 con 0x5607cc836070 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 10 client.0 _ll_register_callbacks cb 0x7f8e4000a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 0x5607ccbf1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 0x5607ccbf1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49830/0 (socket says 172.21.15.92:49830) 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 -- 172.21.15.92:0/3824402921 learned_addr learned my addr 172.21.15.92:0/3824402921 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4e518640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbf1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 -- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc836070 msgr2=0x5607ccbec7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4ed19640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc836070 0x5607ccbec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc836070 0x5607ccbec7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 -- 172.21.15.92:0/3824402921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 msgr2=0x5607ccbf1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbf1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 -- 172.21.15.92:0/3824402921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ccbecf30 con 0x5607cc815390 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4e518640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbf1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e4dd17640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 0x5607ccbf1720 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c105620 tx=0x7f8e3c031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a27bce5252cafc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e2dffb640 1 -- 172.21.15.92:0/3824402921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c049040 con 0x5607cc815390 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e2dffb640 1 -- 172.21.15.92:0/3824402921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8e3c0d7240 con 0x5607cc815390 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.522+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e4000ada0 con 0x5607cc815390 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e2dffb640 1 -- 172.21.15.92:0/3824402921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c0cd6b0 con 0x5607cc815390 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 10 client.18351 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5607ccc652a0 con 0x5607cc815390 2024-09-21T02:14:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e2dffb640 1 -- 172.21.15.92:0/3824402921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e4000ada0 con 0x5607cc815390 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e2dffb640 1 -- 172.21.15.92:0/3824402921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e3c0f10b0 con 0x5607cc815390 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 10 client.18351 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 10 client.18351 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 20 client.18351 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7c9da0 0x5607ccc67a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 4 client.18351 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607ccb518a0 con 0x5607cc7c9da0 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.523+0000 7f8e4e518640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7c9da0 0x5607ccc67a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e2dffb640 10 client.18351 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e4e518640 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7c9da0 0x5607ccc67a60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15ea3995b12cbf7e server_cookie=ea04d968a0ab13e3 in_seq=0 out_seq=0 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e2dffb640 1 -- 172.21.15.92:0/3824402921 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607ccb518a0 con 0x5607cc7c9da0 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e2dffb640 10 client.18351 handle_command_reply: tid=0 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 client.18351 shutdown 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 msgr2=0x5607ccbf1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 0x5607ccbf1720 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c105620 tx=0x7f8e3c031f10 comp rx=0 tx=0).stop 2024-09-21T02:14:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 shutdown_connections 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5607cc836070 0x5607ccbec7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5607cc815390 0x5607ccbf1720 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x5607cc7c9da0 0x5607ccc67a60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 --2- 172.21.15.92:0/3824402921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607cc80d1b0 0x5607ccbf1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 >> 172.21.15.92:0/3824402921 conn(0x5607cc82a460 msgr2=0x5607ccbf3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e2dffb640 0 client.18351 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 shutdown_connections 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 1 -- 172.21.15.92:0/3824402921 wait complete. 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.524+0000 7f8e56cf0740 20 client.18351 trim_cache size 0 max 16384 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8e300b6540 msgr2=0x7f8e300b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8e300b6540 0x7f8e300b8a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000ac90 tx=0x7f8e4000a250 comp rx=0 tx=0).stop 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 msgr2=0x7f8e4819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4819aa90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c031440 tx=0x7f8e3c003d50 comp rx=0 tx=0).stop 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 shutdown_connections 2024-09-21T02:14:52.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f8e300b6540 0x7f8e300b8a50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e4810de30 0x7f8e4819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48105320 0x7f8e4819aa90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 --2- 172.21.15.92:0/1542503183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e481049d0 0x7f8e4819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 >> 172.21.15.92:0/1542503183 conn(0x7f8e48079740 msgr2=0x7f8e481007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 shutdown_connections 2024-09-21T02:14:52.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:52.525+0000 7f8e5077c640 1 -- 172.21.15.92:0/1542503183 wait complete. 2024-09-21T02:14:52.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:52 smithi092 ceph-mon[31190]: pgmap v657: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:53.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:52 smithi136 ceph-mon[34754]: pgmap v657: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:53.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:52 smithi099 ceph-mon[34404]: pgmap v657: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:53.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:14:53.547 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 -- 172.21.15.92:0/17038908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38108d10 msgr2=0x7fbe3810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 --2- 172.21.15.92:0/17038908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38108d10 0x7fbe3810f540 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009ca0 tx=0x7fbe3002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 -- 172.21.15.92:0/17038908 shutdown_connections 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 --2- 172.21.15.92:0/17038908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38108d10 0x7fbe3810f540 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 --2- 172.21.15.92:0/17038908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe381083c0 0x7fbe381087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 --2- 172.21.15.92:0/17038908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe38102a80 0x7fbe38102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 -- 172.21.15.92:0/17038908 >> 172.21.15.92:0/17038908 conn(0x7fbe38079740 msgr2=0x7fbe38079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 -- 172.21.15.92:0/17038908 shutdown_connections 2024-09-21T02:14:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 -- 172.21.15.92:0/17038908 wait complete. 2024-09-21T02:14:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.083+0000 7fbe405aa640 1 Processor -- start 2024-09-21T02:14:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 -- start start 2024-09-21T02:14:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 0x7fbe381994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe381083c0 0x7fbe38199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38108d10 0x7fbe3819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112980 con 0x7fbe38102a80 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112800 con 0x7fbe38108d10 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe405aa640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112b00 con 0x7fbe381083c0 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 0x7fbe381994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 0x7fbe381994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53622/0 (socket says 172.21.15.92:53622) 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 -- 172.21.15.92:0/3846497375 learned_addr learned my addr 172.21.15.92:0/3846497375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38108d10 0x7fbe3819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3db45640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe381083c0 0x7fbe38199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 -- 172.21.15.92:0/3846497375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe381083c0 msgr2=0x7fbe38199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe381083c0 0x7fbe38199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 -- 172.21.15.92:0/3846497375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38108d10 msgr2=0x7fbe3819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38108d10 0x7fbe3819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.084+0000 7fbe3e346640 1 -- 172.21.15.92:0/3846497375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3819f360 con 0x7fbe38102a80 2024-09-21T02:14:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.085+0000 7fbe3e346640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 0x7fbe381994b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c970 tx=0x7fbe3400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.085+0000 7fbe2f7fe640 1 -- 172.21.15.92:0/3846497375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34007bb0 con 0x7fbe38102a80 2024-09-21T02:14:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.085+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3819f650 con 0x7fbe38102a80 2024-09-21T02:14:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.085+0000 7fbe2f7fe640 1 -- 172.21.15.92:0/3846497375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe34007d50 con 0x7fbe38102a80 2024-09-21T02:14:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.085+0000 7fbe2f7fe640 1 -- 172.21.15.92:0/3846497375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34004de0 con 0x7fbe38102a80 2024-09-21T02:14:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.085+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381a5e70 con 0x7fbe38102a80 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.087+0000 7fbe46b1e740 1 Processor -- start 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.087+0000 7fbe2f7fe640 1 -- 172.21.15.92:0/3846497375 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fbe34004f80 con 0x7fbe38102a80 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 -- start start 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7478e070 0x55ef74b356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef747651b0 0x55ef74b35be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 0x55ef74b36120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ef74aabea0 con 0x55ef7476d390 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ef74a9b2d0 con 0x55ef7478e070 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ef74a9f0e0 con 0x55ef747651b0 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 0x55ef74b36120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 0x55ef74b36120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53634/0 (socket says 172.21.15.92:53634) 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 -- 172.21.15.92:0/1384452535 learned_addr learned my addr 172.21.15.92:0/1384452535 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3e346640 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef747651b0 0x55ef74b35be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7478e070 0x55ef74b356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe2f7fe640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbe240b6540 0x7fbe240b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 -- 172.21.15.92:0/1384452535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef747651b0 msgr2=0x55ef74b35be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef747651b0 0x55ef74b35be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 -- 172.21.15.92:0/1384452535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7478e070 msgr2=0x55ef74b356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7478e070 0x55ef74b356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.088+0000 7fbe3db45640 1 -- 172.21.15.92:0/1384452535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef747f9650 con 0x55ef7476d390 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.089+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbe240b6540 0x7fbe240b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.089+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 0x55ef74b36120 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ab70 tx=0x7fbe2800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a13386dc0e488 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.089+0000 7fbe2f7fe640 1 -- 172.21.15.92:0/3846497375 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbe340e6f40 con 0x7fbe38102a80 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.089+0000 7fbe2cff9640 1 -- 172.21.15.92:0/1384452535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2801d070 con 0x55ef7476d390 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.089+0000 7fbe2cff9640 1 -- 172.21.15.92:0/1384452535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe280188c0 con 0x55ef7476d390 2024-09-21T02:14:54.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.089+0000 7fbe2cff9640 1 -- 172.21.15.92:0/1384452535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280179a0 con 0x55ef7476d390 2024-09-21T02:14:54.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1384452535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 msgr2=0x55ef74b36120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbe240b6540 0x7fbe240b8a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098d0 tx=0x7fbe300095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 0x55ef74b36120 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ab70 tx=0x7fbe2800f920 comp rx=0 tx=0).stop 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1384452535 shutdown_connections 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef7476d390 0x55ef74b36120 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef747651b0 0x55ef74b35be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1384452535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7478e070 0x55ef74b356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1384452535 >> 172.21.15.92:0/1384452535 conn(0x55ef74782460 msgr2=0x55ef74b33590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1384452535 shutdown_connections 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1384452535 wait complete. 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.090+0000 7fbe46b1e740 1 -- start start 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 Processor -- start 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b40d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7476d390 0x55ef74b412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7478e070 0x55ef74b4d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ef748fc700 con 0x55ef747651b0 2024-09-21T02:14:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ef74a36160 con 0x55ef7476d390 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ef74b43e80 con 0x55ef7478e070 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe46b1e740 10 client.0 _ll_register_callbacks cb 0x55ef74a4f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b40d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b40d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53638/0 (socket says 172.21.15.92:53638) 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 -- 172.21.15.92:0/2303288198 learned_addr learned my addr 172.21.15.92:0/2303288198 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3db45640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7476d390 0x55ef74b412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7478e070 0x55ef74b4d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 -- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7478e070 msgr2=0x55ef74b4d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7478e070 0x55ef74b4d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 -- 172.21.15.92:0/2303288198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7476d390 msgr2=0x55ef74b412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7476d390 0x55ef74b412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3e346640 1 -- 172.21.15.92:0/2303288198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef74a63c90 con 0x55ef747651b0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3db45640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7476d390 0x55ef74b412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.091+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7478e070 0x55ef74b4d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe3e346640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b40d80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbe340058c0 tx=0x7fbe34005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58bb02e558b2e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe21ffb640 1 -- 172.21.15.92:0/2303288198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3401d040 con 0x55ef747651b0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe21ffb640 1 -- 172.21.15.92:0/2303288198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbe340a8c00 con 0x55ef747651b0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe280188c0 con 0x55ef747651b0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe21ffb640 1 -- 172.21.15.92:0/2303288198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe340ace80 con 0x55ef747651b0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 10 client.18363 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef74b4fbf0 con 0x55ef747651b0 2024-09-21T02:14:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe21ffb640 1 -- 172.21.15.92:0/2303288198 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe280188c0 con 0x55ef747651b0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe21ffb640 1 -- 172.21.15.92:0/2303288198 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbe340980d0 con 0x55ef747651b0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 10 client.18363 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 10 client.18363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 20 client.18363 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7474c9f0 0x55ef74b530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 4 client.18363 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.092+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef74aaddf0 con 0x55ef7474c9f0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.093+0000 7fbe3db45640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7474c9f0 0x55ef74b530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.093+0000 7fbe21ffb640 10 client.18363 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.093+0000 7fbe3db45640 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7474c9f0 0x55ef74b530f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1efd7ed3ad2d86b9 server_cookie=d111bb8682196f52 in_seq=0 out_seq=0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe21ffb640 1 -- 172.21.15.92:0/2303288198 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef74aaddf0 con 0x55ef7474c9f0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe21ffb640 10 client.18363 handle_command_reply: tid=0 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 client.18363 shutdown 2024-09-21T02:14:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 msgr2=0x55ef74b40d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b40d80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbe340058c0 tx=0x7fbe34005640 comp rx=0 tx=0).stop 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 shutdown_connections 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7474c9f0 0x55ef74b530f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7478e070 0x55ef74b4d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7476d390 0x55ef74b412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe21ffb640 0 client.18363 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/2303288198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b40d80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 >> 172.21.15.92:0/2303288198 conn(0x55ef74782460 msgr2=0x55ef74b38830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 shutdown_connections 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 1 -- 172.21.15.92:0/2303288198 wait complete. 2024-09-21T02:14:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.094+0000 7fbe46b1e740 20 client.18363 trim_cache size 0 max 16384 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 Processor -- start 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 -- start start 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b41300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b42c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381ad1d0 con 0x55ef747651b0 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381aed10 con 0x55ef7474c9f0 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381ad8a0 con 0x55ef7476d390 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe3db45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b42c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.107+0000 7fbe3db45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b42c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53650/0 (socket says 172.21.15.92:53650) 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 -- 172.21.15.92:0/1900181702 learned_addr learned my addr 172.21.15.92:0/1900181702 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 -- 172.21.15.92:0/1900181702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 msgr2=0x55ef74b36de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 -- 172.21.15.92:0/1900181702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 msgr2=0x55ef74b41300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b41300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 -- 172.21.15.92:0/1900181702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef74b43360 con 0x55ef747651b0 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b42c60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800fe20 tx=0x7fbe28005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd3d9a7dd9e93d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe23fff640 1 -- 172.21.15.92:0/1900181702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2801d040 con 0x55ef747651b0 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe23fff640 1 -- 172.21.15.92:0/1900181702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe2801e040 con 0x55ef747651b0 2024-09-21T02:14:54.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.108+0000 7fbe23fff640 1 -- 172.21.15.92:0/1900181702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28017930 con 0x55ef747651b0 2024-09-21T02:14:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1900181702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 msgr2=0x55ef74b42c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b42c60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800fe20 tx=0x7fbe28005670 comp rx=0 tx=0).stop 2024-09-21T02:14:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1900181702 shutdown_connections 2024-09-21T02:14:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b36de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b42c60 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1900181702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b41300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1900181702 >> 172.21.15.92:0/1900181702 conn(0x55ef74782460 msgr2=0x55ef74b366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1900181702 shutdown_connections 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1900181702 wait complete. 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 -- start start 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.109+0000 7fbe46b1e740 1 Processor -- start 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b45cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381ae640 con 0x55ef747651b0 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381adf70 con 0x55ef7474c9f0 2024-09-21T02:14:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381ab1b0 con 0x55ef7476d390 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 10 client.0 _ll_register_callbacks cb 0x7fbe3400af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b45780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53654/0 (socket says 172.21.15.92:53654) 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 -- 172.21.15.92:0/1348665713 learned_addr learned my addr 172.21.15.92:0/1348665713 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3eb47640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b45cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3e346640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 -- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 msgr2=0x55ef74b45cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b45cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 -- 172.21.15.92:0/1348665713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 msgr2=0x55ef74b45240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b45240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 -- 172.21.15.92:0/1348665713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef74bbcc10 con 0x55ef747651b0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe3db45640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b45780 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fbe280057c0 tx=0x7fbe28017760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4574736caabb30b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe21ffb640 1 -- 172.21.15.92:0/1348665713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2801d030 con 0x55ef747651b0 2024-09-21T02:14:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe21ffb640 1 -- 172.21.15.92:0/1348665713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbe28017b60 con 0x55ef747651b0 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.110+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2801e040 con 0x55ef747651b0 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe21ffb640 1 -- 172.21.15.92:0/1348665713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe28017d60 con 0x55ef747651b0 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 10 client.18369 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef74bbde90 con 0x55ef747651b0 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe21ffb640 1 -- 172.21.15.92:0/1348665713 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe2801e040 con 0x55ef747651b0 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe21ffb640 1 -- 172.21.15.92:0/1348665713 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbe28025580 con 0x55ef747651b0 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 10 client.18369 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 10 client.18369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 20 client.18369 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7478e070 0x55ef74bc0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 4 client.18369 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef74aa98a0 con 0x55ef7478e070 2024-09-21T02:14:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.111+0000 7fbe3e346640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7478e070 0x55ef74bc0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe21ffb640 10 client.18369 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe3e346640 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7478e070 0x55ef74bc0480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42e6a6d1de0ab847 server_cookie=e0bb818ee9831758 in_seq=0 out_seq=0 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe21ffb640 1 -- 172.21.15.92:0/1348665713 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef74aa98a0 con 0x55ef7478e070 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe21ffb640 10 client.18369 handle_command_reply: tid=0 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 client.18369 shutdown 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 msgr2=0x55ef74b45780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b45780 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fbe280057c0 tx=0x7fbe28017760 comp rx=0 tx=0).stop 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 shutdown_connections 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55ef7478e070 0x55ef74bc0480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ef7476d390 0x55ef74b45cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef747651b0 0x55ef74b45780 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe21ffb640 0 client.18369 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 --2- 172.21.15.92:0/1348665713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ef7474c9f0 0x55ef74b45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 >> 172.21.15.92:0/1348665713 conn(0x55ef74782460 msgr2=0x55ef74b429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 shutdown_connections 2024-09-21T02:14:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 1 -- 172.21.15.92:0/1348665713 wait complete. 2024-09-21T02:14:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.112+0000 7fbe46b1e740 20 client.18369 trim_cache size 0 max 16384 2024-09-21T02:14:54.118 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbe240b6540 msgr2=0x7fbe240b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbe240b6540 0x7fbe240b8a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe300098d0 tx=0x7fbe300095c0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 msgr2=0x7fbe381994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 0x7fbe381994b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c970 tx=0x7fbe3400cd70 comp rx=0 tx=0).stop 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 shutdown_connections 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbe240b6540 0x7fbe240b8a50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38108d10 0x7fbe3819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe381083c0 0x7fbe38199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 --2- 172.21.15.92:0/3846497375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38102a80 0x7fbe381994b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 >> 172.21.15.92:0/3846497375 conn(0x7fbe38079740 msgr2=0x7fbe381044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 shutdown_connections 2024-09-21T02:14:54.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:54.113+0000 7fbe405aa640 1 -- 172.21.15.92:0/3846497375 wait complete. 2024-09-21T02:14:54.776 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:54 smithi092 ceph-mon[31190]: pgmap v658: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:14:54.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:14:55.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:54 smithi136 ceph-mon[34754]: pgmap v658: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:14:55.061 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:55.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:54 smithi099 ceph-mon[34404]: pgmap v658: 97 pgs: 97 active+clean; 3.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:14:55.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 -- 172.21.15.92:0/1256139955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108d70 msgr2=0x7fe13410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1256139955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108d70 0x7fe13410f5a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009ca0 tx=0x7fe12c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 -- 172.21.15.92:0/1256139955 shutdown_connections 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1256139955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108d70 0x7fe13410f5a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1256139955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134108420 0x7fe134108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1256139955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe134102ae0 0x7fe134102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.568+0000 7fe13a53c640 1 -- 172.21.15.92:0/1256139955 >> 172.21.15.92:0/1256139955 conn(0x7fe134079770 msgr2=0x7fe134079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.569+0000 7fe13a53c640 1 -- 172.21.15.92:0/1256139955 shutdown_connections 2024-09-21T02:14:55.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.569+0000 7fe13a53c640 1 -- 172.21.15.92:0/1256139955 wait complete. 2024-09-21T02:14:55.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.569+0000 7fe13a53c640 1 Processor -- start 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.569+0000 7fe13a53c640 1 -- start start 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe13a53c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134102ae0 0x7fe13419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe13a53c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 0x7fe13419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe13a53c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe134108d70 0x7fe13419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe13a53c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe134112a30 con 0x7fe134108420 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe13a53c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341128b0 con 0x7fe134102ae0 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe13a53c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe134112bb0 con 0x7fe134108d70 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 0x7fe13419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 0x7fe13419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53660/0 (socket says 172.21.15.92:53660) 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 -- 172.21.15.92:0/1806796180 learned_addr learned my addr 172.21.15.92:0/1806796180 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe133fff640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134102ae0 0x7fe13419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe138ad9640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe134108d70 0x7fe13419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 -- 172.21.15.92:0/1806796180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe134108d70 msgr2=0x7fe13419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe134108d70 0x7fe13419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 -- 172.21.15.92:0/1806796180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134102ae0 msgr2=0x7fe13419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134102ae0 0x7fe13419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 -- 172.21.15.92:0/1806796180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1341a0420 con 0x7fe134108420 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.570+0000 7fe1337fe640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 0x7fe13419aa50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe12800e9f0 tx=0x7fe12800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.571+0000 7fe1317fa640 1 -- 172.21.15.92:0/1806796180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12800ccc0 con 0x7fe134108420 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.571+0000 7fe1317fa640 1 -- 172.21.15.92:0/1806796180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1280108e0 con 0x7fe134108420 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.571+0000 7fe1317fa640 1 -- 172.21.15.92:0/1806796180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128018860 con 0x7fe134108420 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.571+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1341a0710 con 0x7fe134108420 2024-09-21T02:14:55.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.571+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1341a6e70 con 0x7fe134108420 2024-09-21T02:14:55.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.573+0000 7fe140ab0740 1 Processor -- start 2024-09-21T02:14:55.574 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.573+0000 7fe140ab0740 1 -- start start 2024-09-21T02:14:55.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc62e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.575 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc62ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe140ab0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5605fc5a4ea0 con 0x5605fc266390 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe140ab0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5605fc5942d0 con 0x5605fc287070 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe140ab0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5605fc5980e0 con 0x5605fc25e1b0 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1317fa640 1 -- 172.21.15.92:0/1806796180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fe128010430 con 0x7fe134108420 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe133fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc62ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53676/0 (socket says 172.21.15.92:53676) 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 -- 172.21.15.92:0/1054544757 learned_addr learned my addr 172.21.15.92:0/1054544757 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe138ad9640 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc62e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 -- 172.21.15.92:0/1054544757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 msgr2=0x5605fc62ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc62ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 -- 172.21.15.92:0/1054544757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 msgr2=0x5605fc62e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc62e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.574+0000 7fe1337fe640 1 -- 172.21.15.92:0/1054544757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605fc2f2650 con 0x5605fc266390 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe1337fe640 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62f120 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe1280e4ea0 tx=0x7fe1280edab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3148de1c08b93df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe1317fa640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe11c0b6590 0x7fe11c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe11a7fc640 1 -- 172.21.15.92:0/1054544757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280fa070 con 0x5605fc266390 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe138ad9640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe11c0b6590 0x7fe11c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe11a7fc640 1 -- 172.21.15.92:0/1054544757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe128010720 con 0x5605fc266390 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe11a7fc640 1 -- 172.21.15.92:0/1054544757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280ad470 con 0x5605fc266390 2024-09-21T02:14:55.577 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.575+0000 7fe1317fa640 1 -- 172.21.15.92:0/1806796180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe1280e7480 con 0x7fe134108420 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 -- 172.21.15.92:0/1054544757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 msgr2=0x5605fc62f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe138ad9640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe11c0b6590 0x7fe11c0b8aa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe12c02f500 tx=0x7fe12c02f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62f120 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe1280e4ea0 tx=0x7fe1280edab0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 -- 172.21.15.92:0/1054544757 shutdown_connections 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62f120 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc62ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 --2- 172.21.15.92:0/1054544757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc62e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 -- 172.21.15.92:0/1054544757 >> 172.21.15.92:0/1054544757 conn(0x5605fc27b460 msgr2=0x5605fc62c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 -- 172.21.15.92:0/1054544757 shutdown_connections 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 -- 172.21.15.92:0/1054544757 wait complete. 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.576+0000 7fe140ab0740 1 -- start start 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 Processor -- start 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 0x5605fc639d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc266390 0x5605fc63a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc64a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5605fc3f5700 con 0x5605fc25e1b0 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5605fc52f160 con 0x5605fc287070 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5605fc641290 con 0x5605fc266390 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe140ab0740 10 client.0 _ll_register_callbacks cb 0x5605fc5489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 0x5605fc639d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 0x5605fc639d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53682/0 (socket says 172.21.15.92:53682) 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 -- 172.21.15.92:0/748260802 learned_addr learned my addr 172.21.15.92:0/748260802 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe1337fe640 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc266390 0x5605fc63a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 -- 172.21.15.92:0/748260802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc266390 msgr2=0x5605fc63a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc266390 0x5605fc63a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 -- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 msgr2=0x5605fc64a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc64a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 -- 172.21.15.92:0/748260802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605fc55cc90 con 0x5605fc25e1b0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.577+0000 7fe133fff640 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 0x5605fc639d10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe12400bfc0 tx=0x7fe1240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6b2b9cf84fb80d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe1137fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe1137fe640 1 -- 172.21.15.92:0/748260802 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124004020 con 0x5605fc25e1b0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe1137fe640 1 -- 172.21.15.92:0/748260802 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe1240041c0 con 0x5605fc25e1b0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605fc5a6df0 con 0x5605fc25e1b0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe1137fe640 1 -- 172.21.15.92:0/748260802 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1240043c0 con 0x5605fc25e1b0 2024-09-21T02:14:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 10 client.18381 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605fc64cfc0 con 0x5605fc25e1b0 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe1137fe640 1 -- 172.21.15.92:0/748260802 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5605fc5a6df0 con 0x5605fc25e1b0 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe1137fe640 1 -- 172.21.15.92:0/748260802 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe124003a60 con 0x5605fc25e1b0 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 10 client.18381 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 10 client.18381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 20 client.18381 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc2459f0 0x5605fc6505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 4 client.18381 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.578+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605fc5a28a0 con 0x5605fc2459f0 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.579+0000 7fe1337fe640 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc2459f0 0x5605fc6505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.579+0000 7fe1137fe640 10 client.18381 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:55.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.579+0000 7fe1337fe640 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc2459f0 0x5605fc6505e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7f821dfc71286b6 server_cookie=347b182a28f5c3d in_seq=0 out_seq=0 2024-09-21T02:14:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe1137fe640 1 -- 172.21.15.92:0/748260802 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605fc5a28a0 con 0x5605fc2459f0 2024-09-21T02:14:55.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe1137fe640 10 client.18381 handle_command_reply: tid=0 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 client.18381 shutdown 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 msgr2=0x5605fc639d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 0x5605fc639d10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe12400bfc0 tx=0x7fe1240118d0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 shutdown_connections 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc64a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc266390 0x5605fc63a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc2459f0 0x5605fc6505e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 --2- 172.21.15.92:0/748260802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc25e1b0 0x5605fc639d10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 >> 172.21.15.92:0/748260802 conn(0x5605fc27b460 msgr2=0x5605fc631830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe1137fe640 0 client.18381 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 shutdown_connections 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 1 -- 172.21.15.92:0/748260802 wait complete. 2024-09-21T02:14:55.585 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.580+0000 7fe140ab0740 20 client.18381 trim_cache size 0 max 16384 2024-09-21T02:14:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 Processor -- start 2024-09-21T02:14:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 -- start start 2024-09-21T02:14:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc25e1b0 0x5605fc6354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc287070 0x5605fc64a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ad290 con 0x5605fc266390 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341aedd0 con 0x5605fc25e1b0 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.593+0000 7fe140ab0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ad960 con 0x5605fc287070 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53692/0 (socket says 172.21.15.92:53692) 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 -- 172.21.15.92:0/3490774701 learned_addr learned my addr 172.21.15.92:0/3490774701 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe138ad9640 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc287070 0x5605fc64a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe133fff640 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc25e1b0 0x5605fc6354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 -- 172.21.15.92:0/3490774701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc287070 msgr2=0x5605fc64a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc287070 0x5605fc64a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 -- 172.21.15.92:0/3490774701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc25e1b0 msgr2=0x5605fc6354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc25e1b0 0x5605fc6354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 -- 172.21.15.92:0/3490774701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605fc245c20 con 0x5605fc266390 2024-09-21T02:14:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe138ad9640 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc287070 0x5605fc64a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe1337fe640 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62fde0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c6a0 tx=0x7fe1280096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdb0c5964a41f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe119ffb640 1 -- 172.21.15.92:0/3490774701 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280511e0 con 0x5605fc266390 2024-09-21T02:14:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe119ffb640 1 -- 172.21.15.92:0/3490774701 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1280ad1b0 con 0x5605fc266390 2024-09-21T02:14:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.594+0000 7fe119ffb640 1 -- 172.21.15.92:0/3490774701 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280513b0 con 0x5605fc266390 2024-09-21T02:14:55.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 -- 172.21.15.92:0/3490774701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 msgr2=0x5605fc62fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62fde0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c6a0 tx=0x7fe1280096a0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 -- 172.21.15.92:0/3490774701 shutdown_connections 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc287070 0x5605fc64a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc62fde0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 --2- 172.21.15.92:0/3490774701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc25e1b0 0x5605fc6354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 -- 172.21.15.92:0/3490774701 >> 172.21.15.92:0/3490774701 conn(0x5605fc27b460 msgr2=0x5605fc62f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 -- 172.21.15.92:0/3490774701 shutdown_connections 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.595+0000 7fe140ab0740 1 -- 172.21.15.92:0/3490774701 wait complete. 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 -- start start 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 Processor -- start 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc641fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc6425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc6b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ae700 con 0x5605fc266390 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ae030 con 0x5605fc287070 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ab270 con 0x5605fc25e1b0 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe140ab0740 10 client.0 _ll_register_callbacks cb 0x7fe12400bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc6425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc6425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53708/0 (socket says 172.21.15.92:53708) 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe1337fe640 1 -- 172.21.15.92:0/318182213 learned_addr learned my addr 172.21.15.92:0/318182213 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.596+0000 7fe133fff640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc641fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe138ad9640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc6b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1337fe640 1 -- 172.21.15.92:0/318182213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 msgr2=0x5605fc641fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1337fe640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc641fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1337fe640 1 -- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 msgr2=0x5605fc6b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1337fe640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc6b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1337fe640 1 -- 172.21.15.92:0/318182213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605fc6b3710 con 0x5605fc266390 2024-09-21T02:14:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1337fe640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc6425a0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe12804f2d0 tx=0x7fe12804f260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35c372308e19e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1137fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1137fe640 1 -- 172.21.15.92:0/318182213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1280515a0 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1137fe640 1 -- 172.21.15.92:0/318182213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe1280510d0 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1280ad1b0 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1137fe640 1 -- 172.21.15.92:0/318182213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe12804f0c0 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe140ab0740 10 client.18387 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5605fc6bab00 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.597+0000 7fe1137fe640 1 -- 172.21.15.92:0/318182213 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1280ad1b0 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe1137fe640 1 -- 172.21.15.92:0/318182213 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe1280a9910 con 0x5605fc266390 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe140ab0740 10 client.18387 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe140ab0740 10 client.18387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe140ab0740 20 client.18387 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe140ab0740 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc21ada0 0x5605fc6bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe140ab0740 4 client.18387 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605fc59fcd0 con 0x5605fc21ada0 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe133fff640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc21ada0 0x5605fc6bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe1137fe640 10 client.18387 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.598+0000 7fe133fff640 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc21ada0 0x5605fc6bcd90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=feada04bf73b1157 server_cookie=fa28f61345c6eee0 in_seq=0 out_seq=0 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe1137fe640 1 -- 172.21.15.92:0/318182213 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605fc59fcd0 con 0x5605fc21ada0 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe1137fe640 10 client.18387 handle_command_reply: tid=0 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 client.18387 shutdown 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 msgr2=0x5605fc6425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc6425a0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe12804f2d0 tx=0x7fe12804f260 comp rx=0 tx=0).stop 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 shutdown_connections 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605fc287070 0x5605fc6b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x5605fc21ada0 0x5605fc6bcd90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5605fc266390 0x5605fc6425a0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 --2- 172.21.15.92:0/318182213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5605fc25e1b0 0x5605fc641fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe1137fe640 0 client.18387 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:14:55.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 >> 172.21.15.92:0/318182213 conn(0x5605fc27b460 msgr2=0x5605fc63b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 shutdown_connections 2024-09-21T02:14:55.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 1 -- 172.21.15.92:0/318182213 wait complete. 2024-09-21T02:14:55.605 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.599+0000 7fe140ab0740 20 client.18387 trim_cache size 0 max 16384 2024-09-21T02:14:55.605 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe11c0b6590 msgr2=0x7fe11c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe11c0b6590 0x7fe11c0b8aa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe12c02f500 tx=0x7fe12c02f310 comp rx=0 tx=0).stop 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 msgr2=0x7fe13419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 0x7fe13419aa50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe12800e9f0 tx=0x7fe12800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 shutdown_connections 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fe11c0b6590 0x7fe11c0b8aa0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe134108d70 0x7fe13419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe134108420 0x7fe13419aa50 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 --2- 172.21.15.92:0/1806796180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134102ae0 0x7fe13419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 >> 172.21.15.92:0/1806796180 conn(0x7fe134079770 msgr2=0x7fe134101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 shutdown_connections 2024-09-21T02:14:55.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:55.600+0000 7fe13a53c640 1 -- 172.21.15.92:0/1806796180 wait complete. 2024-09-21T02:14:56.210 INFO:tasks.workunit.client.0.smithi092.stdout:499/600: open fd = 3 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:500/600: open fd = 3 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:501/600: open fd = 3 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:502/600: open fd = 3 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.211 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:503/600: open fd = 3 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:504/600: open fd = 3 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:505/600: open fd = 3 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:506/600: open fd = 3 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.212 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:507/600: open fd = 3 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:508/600: open fd = 3 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:509/600: open fd = 3 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.213 INFO:tasks.workunit.client.0.smithi092.stdout:510/600: open fd = 3 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:511/600: open fd = 3 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:512/600: open fd = 3 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:513/600: open fd = 3 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.214 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:514/600: open fd = 3 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:515/600: open fd = 3 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:516/600: open fd = 3 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:517/600: open fd = 3 2024-09-21T02:14:56.215 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:518/600: open fd = 3 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:519/600: open fd = 3 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:520/600: open fd = 3 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.216 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:521/600: open fd = 3 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:522/600: open fd = 3 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:523/600: open fd = 3 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:524/600: open fd = 3 2024-09-21T02:14:56.217 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:525/600: open fd = 3 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:526/600: open fd = 3 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:527/600: open fd = 3 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.218 INFO:tasks.workunit.client.0.smithi092.stdout:528/600: open fd = 3 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:529/600: open fd = 3 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:530/600: open fd = 3 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:531/600: open fd = 3 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.219 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:532/600: open fd = 3 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:533/600: open fd = 3 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:534/600: open fd = 3 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:535/600: open fd = 3 2024-09-21T02:14:56.220 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:536/600: open fd = 3 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:537/600: open fd = 3 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:538/600: open fd = 3 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:539/600: open fd = 3 2024-09-21T02:14:56.221 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:540/600: open fd = 3 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:541/600: open fd = 3 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:542/600: open fd = 3 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.222 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:543/600: open fd = 3 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:544/600: open fd = 3 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:545/600: open fd = 3 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:546/600: open fd = 3 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.223 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:547/600: open fd = 3 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:548/600: open fd = 3 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:549/600: open fd = 3 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.224 INFO:tasks.workunit.client.0.smithi092.stdout:550/600: open fd = 3 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:551/600: open fd = 3 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:552/600: open fd = 3 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.225 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:553/600: open fd = 3 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:554/600: open fd = 3 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.226 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:555/600: open fd = 3 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:556/600: open fd = 3 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.227 INFO:tasks.workunit.client.0.smithi092.stdout:557/600: open fd = 3 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:558/600: open fd = 3 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:559/600: open fd = 3 2024-09-21T02:14:56.228 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.229 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.229 INFO:tasks.workunit.client.0.smithi092.stdout:560/600: open fd = 3 2024-09-21T02:14:56.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.229 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:14:56.229 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:14:56.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:14:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:56.937 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:56 smithi092 ceph-mon[31190]: pgmap v659: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:14:57.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:56 smithi136 ceph-mon[34754]: pgmap v659: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:14:57.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 -- 172.21.15.92:0/3137962229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044106c20 msgr2=0x7fb044110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 --2- 172.21.15.92:0/3137962229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044106c20 0x7fb044110670 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009900 tx=0x7fb03c02a410 comp rx=0 tx=0).stop 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 -- 172.21.15.92:0/3137962229 shutdown_connections 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 --2- 172.21.15.92:0/3137962229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb044106c20 0x7fb044110670 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 --2- 172.21.15.92:0/3137962229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0441062d0 0x7fb0441066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 --2- 172.21.15.92:0/3137962229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb044101ff0 0x7fb044102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 -- 172.21.15.92:0/3137962229 >> 172.21.15.92:0/3137962229 conn(0x7fb044079e30 msgr2=0x7fb04407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 -- 172.21.15.92:0/3137962229 shutdown_connections 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.029+0000 7fb048c7c640 1 -- 172.21.15.92:0/3137962229 wait complete. 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 Processor -- start 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 -- start start 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb044101ff0 0x7fb04419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 0x7fb04419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb044106c20 0x7fb04419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb044113ac0 con 0x7fb0441062d0 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb044113940 con 0x7fb044101ff0 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb048c7c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb044113c40 con 0x7fb044106c20 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb041d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 0x7fb04419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb041d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 0x7fb04419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53716/0 (socket says 172.21.15.92:53716) 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.030+0000 7fb041d9b640 1 -- 172.21.15.92:0/1600540630 learned_addr learned my addr 172.21.15.92:0/1600540630 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb04259c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb044101ff0 0x7fb04419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb041d9b640 1 -- 172.21.15.92:0/1600540630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb044106c20 msgr2=0x7fb04419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb042d9d640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb044106c20 0x7fb04419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb041d9b640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb044106c20 0x7fb04419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb041d9b640 1 -- 172.21.15.92:0/1600540630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb044101ff0 msgr2=0x7fb04419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb041d9b640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb044101ff0 0x7fb04419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb041d9b640 1 -- 172.21.15.92:0/1600540630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0441a02a0 con 0x7fb0441062d0 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb04259c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb044101ff0 0x7fb04419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:57.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb041d9b640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 0x7fb04419a990 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c970 tx=0x7fb03800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb0337fe640 1 -- 172.21.15.92:0/1600540630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038014070 con 0x7fb0441062d0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb0337fe640 1 -- 172.21.15.92:0/1600540630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb038004440 con 0x7fb0441062d0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0441a0590 con 0x7fb0441062d0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb0337fe640 1 -- 172.21.15.92:0/1600540630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038003e50 con 0x7fb0441062d0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.031+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a6db0 con 0x7fb0441062d0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb0337fe640 1 -- 172.21.15.92:0/1600540630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fb038017070 con 0x7fb0441062d0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 Processor -- start 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 -- start start 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533534be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533535120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5655334aaea0 con 0x56553318d070 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56553349a2d0 con 0x5655331641b0 2024-09-21T02:14:57.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.034+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56553349e0e0 con 0x56553316c390 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53722/0 (socket says 172.21.15.92:53722) 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 -- 172.21.15.92:0/2495265208 learned_addr learned my addr 172.21.15.92:0/2495265208 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb041d9b640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533535120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb04259c640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533534be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb0337fe640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb02c0b66e0 0x7fb02c0b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 -- 172.21.15.92:0/2495265208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 msgr2=0x565533535120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533535120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 -- 172.21.15.92:0/2495265208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 msgr2=0x565533534be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533534be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 -- 172.21.15.92:0/2495265208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655331f8650 con 0x56553318d070 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb04259c640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533534be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb041d9b640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533535120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:57.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb02c0b66e0 0x7fb02c0b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb0337fe640 1 -- 172.21.15.92:0/1600540630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb0380e6ef0 con 0x7fb0441062d0 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb042d9d640 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335346a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005940 tx=0x7fb03c032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a55d3c6e833bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.035+0000 7fb030ff9640 1 -- 172.21.15.92:0/2495265208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c035070 con 0x56553318d070 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb030ff9640 1 -- 172.21.15.92:0/2495265208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c005570 con 0x56553318d070 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb030ff9640 1 -- 172.21.15.92:0/2495265208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004b20 con 0x56553318d070 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2495265208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 msgr2=0x5655335346a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb042d9d640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb02c0b66e0 0x7fb02c0b8bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb03c044000 tx=0x7fb03c045000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335346a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005940 tx=0x7fb03c032db0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2495265208 shutdown_connections 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533535120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533534be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2495265208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335346a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.036+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2495265208 >> 172.21.15.92:0/2495265208 conn(0x565533181460 msgr2=0x565533532590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2495265208 shutdown_connections 2024-09-21T02:14:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2495265208 wait complete. 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 -- start start 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 Processor -- start 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x56553353fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 0x565533540280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553318d070 0x56553354c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5655332fb700 con 0x56553316c390 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565533435160 con 0x5655331641b0 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565533542e80 con 0x56553318d070 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb04f1f0740 10 client.0 _ll_register_callbacks cb 0x56553344e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.037+0000 7fb041d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 0x565533540280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 0x565533540280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53726/0 (socket says 172.21.15.92:53726) 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 -- 172.21.15.92:0/4169738850 learned_addr learned my addr 172.21.15.92:0/4169738850 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb042d9d640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553318d070 0x56553354c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb04259c640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x56553353fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 -- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553318d070 msgr2=0x56553354c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553318d070 0x56553354c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 -- 172.21.15.92:0/4169738850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 msgr2=0x56553353fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x56553353fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 -- 172.21.15.92:0/4169738850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565533462c90 con 0x56553316c390 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb041d9b640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 0x565533540280 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb0380a9630 tx=0x7fb0380038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a438390d3d74da65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb029ffb640 1 -- 172.21.15.92:0/4169738850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380ac4c0 con 0x56553316c390 2024-09-21T02:14:57.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb029ffb640 1 -- 172.21.15.92:0/4169738850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb0380a89b0 con 0x56553316c390 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03c005570 con 0x56553316c390 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb029ffb640 1 -- 172.21.15.92:0/4169738850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0380ad760 con 0x56553316c390 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb04f1f0740 10 client.18399 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.038+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56553354ebb0 con 0x56553316c390 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb029ffb640 1 -- 172.21.15.92:0/4169738850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c005570 con 0x56553316c390 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb029ffb640 1 -- 172.21.15.92:0/4169738850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb0380abdc0 con 0x56553316c390 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04f1f0740 10 client.18399 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04f1f0740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04f1f0740 20 client.18399 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56553314b9f0 0x5655335520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04f1f0740 4 client.18399 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655334acdf0 con 0x56553314b9f0 2024-09-21T02:14:57.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.039+0000 7fb04259c640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56553314b9f0 0x5655335520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.040+0000 7fb029ffb640 10 client.18399 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.040+0000 7fb04259c640 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56553314b9f0 0x5655335520c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75348323404ad33b server_cookie=f371f5502f7ca4f9 in_seq=0 out_seq=0 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.040+0000 7fb029ffb640 1 -- 172.21.15.92:0/4169738850 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5655334acdf0 con 0x56553314b9f0 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.040+0000 7fb029ffb640 10 client.18399 handle_command_reply: tid=0 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.040+0000 7fb04f1f0740 1 client.18399 shutdown 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 msgr2=0x565533540280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 0x565533540280 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb0380a9630 tx=0x7fb0380038b0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 shutdown_connections 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56553314b9f0 0x5655335520c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553318d070 0x56553354c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553316c390 0x565533540280 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/4169738850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x56553353fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb029ffb640 0 client.18399 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 >> 172.21.15.92:0/4169738850 conn(0x565533181460 msgr2=0x565533537850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 shutdown_connections 2024-09-21T02:14:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 1 -- 172.21.15.92:0/4169738850 wait complete. 2024-09-21T02:14:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.041+0000 7fb04f1f0740 20 client.18399 trim_cache size 0 max 16384 2024-09-21T02:14:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 Processor -- start 2024-09-21T02:14:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 -- start start 2024-09-21T02:14:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533535de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x56553353cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x56553353d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441af380 con 0x56553318d070 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b0ec0 con 0x5655331641b0 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441afa50 con 0x56553316c390 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x56553353d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x56553353d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53740/0 (socket says 172.21.15.92:53740) 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 -- 172.21.15.92:0/383443401 learned_addr learned my addr 172.21.15.92:0/383443401 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb041d9b640 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x56553353cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb04259c640 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533535de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 -- 172.21.15.92:0/383443401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 msgr2=0x56553353cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x56553353cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 -- 172.21.15.92:0/383443401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 msgr2=0x565533535de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533535de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 -- 172.21.15.92:0/383443401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565533541000 con 0x56553318d070 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.054+0000 7fb042d9d640 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x56553353d470 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005ab0 tx=0x7fb03c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f5876ad5484dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb02bfff640 1 -- 172.21.15.92:0/383443401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c035040 con 0x56553318d070 2024-09-21T02:14:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb02bfff640 1 -- 172.21.15.92:0/383443401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c0046c0 con 0x56553318d070 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb02bfff640 1 -- 172.21.15.92:0/383443401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0039f0 con 0x56553318d070 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 -- 172.21.15.92:0/383443401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 msgr2=0x56553353d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x56553353d470 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005ab0 tx=0x7fb03c03b000 comp rx=0 tx=0).stop 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 -- 172.21.15.92:0/383443401 shutdown_connections 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x56553353d470 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x56553353cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/383443401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x565533535de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.055+0000 7fb04f1f0740 1 -- 172.21.15.92:0/383443401 >> 172.21.15.92:0/383443401 conn(0x565533181460 msgr2=0x5655335356d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 -- 172.21.15.92:0/383443401 shutdown_connections 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 -- 172.21.15.92:0/383443401 wait complete. 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 -- start start 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 Processor -- start 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x5655335486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533543b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b07f0 con 0x56553318d070 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b0120 con 0x5655331641b0 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441ad360 con 0x56553316c390 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb04f1f0740 10 client.0 _ll_register_callbacks cb 0x7fb03800a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb042d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53752/0 (socket says 172.21.15.92:53752) 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb042d9d640 1 -- 172.21.15.92:0/2244020946 learned_addr learned my addr 172.21.15.92:0/2244020946 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.056+0000 7fb041d9b640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533543b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb04259c640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x5655335486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb042d9d640 1 -- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 msgr2=0x565533543b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb042d9d640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533543b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb042d9d640 1 -- 172.21.15.92:0/2244020946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 msgr2=0x5655335486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb042d9d640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x5655335486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb042d9d640 1 -- 172.21.15.92:0/2244020946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565533544660 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb042d9d640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335440a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb03c02a450 tx=0x7fb03c004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c917f9c1e5bfd6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb029ffb640 1 -- 172.21.15.92:0/2244020946 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c035070 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb029ffb640 1 -- 172.21.15.92:0/2244020946 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fb03c031c90 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03c0046c0 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb029ffb640 1 -- 172.21.15.92:0/2244020946 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb03c004100 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb04f1f0740 10 client.18405 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5655335bc780 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.057+0000 7fb029ffb640 1 -- 172.21.15.92:0/2244020946 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c0046c0 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb029ffb640 1 -- 172.21.15.92:0/2244020946 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb03c032030 con 0x56553318d070 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04f1f0740 10 client.18405 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04f1f0740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04f1f0740 20 client.18405 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565533120da0 0x5655335bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04f1f0740 4 client.18405 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:14:57.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5655334a88a0 con 0x565533120da0 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04259c640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565533120da0 0x5655335bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb029ffb640 10 client.18405 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.058+0000 7fb04259c640 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565533120da0 0x5655335bea60 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5a44e88f5caa74 server_cookie=547fb522c69ca264 in_seq=0 out_seq=0 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb029ffb640 1 -- 172.21.15.92:0/2244020946 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5655334a88a0 con 0x565533120da0 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb029ffb640 10 client.18405 handle_command_reply: tid=0 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 client.18405 shutdown 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 msgr2=0x5655335440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335440a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb03c02a450 tx=0x7fb03c004aa0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 shutdown_connections 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x565533120da0 0x5655335bea60 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56553318d070 0x5655335440a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56553316c390 0x565533543b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 --2- 172.21.15.92:0/2244020946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5655331641b0 0x5655335486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 >> 172.21.15.92:0/2244020946 conn(0x565533181460 msgr2=0x565533535580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb029ffb640 0 client.18405 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:57.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 shutdown_connections 2024-09-21T02:14:57.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 1 -- 172.21.15.92:0/2244020946 wait complete. 2024-09-21T02:14:57.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb04f1f0740 20 client.18405 trim_cache size 0 max 16384 2024-09-21T02:14:57.066 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb02c0b66e0 msgr2=0x7fb02c0b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb048c7c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb02c0b66e0 0x7fb02c0b8bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb03c044000 tx=0x7fb03c045000 comp rx=0 tx=0).stop 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 msgr2=0x7fb04419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.059+0000 7fb048c7c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 0x7fb04419a990 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c970 tx=0x7fb03800ce30 comp rx=0 tx=0).stop 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 shutdown_connections 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fb02c0b66e0 0x7fb02c0b8bf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb044106c20 0x7fb04419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0441062d0 0x7fb04419a990 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 --2- 172.21.15.92:0/1600540630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb044101ff0 0x7fb04419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 >> 172.21.15.92:0/1600540630 conn(0x7fb044079e30 msgr2=0x7fb0441074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 shutdown_connections 2024-09-21T02:14:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:57.060+0000 7fb048c7c640 1 -- 172.21.15.92:0/1600540630 wait complete. 2024-09-21T02:14:57.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:56 smithi099 ceph-mon[34404]: pgmap v659: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-09-21T02:14:58.697 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:14:58.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:14:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:14:59.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:14:58 smithi136 ceph-mon[34754]: pgmap v660: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:59.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:14:58 smithi092 ceph-mon[31190]: pgmap v660: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:14:58 smithi099 ceph-mon[34404]: pgmap v660: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:14:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 -- 172.21.15.92:0/2567692310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 msgr2=0x7f3a6c10ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 --2- 172.21.15.92:0/2567692310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c10ed00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009ca0 tx=0x7f3a6402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 -- 172.21.15.92:0/2567692310 shutdown_connections 2024-09-21T02:14:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 --2- 172.21.15.92:0/2567692310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c10ed00 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 --2- 172.21.15.92:0/2567692310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c107b80 0x7f3a6c107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 --2- 172.21.15.92:0/2567692310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a6c102240 0x7f3a6c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 -- 172.21.15.92:0/2567692310 >> 172.21.15.92:0/2567692310 conn(0x7f3a6c079f90 msgr2=0x7f3a6c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 -- 172.21.15.92:0/2567692310 shutdown_connections 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.472+0000 7f3a71c41640 1 -- 172.21.15.92:0/2567692310 wait complete. 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 Processor -- start 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 -- start start 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c102240 0x7f3a6c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a6c107b80 0x7f3a6c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c112a20 con 0x7f3a6c1084d0 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1128a0 con 0x7f3a6c102240 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a71c41640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c112ba0 con 0x7f3a6c107b80 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53780/0 (socket says 172.21.15.92:53780) 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 -- 172.21.15.92:0/1849368476 learned_addr learned my addr 172.21.15.92:0/1849368476 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6affd640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a6c107b80 0x7f3a6c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c102240 0x7f3a6c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 -- 172.21.15.92:0/1849368476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a6c107b80 msgr2=0x7f3a6c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a6c107b80 0x7f3a6c19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 -- 172.21.15.92:0/1849368476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c102240 msgr2=0x7f3a6c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c102240 0x7f3a6c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.473+0000 7f3a6bfff640 1 -- 172.21.15.92:0/1849368476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c1a65e0 con 0x7f3a6c1084d0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c102240 0x7f3a6c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a6bfff640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c19bdd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098d0 tx=0x7f3a640317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a68ff9640 1 -- 172.21.15.92:0/1849368476 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64003b70 con 0x7f3a6c1084d0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c1a6870 con 0x7f3a6c1084d0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a68ff9640 1 -- 172.21.15.92:0/1849368476 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a64003d10 con 0x7f3a6c1084d0 2024-09-21T02:14:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a68ff9640 1 -- 172.21.15.92:0/1849368476 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6402fa20 con 0x7f3a6c1084d0 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.474+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c1a6e00 con 0x7f3a6c1084d0 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 Processor -- start 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a68ff9640 1 -- 172.21.15.92:0/1849368476 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f3a6402fc00 con 0x7f3a6c1084d0 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 -- start start 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85270d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d85270d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852345670 0x55d85270deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d852683e60 con 0x55d85233d490 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d852673290 con 0x55d852366350 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a781b5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d8526770a0 con 0x55d852345670 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d85270d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d85270d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53786/0 (socket says 172.21.15.92:53786) 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2442393627 learned_addr learned my addr 172.21.15.92:0/2442393627 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.477+0000 7f3a6affd640 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85270d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6affd640 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852345670 0x55d85270deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a68ff9640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a540b6540 0x7f3a540b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2442393627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852345670 msgr2=0x55d85270deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852345670 0x55d85270deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2442393627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 msgr2=0x55d85270d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85270d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2442393627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8523d1930 con 0x55d85233d490 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6bfff640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a540b6540 0x7f3a540b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d85270d970 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009c20 tx=0x7f3a5c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1e5148625df6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a68ff9640 1 -- 172.21.15.92:0/1849368476 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3a641085e0 con 0x7f3a6c1084d0 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a51ffb640 1 -- 172.21.15.92:0/2442393627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c004030 con 0x55d85233d490 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a51ffb640 1 -- 172.21.15.92:0/2442393627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a5c0035f0 con 0x55d85233d490 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.478+0000 7f3a51ffb640 1 -- 172.21.15.92:0/2442393627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0126b0 con 0x55d85233d490 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 -- 172.21.15.92:0/2442393627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 msgr2=0x55d85270d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a6bfff640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a540b6540 0x7f3a540b8a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3a640ce450 tx=0x7f3a640ff1f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d85270d970 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009c20 tx=0x7f3a5c010740 comp rx=0 tx=0).stop 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 -- 172.21.15.92:0/2442393627 shutdown_connections 2024-09-21T02:14:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852345670 0x55d85270deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d85270d970 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2442393627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85270d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 -- 172.21.15.92:0/2442393627 >> 172.21.15.92:0/2442393627 conn(0x55d85235a740 msgr2=0x55d85270b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 -- 172.21.15.92:0/2442393627 shutdown_connections 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.479+0000 7f3a781b5740 1 -- 172.21.15.92:0/2442393627 wait complete. 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 -- start start 2024-09-21T02:14:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 Processor -- start 2024-09-21T02:14:59.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d852718f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d8527194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852366350 0x55d852724f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8524d4890 con 0x55d85233d490 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d85260dd40 con 0x55d852345670 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d85271bc10 con 0x55d852366350 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a781b5740 10 client.0 _ll_register_callbacks cb 0x55d852671c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d852718f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d852718f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53792/0 (socket says 172.21.15.92:53792) 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a6affd640 1 -- 172.21.15.92:0/310896069 learned_addr learned my addr 172.21.15.92:0/310896069 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a6affd640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852366350 0x55d852724f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.480+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d8527194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a6affd640 1 -- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852366350 msgr2=0x55d852724f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a6affd640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852366350 0x55d852724f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a6affd640 1 -- 172.21.15.92:0/310896069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 msgr2=0x55d8527194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a6affd640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d8527194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a6affd640 1 -- 172.21.15.92:0/310896069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d852725470 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a6affd640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d852718f80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c06a530 tx=0x7f3a6000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1822a96c36b7d946 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a4affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a4affd640 1 -- 172.21.15.92:0/310896069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60011ac0 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a4affd640 1 -- 172.21.15.92:0/310896069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3a60011c60 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a5c0035f0 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a4affd640 1 -- 172.21.15.92:0/310896069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a60011e60 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a781b5740 10 client.18417 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d852727db0 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.481+0000 7f3a4affd640 1 -- 172.21.15.92:0/310896069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a5c0035f0 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a4affd640 1 -- 172.21.15.92:0/310896069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3a6000f480 con 0x55d85233d490 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a781b5740 10 client.18417 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a781b5740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a781b5740 20 client.18417 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a781b5740 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d852324cd0 0x55d85272b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a781b5740 4 client.18417 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d852685db0 con 0x55d852324cd0 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d852324cd0 0x55d85272b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a4affd640 10 client.18417 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.482+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d852324cd0 0x55d85272b2a0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ca14fbd329cf95 server_cookie=b4170e8bdb1e9e7 in_seq=0 out_seq=0 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a4affd640 1 -- 172.21.15.92:0/310896069 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d852685db0 con 0x55d852324cd0 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a4affd640 10 client.18417 handle_command_reply: tid=0 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 client.18417 shutdown 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 msgr2=0x55d852718f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d852718f80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c06a530 tx=0x7f3a6000b650 comp rx=0 tx=0).stop 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 shutdown_connections 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d852324cd0 0x55d85272b2a0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d852366350 0x55d852724f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d8527194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 --2- 172.21.15.92:0/310896069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d85233d490 0x55d852718f80 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a4affd640 0 client.18417 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 >> 172.21.15.92:0/310896069 conn(0x55d85235a740 msgr2=0x55d8527105c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 shutdown_connections 2024-09-21T02:14:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.483+0000 7f3a781b5740 1 -- 172.21.15.92:0/310896069 wait complete. 2024-09-21T02:14:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.484+0000 7f3a781b5740 20 client.18417 trim_cache size 0 max 16384 2024-09-21T02:14:59.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 Processor -- start 2024-09-21T02:14:59.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 -- start start 2024-09-21T02:14:59.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852724e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d85270eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 0x55d852715f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1af580 con 0x55d852366350 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1b10c0 con 0x55d852345670 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a781b5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1afc50 con 0x55d85233d490 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 0x55d852715f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 0x55d852715f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53804/0 (socket says 172.21.15.92:53804) 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 -- 172.21.15.92:0/1830262412 learned_addr learned my addr 172.21.15.92:0/1830262412 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d85270eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 -- 172.21.15.92:0/1830262412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 msgr2=0x55d852724e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852724e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 -- 172.21.15.92:0/1830262412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 msgr2=0x55d85270eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d85270eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6affd640 1 -- 172.21.15.92:0/1830262412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d852717ce0 con 0x55d852366350 2024-09-21T02:14:59.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.501+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d85270eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.502+0000 7f3a6affd640 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 0x55d852715f70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005fd0 tx=0x7f3a60010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d875d07e5bc26e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.502+0000 7f3a517fa640 1 -- 172.21.15.92:0/1830262412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60012450 con 0x55d852366350 2024-09-21T02:14:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.502+0000 7f3a517fa640 1 -- 172.21.15.92:0/1830262412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a60011ac0 con 0x55d852366350 2024-09-21T02:14:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.502+0000 7f3a517fa640 1 -- 172.21.15.92:0/1830262412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600043a0 con 0x55d852366350 2024-09-21T02:14:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.502+0000 7f3a781b5740 1 -- 172.21.15.92:0/1830262412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 msgr2=0x55d852715f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.502+0000 7f3a781b5740 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 0x55d852715f70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005fd0 tx=0x7f3a60010ee0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- 172.21.15.92:0/1830262412 shutdown_connections 2024-09-21T02:14:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852366350 0x55d852715f70 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852345670 0x55d85270eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 --2- 172.21.15.92:0/1830262412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852724e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- 172.21.15.92:0/1830262412 >> 172.21.15.92:0/1830262412 conn(0x55d85235a740 msgr2=0x55d85270e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- 172.21.15.92:0/1830262412 shutdown_connections 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- 172.21.15.92:0/1830262412 wait complete. 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- start start 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 Processor -- start 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852721770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 0x55d85271dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85271e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1b09f0 con 0x55d852345670 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1b0320 con 0x55d852366350 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1ad560 con 0x55d85233d490 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.503+0000 7f3a781b5740 10 client.0 _ll_register_callbacks cb 0x7f3a6000bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 0x55d85271dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 0x55d85271dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53810/0 (socket says 172.21.15.92:53810) 2024-09-21T02:14:59.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2119752591 learned_addr learned my addr 172.21.15.92:0/2119752591 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6affd640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852721770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6affd640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85271e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 msgr2=0x55d852721770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852721770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2119752591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 msgr2=0x55d85271e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85271e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 -- 172.21.15.92:0/2119752591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d85271e9f0 con 0x55d852345670 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a6b7fe640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 0x55d85271dc30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x55d85270c0b0 tx=0x7f3a5c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3881e37ed06697 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a4affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a4affd640 1 -- 172.21.15.92:0/2119752591 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0052b0 con 0x55d852345670 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a4affd640 1 -- 172.21.15.92:0/2119752591 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f3a5c005450 con 0x55d852345670 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a60011ac0 con 0x55d852345670 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a4affd640 1 -- 172.21.15.92:0/2119752591 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a5c037050 con 0x55d852345670 2024-09-21T02:14:59.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.504+0000 7f3a781b5740 10 client.18423 fetch_fsmap learned FSMap version 29 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d852796c90 con 0x55d852345670 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a4affd640 1 -- 172.21.15.92:0/2119752591 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a60011ac0 con 0x55d852345670 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a4affd640 1 -- 172.21.15.92:0/2119752591 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3a5c00f040 con 0x55d852345670 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 10 client.18423 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 20 client.18423 populate_metadata read hostname 'smithi092' 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d8522fa080 0x55d852799140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 4 client.18423 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d852681860 con 0x55d8522fa080 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.505+0000 7f3a6affd640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d8522fa080 0x55d852799140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.506+0000 7f3a6affd640 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d8522fa080 0x55d852799140 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5859293b2a0901c server_cookie=a1657ced16461640 in_seq=0 out_seq=0 2024-09-21T02:14:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.506+0000 7f3a4affd640 10 client.18423 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.507+0000 7f3a4affd640 1 -- 172.21.15.92:0/2119752591 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d852681860 con 0x55d8522fa080 2024-09-21T02:14:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.507+0000 7f3a4affd640 10 client.18423 handle_command_reply: tid=0 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.507+0000 7f3a781b5740 1 client.18423 shutdown 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 msgr2=0x55d85271dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 0x55d85271dc30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x55d85270c0b0 tx=0x7f3a5c005d90 comp rx=0 tx=0).stop 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 shutdown_connections 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55d8522fa080 0x55d852799140 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d852366350 0x55d85271e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d852345670 0x55d85271dc30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 --2- 172.21.15.92:0/2119752591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d85233d490 0x55d852721770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 >> 172.21.15.92:0/2119752591 conn(0x55d85235a740 msgr2=0x55d85271a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a4affd640 0 client.18423 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 shutdown_connections 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 1 -- 172.21.15.92:0/2119752591 wait complete. 2024-09-21T02:14:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a781b5740 20 client.18423 trim_cache size 0 max 16384 2024-09-21T02:14:59.512 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:14:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2024-09-21T02:14:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "7f554d8f-2849-42ec-8c87-5032945b82d0", 2024-09-21T02:14:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2024-09-21T02:14:59.512 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a540b6540 msgr2=0x7f3a540b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a540b6540 0x7f3a540b8a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3a640ce450 tx=0x7f3a640ff1f0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 msgr2=0x7f3a6c19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c19bdd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3a640098d0 tx=0x7f3a640317b0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 shutdown_connections 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f3a540b6540 0x7f3a540b8a50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a6c1084d0 0x7f3a6c19bdd0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.508+0000 7f3a71c41640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a6c107b80 0x7f3a6c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.509+0000 7f3a71c41640 1 --2- 172.21.15.92:0/1849368476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a6c102240 0x7f3a6c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.509+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 >> 172.21.15.92:0/1849368476 conn(0x7f3a6c079f90 msgr2=0x7f3a6c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.509+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 shutdown_connections 2024-09-21T02:14:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:14:59.509+0000 7f3a71c41640 1 -- 172.21.15.92:0/1849368476 wait complete. 2024-09-21T02:14:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/1669435493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e08103fc0 msgr2=0x7f5e0810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/1669435493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e08103fc0 0x7f5e0810a960 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009c90 tx=0x7f5e0002a860 comp rx=0 tx=0).stop 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/1669435493 shutdown_connections 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/1669435493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e08103fc0 0x7f5e0810a960 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/1669435493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103620 0x7f5e08103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/1669435493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e081025a0 0x7f5e08102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/1669435493 >> 172.21.15.92:0/1669435493 conn(0x7f5e080fdee0 msgr2=0x7f5e08100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/1669435493 shutdown_connections 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.983+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/1669435493 wait complete. 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 Processor -- start 2024-09-21T02:14:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 -- start start 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 0x7f5e08198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e08103620 0x7f5e08199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103fc0 0x7f5e0819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810e220 con 0x7f5e081025a0 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810e0a0 con 0x7f5e08103fc0 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ecf9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810e3a0 con 0x7f5e08103620 2024-09-21T02:14:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 0x7f5e08198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 0x7f5e08198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53832/0 (socket says 172.21.15.92:53832) 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.984+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3852719950 learned_addr learned my addr 172.21.15.92:0/3852719950 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0d296640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103fc0 0x7f5e0819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e07fff640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e08103620 0x7f5e08199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3852719950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e08103620 msgr2=0x7f5e08199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e08103620 0x7f5e08199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3852719950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103fc0 msgr2=0x7f5e0819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103fc0 0x7f5e0819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3852719950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0819a850 con 0x7f5e081025a0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0d296640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103fc0 0x7f5e0819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 0x7f5e08198e10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00d950 tx=0x7f5dfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e05ffb640 1 -- 172.21.15.92:0/3852719950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc017070 con 0x7f5e081025a0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e05ffb640 1 -- 172.21.15.92:0/3852719950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc009c30 con 0x7f5e081025a0 2024-09-21T02:14:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0819ab40 con 0x7f5e081025a0 2024-09-21T02:14:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e05ffb640 1 -- 172.21.15.92:0/3852719950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00f620 con 0x7f5e081025a0 2024-09-21T02:14:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.985+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e081a1230 con 0x7f5e081025a0 2024-09-21T02:14:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e05ffb640 1 -- 172.21.15.92:0/3852719950 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5dfc010460 con 0x7f5e081025a0 2024-09-21T02:14:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 Processor -- start 2024-09-21T02:14:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 -- start start 2024-09-21T02:14:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f1bd1e6f0 con 0x560f1b9db2f0 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f1bd0db20 con 0x560f1b9e34d0 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e15c65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560f1bd11930 con 0x560f1ba041b0 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53834/0 (socket says 172.21.15.92:53834) 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e0ca95640 1 -- 172.21.15.92:0/2795592407 learned_addr learned my addr 172.21.15.92:0/2795592407 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e07fff640 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e0d296640 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.988+0000 7f5e05ffb640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5df80b6510 0x7f5df80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0ca95640 1 -- 172.21.15.92:0/2795592407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 msgr2=0x560f1bdabe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdabe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0ca95640 1 -- 172.21.15.92:0/2795592407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 msgr2=0x560f1bdad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e05ffb640 1 -- 172.21.15.92:0/3852719950 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc0e7010 con 0x7f5e081025a0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0ca95640 1 -- 172.21.15.92:0/2795592407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1bcd3530 con 0x560f1b9db2f0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0d296640 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0d296640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5df80b6510 0x7f5df80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdac4c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0e4af0 tx=0x7f5dfc010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a9456cf9758ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5df2ffd640 1 -- 172.21.15.92:0/2795592407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc020070 con 0x560f1b9db2f0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5df2ffd640 1 -- 172.21.15.92:0/2795592407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc0acc90 con 0x560f1b9db2f0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.989+0000 7f5df2ffd640 1 -- 172.21.15.92:0/2795592407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc05ae60 con 0x560f1b9db2f0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 -- 172.21.15.92:0/2795592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 msgr2=0x560f1bdac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e0d296640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5df80b6510 0x7f5df80b8a20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e00031040 tx=0x7f5e000095a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdac4c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0e4af0 tx=0x7f5dfc010cb0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 -- 172.21.15.92:0/2795592407 shutdown_connections 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdac4c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 --2- 172.21.15.92:0/2795592407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 -- 172.21.15.92:0/2795592407 >> 172.21.15.92:0/2795592407 conn(0x560f1b9f8300 msgr2=0x560f1bda78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 -- 172.21.15.92:0/2795592407 shutdown_connections 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 -- 172.21.15.92:0/2795592407 wait complete. 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.990+0000 7f5e15c65740 1 -- start start 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 Processor -- start 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdb5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdb6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560f1bc45ab0 con 0x560f1b9db2f0 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f1bcb3730 con 0x560f1b9e34d0 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560f1bdadad0 con 0x560f1ba041b0 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e15c65740 10 client.0 _ll_register_callbacks cb 0x560f1bd076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdb5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53848/0 (socket says 172.21.15.92:53848) 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 -- 172.21.15.92:0/1886469777 learned_addr learned my addr 172.21.15.92:0/1886469777 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e07fff640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdb5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0d296640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdb6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 -- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 msgr2=0x560f1bdb6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdb6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 -- 172.21.15.92:0/1886469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 msgr2=0x560f1bdb5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdb5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0ca95640 1 -- 172.21.15.92:0/1886469777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1ba6f790 con 0x560f1b9db2f0 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e07fff640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.991+0000 7f5e0d296640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdb6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdb5560 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0a9750 tx=0x7f5dfc0acad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d199ee665cf874 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5de7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5de7fff640 1 -- 172.21.15.92:0/1886469777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc020070 con 0x560f1b9db2f0 2024-09-21T02:14:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5de7fff640 1 -- 172.21.15.92:0/1886469777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc058da0 con 0x560f1b9db2f0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc0acc90 con 0x560f1b9db2f0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5de7fff640 1 -- 172.21.15.92:0/1886469777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dfc009de0 con 0x560f1b9db2f0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 10 client.18435 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f1bae8620 con 0x560f1b9db2f0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5de7fff640 1 -- 172.21.15.92:0/1886469777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc0acc90 con 0x560f1b9db2f0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5de7fff640 1 -- 172.21.15.92:0/1886469777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dfc010750 con 0x560f1b9db2f0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 10 client.18435 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 10 client.18435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 20 client.18435 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9c2d80 0x560f1bdc2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 4 client.18435 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.992+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f1bd20640 con 0x560f1b9c2d80 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.993+0000 7f5e07fff640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9c2d80 0x560f1bdc2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.993+0000 7f5e07fff640 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9c2d80 0x560f1bdc2870 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb232552f45c55c server_cookie=29338b08490bf5df in_seq=0 out_seq=0 2024-09-21T02:15:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.993+0000 7f5de7fff640 10 client.18435 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5de7fff640 1 -- 172.21.15.92:0/1886469777 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f1bd20640 con 0x560f1b9c2d80 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5de7fff640 10 client.18435 handle_command_reply: tid=0 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 client.18435 shutdown 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 msgr2=0x560f1bdb5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdb5560 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0a9750 tx=0x7f5dfc0acad0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 shutdown_connections 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9c2d80 0x560f1bdc2870 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1ba041b0 0x560f1bdb6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1b9e34d0 0x560f1bdb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5de7fff640 0 client.18435 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 --2- 172.21.15.92:0/1886469777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bdb5560 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 >> 172.21.15.92:0/1886469777 conn(0x560f1b9f8300 msgr2=0x560f1bdacec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 shutdown_connections 2024-09-21T02:15:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 1 -- 172.21.15.92:0/1886469777 wait complete. 2024-09-21T02:15:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:14:59.994+0000 7f5e15c65740 20 client.18435 trim_cache size 0 max 16384 2024-09-21T02:15:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 Processor -- start 2024-09-21T02:15:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 -- start start 2024-09-21T02:15:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bda7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1bdb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1bdb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081a9f10 con 0x560f1b9db2f0 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081aba50 con 0x560f1ba041b0 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e15c65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081aa5e0 con 0x560f1b9e34d0 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bda7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bda7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53862/0 (socket says 172.21.15.92:53862) 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 -- 172.21.15.92:0/225967389 learned_addr learned my addr 172.21.15.92:0/225967389 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 -- 172.21.15.92:0/225967389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 msgr2=0x560f1bdb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1bdb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 -- 172.21.15.92:0/225967389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 msgr2=0x560f1bdb4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1bdb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 -- 172.21.15.92:0/225967389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1bdb3770 con 0x560f1b9db2f0 2024-09-21T02:15:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bda7930 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ad110 tx=0x7f5dfc0ea990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408d50b47926b8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5df27fc640 1 -- 172.21.15.92:0/225967389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc020070 con 0x560f1b9db2f0 2024-09-21T02:15:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5df27fc640 1 -- 172.21.15.92:0/225967389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc0ac1f0 con 0x560f1b9db2f0 2024-09-21T02:15:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.009+0000 7f5df27fc640 1 -- 172.21.15.92:0/225967389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0d0b00 con 0x560f1b9db2f0 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 -- 172.21.15.92:0/225967389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 msgr2=0x560f1bda7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bda7930 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ad110 tx=0x7f5dfc0ea990 comp rx=0 tx=0).stop 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 -- 172.21.15.92:0/225967389 shutdown_connections 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1bdb4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1bdb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 --2- 172.21.15.92:0/225967389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1bda7930 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 -- 172.21.15.92:0/225967389 >> 172.21.15.92:0/225967389 conn(0x560f1b9f8300 msgr2=0x560f1bda7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 -- 172.21.15.92:0/225967389 shutdown_connections 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.010+0000 7f5e15c65740 1 -- 172.21.15.92:0/225967389 wait complete. 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 -- start start 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 Processor -- start 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1be2c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1be2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1be2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081ab380 con 0x560f1b9db2f0 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081aacb0 con 0x560f1ba041b0 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081a7ef0 con 0x560f1b9e34d0 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e15c65740 10 client.0 _ll_register_callbacks cb 0x7f5df400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1be2c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1be2c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53864/0 (socket says 172.21.15.92:53864) 2024-09-21T02:15:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3483322273 learned_addr learned my addr 172.21.15.92:0/3483322273 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0d296640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1be2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e07fff640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1be2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 msgr2=0x560f1be2cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1be2cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3483322273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 msgr2=0x560f1be2b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1be2b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0ca95640 1 -- 172.21.15.92:0/3483322273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1be2bb40 con 0x560f1b9db2f0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.011+0000 7f5e0d296640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1be2b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e0ca95640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1be2c800 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ace80 tx=0x7f5dfc0ac160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab2370109aa1862 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5de7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5de7fff640 1 -- 172.21.15.92:0/3483322273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc020030 con 0x560f1b9db2f0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5de7fff640 1 -- 172.21.15.92:0/3483322273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc0a8ad0 con 0x560f1b9db2f0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc0ac1f0 con 0x560f1b9db2f0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5de7fff640 1 -- 172.21.15.92:0/3483322273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dfc0acc90 con 0x560f1b9db2f0 2024-09-21T02:15:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 10 client.18441 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f1be2bb40 con 0x560f1b9db2f0 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5de7fff640 1 -- 172.21.15.92:0/3483322273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc0ac1f0 con 0x560f1b9db2f0 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5de7fff640 1 -- 172.21.15.92:0/3483322273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dfc0b2f10 con 0x560f1b9db2f0 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 10 client.18441 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 10 client.18441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 20 client.18441 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9981c0 0x560f1be37cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 4 client.18441 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.012+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f1bd1c0f0 con 0x560f1b9981c0 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.013+0000 7f5e07fff640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9981c0 0x560f1be37cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.013+0000 7f5de7fff640 10 client.18441 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.013+0000 7f5e07fff640 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9981c0 0x560f1be37cb0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef10742586348ab2 server_cookie=30abae3a6bc901a2 in_seq=0 out_seq=0 2024-09-21T02:15:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5de7fff640 1 -- 172.21.15.92:0/3483322273 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f1bd1c0f0 con 0x560f1b9981c0 2024-09-21T02:15:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5de7fff640 10 client.18441 handle_command_reply: tid=0 2024-09-21T02:15:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 client.18441 shutdown 2024-09-21T02:15:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 msgr2=0x560f1be2c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1be2c800 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ace80 tx=0x7f5dfc0ac160 comp rx=0 tx=0).stop 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 shutdown_connections 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x560f1b9981c0 0x560f1be37cb0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f1ba041b0 0x560f1be2b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5de7fff640 0 client.18441 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560f1b9e34d0 0x560f1be2cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 --2- 172.21.15.92:0/3483322273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560f1b9db2f0 0x560f1be2c800 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 >> 172.21.15.92:0/3483322273 conn(0x560f1b9f8300 msgr2=0x560f1bda5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 shutdown_connections 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 1 -- 172.21.15.92:0/3483322273 wait complete. 2024-09-21T02:15:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.028+0000 7f5e15c65740 20 client.18441 trim_cache size 0 max 16384 2024-09-21T02:15:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:15:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:15:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:15:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5df80b6510 msgr2=0x7f5df80b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5df80b6510 0x7f5df80b8a20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e00031040 tx=0x7f5e000095a0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 msgr2=0x7f5e08198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 0x7f5e08198e10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00d950 tx=0x7f5dfc00de10 comp rx=0 tx=0).stop 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 shutdown_connections 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5df80b6510 0x7f5df80b8a20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e08103fc0 0x7f5e0819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e08103620 0x7f5e08199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 --2- 172.21.15.92:0/3852719950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e081025a0 0x7f5e08198e10 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 >> 172.21.15.92:0/3852719950 conn(0x7f5e080fdee0 msgr2=0x7f5e08106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 shutdown_connections 2024-09-21T02:15:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:00.029+0000 7f5e0ecf9640 1 -- 172.21.15.92:0/3852719950 wait complete. 2024-09-21T02:15:00.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:15:00.499 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:00.884 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:00 smithi099 ceph-mon[34404]: pgmap v661: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:15:00.884 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:00 smithi099 ceph-mon[34404]: scrub queued for path: / 2024-09-21T02:15:00.884 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:00 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:15:00.884 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:00 smithi099 ceph-mon[34404]: scrub summary: active paths [/] 2024-09-21T02:15:00.884 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:00 smithi099 ceph-mon[34404]: scrub summary: idle+waiting paths [/] 2024-09-21T02:15:00.913 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:00 smithi092 ceph-mon[31190]: pgmap v661: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:15:00.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:00 smithi092 ceph-mon[31190]: scrub queued for path: / 2024-09-21T02:15:00.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:00 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:15:00.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:00 smithi092 ceph-mon[31190]: scrub summary: active paths [/] 2024-09-21T02:15:00.914 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:00 smithi092 ceph-mon[31190]: scrub summary: idle+waiting paths [/] 2024-09-21T02:15:01.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/1980116831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc64102a60 msgr2=0x7fbc64102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/1980116831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc64102a60 0x7fbc64102e70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009900 tx=0x7fbc6002a410 comp rx=0 tx=0).stop 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/1980116831 shutdown_connections 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/1980116831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc64108cf0 0x7fbc6410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/1980116831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc641083a0 0x7fbc641087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/1980116831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc64102a60 0x7fbc64102e70 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/1980116831 >> 172.21.15.92:0/1980116831 conn(0x7fbc64079730 msgr2=0x7fbc64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/1980116831 shutdown_connections 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/1980116831 wait complete. 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 Processor -- start 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.005+0000 7fbc6c6a5640 1 -- start start 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc64102a60 0x7fbc6419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 0x7fbc6419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc64108cf0 0x7fbc6419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64112960 con 0x7fbc641083a0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641127e0 con 0x7fbc64102a60 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64112ae0 con 0x7fbc64108cf0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 0x7fbc6419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 0x7fbc6419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53886/0 (socket says 172.21.15.92:53886) 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 -- 172.21.15.92:0/3827515745 learned_addr learned my addr 172.21.15.92:0/3827515745 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6a441640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc64102a60 0x7fbc6419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc64108cf0 0x7fbc6419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 -- 172.21.15.92:0/3827515745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc64108cf0 msgr2=0x7fbc6419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc64108cf0 0x7fbc6419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 -- 172.21.15.92:0/3827515745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc64102a60 msgr2=0x7fbc6419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc64102a60 0x7fbc6419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 -- 172.21.15.92:0/3827515745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc641a6520 con 0x7fbc641083a0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc69c40640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 0x7fbc6419b7d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400e9c0 tx=0x7fbc5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc5b7fe640 1 -- 172.21.15.92:0/3827515745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5400cd80 con 0x7fbc641083a0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc5b7fe640 1 -- 172.21.15.92:0/3827515745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc540054d0 con 0x7fbc641083a0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.006+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc641a6810 con 0x7fbc641083a0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.007+0000 7fbc5b7fe640 1 -- 172.21.15.92:0/3827515745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54010430 con 0x7fbc641083a0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.007+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc641a6d50 con 0x7fbc641083a0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.009+0000 7fbc72c19740 1 Processor -- start 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 -- start start 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc5b7fe640 1 -- 172.21.15.92:0/3827515745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fbc54004f90 con 0x7fbc641083a0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a9e070 0x55fc58e45650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e45b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc58dbbe50 con 0x55fc58a751b0 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fc58dab280 con 0x55fc58a9e070 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc72c19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc58daf090 con 0x55fc58a7d390 2024-09-21T02:15:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc6a441640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e45b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc6a441640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e45b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53902/0 (socket says 172.21.15.92:53902) 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc6a441640 1 -- 172.21.15.92:0/555108375 learned_addr learned my addr 172.21.15.92:0/555108375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a9e070 0x55fc58e45650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.010+0000 7fbc69c40640 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6a441640 1 -- 172.21.15.92:0/555108375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 msgr2=0x55fc58e460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6a441640 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6a441640 1 -- 172.21.15.92:0/555108375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a9e070 msgr2=0x55fc58e45650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6a441640 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a9e070 0x55fc58e45650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6a441640 1 -- 172.21.15.92:0/555108375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc58b09650 con 0x55fc58a751b0 2024-09-21T02:15:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6a441640 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e45b90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbc6002a930 tx=0x7fbc60032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76ef22d2c01c0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc5b7fe640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc500b6540 0x7fbc500b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc58ff9640 1 -- 172.21.15.92:0/555108375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6003f070 con 0x55fc58a751b0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc58ff9640 1 -- 172.21.15.92:0/555108375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc60004030 con 0x55fc58a751b0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc58ff9640 1 -- 172.21.15.92:0/555108375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60035070 con 0x55fc58a751b0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc500b6540 0x7fbc500b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.011+0000 7fbc5b7fe640 1 -- 172.21.15.92:0/3827515745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbc540e7870 con 0x7fbc641083a0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 -- 172.21.15.92:0/555108375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 msgr2=0x55fc58e45b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc500b6540 0x7fbc500b8a50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c005fd0 tx=0x7fbc5c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e45b90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbc6002a930 tx=0x7fbc60032ba0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 -- 172.21.15.92:0/555108375 shutdown_connections 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e45b90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 --2- 172.21.15.92:0/555108375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a9e070 0x55fc58e45650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 -- 172.21.15.92:0/555108375 >> 172.21.15.92:0/555108375 conn(0x55fc58a92460 msgr2=0x55fc58e43540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 -- 172.21.15.92:0/555108375 shutdown_connections 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 -- 172.21.15.92:0/555108375 wait complete. 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.012+0000 7fbc72c19740 1 -- start start 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 Processor -- start 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e55660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 0x55fc58e61490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc58c0c700 con 0x55fc58a9e070 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fc58d46110 con 0x55fc58a751b0 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc58e58130 con 0x55fc58a7d390 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc72c19740 10 client.0 _ll_register_callbacks cb 0x55fc58d5f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 0x55fc58e61490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 0x55fc58e61490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53912/0 (socket says 172.21.15.92:53912) 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 -- 172.21.15.92:0/2171974413 learned_addr learned my addr 172.21.15.92:0/2171974413 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc69c40640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e55660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6a441640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 -- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 msgr2=0x55fc58e55660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e55660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 -- 172.21.15.92:0/2171974413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 msgr2=0x55fc58e55120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.013+0000 7fbc6ac42640 1 -- 172.21.15.92:0/2171974413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc58d73c40 con 0x55fc58a9e070 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 0x55fc58e61490 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c01f6a0 tx=0x7fbc5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffe7a3d781f3def server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:01.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc4dffb640 1 -- 172.21.15.92:0/2171974413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c02c060 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc4dffb640 1 -- 172.21.15.92:0/2171974413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbc5c017030 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc60004030 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc4dffb640 1 -- 172.21.15.92:0/2171974413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc5c011810 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 10 client.18453 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc58e63ee0 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc4dffb640 1 -- 172.21.15.92:0/2171974413 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc60004030 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc4dffb640 1 -- 172.21.15.92:0/2171974413 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc5c027b40 con 0x55fc58a9e070 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 10 client.18453 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 20 client.18453 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a5c9f0 0x55fc58e67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 4 client.18453 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.014+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc58dbdda0 con 0x55fc58a5c9f0 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.015+0000 7fbc6a441640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a5c9f0 0x55fc58e67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.015+0000 7fbc4dffb640 10 client.18453 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.015+0000 7fbc6a441640 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a5c9f0 0x55fc58e67480 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ba2ca196a94d36 server_cookie=9be5c443a3dbf31d in_seq=0 out_seq=0 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc4dffb640 1 -- 172.21.15.92:0/2171974413 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc58dbdda0 con 0x55fc58a5c9f0 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc4dffb640 10 client.18453 handle_command_reply: tid=0 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 client.18453 shutdown 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 msgr2=0x55fc58e61490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 0x55fc58e61490 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c01f6a0 tx=0x7fbc5c004ab0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 shutdown_connections 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a5c9f0 0x55fc58e67480 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a9e070 0x55fc58e61490 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e55660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc4dffb640 0 client.18453 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 --2- 172.21.15.92:0/2171974413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 >> 172.21.15.92:0/2171974413 conn(0x55fc58a92460 msgr2=0x55fc58e47040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 shutdown_connections 2024-09-21T02:15:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 1 -- 172.21.15.92:0/2171974413 wait complete. 2024-09-21T02:15:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.016+0000 7fbc72c19740 20 client.18453 trim_cache size 0 max 16384 2024-09-21T02:15:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:00 smithi136 ceph-mon[34754]: pgmap v661: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-09-21T02:15:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:00 smithi136 ceph-mon[34754]: scrub queued for path: / 2024-09-21T02:15:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:00 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:15:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:00 smithi136 ceph-mon[34754]: scrub summary: active paths [/] 2024-09-21T02:15:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:00 smithi136 ceph-mon[34754]: scrub summary: idle+waiting paths [/] 2024-09-21T02:15:01.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 Processor -- start 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 -- start start 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 0x55fc58e54ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e52310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ad250 con 0x55fc58a5c9f0 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641aed90 con 0x55fc58a751b0 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc72c19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ad920 con 0x55fc58a7d390 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc6a441640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 0x55fc58e54ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc6a441640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 0x55fc58e54ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53920/0 (socket says 172.21.15.92:53920) 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc6a441640 1 -- 172.21.15.92:0/1718316198 learned_addr learned my addr 172.21.15.92:0/1718316198 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e52310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc69c40640 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc6a441640 1 -- 172.21.15.92:0/1718316198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 msgr2=0x55fc58e52310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.034+0000 7fbc6a441640 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e52310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc6a441640 1 -- 172.21.15.92:0/1718316198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 msgr2=0x55fc58e55410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc6a441640 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc6a441640 1 -- 172.21.15.92:0/1718316198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc58e46490 con 0x55fc58a5c9f0 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc69c40640 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc6a441640 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 0x55fc58e54ed0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009900 tx=0x7fbc60005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef979707d41e8836 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc4ffff640 1 -- 172.21.15.92:0/1718316198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60035040 con 0x55fc58a5c9f0 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc4ffff640 1 -- 172.21.15.92:0/1718316198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc600046c0 con 0x55fc58a5c9f0 2024-09-21T02:15:01.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.035+0000 7fbc4ffff640 1 -- 172.21.15.92:0/1718316198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6003f040 con 0x55fc58a5c9f0 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 -- 172.21.15.92:0/1718316198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 msgr2=0x55fc58e54ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 0x55fc58e54ed0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009900 tx=0x7fbc60005150 comp rx=0 tx=0).stop 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 -- 172.21.15.92:0/1718316198 shutdown_connections 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e52310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a751b0 0x55fc58e55410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 --2- 172.21.15.92:0/1718316198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a5c9f0 0x55fc58e54ed0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 -- 172.21.15.92:0/1718316198 >> 172.21.15.92:0/1718316198 conn(0x55fc58a92460 msgr2=0x55fc58e539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 -- 172.21.15.92:0/1718316198 shutdown_connections 2024-09-21T02:15:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 -- 172.21.15.92:0/1718316198 wait complete. 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 -- start start 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.036+0000 7fbc72c19740 1 Processor -- start 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a5c9f0 0x55fc58e59450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e59990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e59ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ae6c0 con 0x55fc58a751b0 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641adff0 con 0x55fc58a5c9f0 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641ab230 con 0x55fc58a7d390 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e59990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc72c19740 10 client.0 _ll_register_callbacks cb 0x7fbc5c009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e59990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53928/0 (socket says 172.21.15.92:53928) 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 -- 172.21.15.92:0/4243332611 learned_addr learned my addr 172.21.15.92:0/4243332611 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc6ac42640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e59ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc6a441640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a5c9f0 0x55fc58e59450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 -- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 msgr2=0x55fc58e59ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e59ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 -- 172.21.15.92:0/4243332611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a5c9f0 msgr2=0x55fc58e59450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a5c9f0 0x55fc58e59450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 -- 172.21.15.92:0/4243332611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc58ed0f00 con 0x55fc58a751b0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc69c40640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e59990 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbc540a9fb0 tx=0x7fbc540b71f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2832fa59b5130d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.037+0000 7fbc4dffb640 1 -- 172.21.15.92:0/4243332611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540eae60 con 0x55fc58a751b0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc4dffb640 1 -- 172.21.15.92:0/4243332611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fbc540a9540 con 0x55fc58a751b0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc600046c0 con 0x55fc58a751b0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc4dffb640 1 -- 172.21.15.92:0/4243332611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc540ad7b0 con 0x55fc58a751b0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 10 client.18459 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc58ed21f0 con 0x55fc58a751b0 2024-09-21T02:15:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc4dffb640 1 -- 172.21.15.92:0/4243332611 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc600046c0 con 0x55fc58a751b0 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc4dffb640 1 -- 172.21.15.92:0/4243332611 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbc540c3b90 con 0x55fc58a751b0 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 10 client.18459 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 20 client.18459 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a9e070 0x55fc58ed49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 4 client.18459 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc58db9850 con 0x55fc58a9e070 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.038+0000 7fbc6a441640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a9e070 0x55fc58ed49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc4dffb640 10 client.18459 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc6a441640 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a9e070 0x55fc58ed49e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ac1bfba4148b9c server_cookie=da8bf2cec9b78d80 in_seq=0 out_seq=0 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc4dffb640 1 -- 172.21.15.92:0/4243332611 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55fc58db9850 con 0x55fc58a9e070 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc4dffb640 10 client.18459 handle_command_reply: tid=0 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 client.18459 shutdown 2024-09-21T02:15:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 msgr2=0x55fc58e59990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e59990 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbc540a9fb0 tx=0x7fbc540b71f0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 shutdown_connections 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55fc58a9e070 0x55fc58ed49e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc4dffb640 0 client.18459 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc58a7d390 0x55fc58e59ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc58a751b0 0x55fc58e59990 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 --2- 172.21.15.92:0/4243332611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc58a5c9f0 0x55fc58e59450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 >> 172.21.15.92:0/4243332611 conn(0x55fc58a92460 msgr2=0x55fc58e54ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 shutdown_connections 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.039+0000 7fbc72c19740 1 -- 172.21.15.92:0/4243332611 wait complete. 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc72c19740 20 client.18459 trim_cache size 0 max 16384 2024-09-21T02:15:01.045 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "7f554d8f-2849-42ec-8c87-5032945b82d0": { 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "7f554d8f-2849-42ec-8c87-5032945b82d0", 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,force" 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a": { 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:14:14.805000+0000", 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.046 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.047 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.049 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.050 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.051 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:13:30.072031+0000", 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.052 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "7f554d8f-2849-42ec-8c87-5032945b82d0": { 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:14:59.505964+0000", 2024-09-21T02:15:01.053 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.054 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:01.055 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc500b6540 msgr2=0x7fbc500b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc500b6540 0x7fbc500b8a50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c005fd0 tx=0x7fbc5c009500 comp rx=0 tx=0).stop 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 msgr2=0x7fbc6419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 0x7fbc6419b7d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400e9c0 tx=0x7fbc5400ee80 comp rx=0 tx=0).stop 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 shutdown_connections 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fbc500b6540 0x7fbc500b8a50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc64108cf0 0x7fbc6419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc641083a0 0x7fbc6419b7d0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 --2- 172.21.15.92:0/3827515745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc64102a60 0x7fbc6419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 >> 172.21.15.92:0/3827515745 conn(0x7fbc64079730 msgr2=0x7fbc64101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 shutdown_connections 2024-09-21T02:15:01.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:01.040+0000 7fbc6c6a5640 1 -- 172.21.15.92:0/3827515745 wait complete. 2024-09-21T02:15:01.764 INFO:tasks.cephfs.filesystem:scrub status for tag:7f554d8f-2849-42ec-8c87-5032945b82d0 - {'path': '/', 'tag': '7f554d8f-2849-42ec-8c87-5032945b82d0', 'options': 'recursive,force'} 2024-09-21T02:15:03.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:02 smithi136 ceph-mon[34754]: pgmap v662: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:15:03.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:02 smithi092 ceph-mon[31190]: pgmap v662: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:15:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:02 smithi099 ceph-mon[34404]: pgmap v662: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:15:04.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:03 smithi136 ceph-mon[34754]: scrub summary: idle 2024-09-21T02:15:04.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:03 smithi092 ceph-mon[31190]: scrub summary: idle 2024-09-21T02:15:04.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:03 smithi099 ceph-mon[34404]: scrub summary: idle 2024-09-21T02:15:05.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:04 smithi136 ceph-mon[34754]: pgmap v663: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:05.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:04 smithi092 ceph-mon[31190]: pgmap v663: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:05.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:04 smithi099 ceph-mon[34404]: pgmap v663: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:07.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:06 smithi136 ceph-mon[34754]: pgmap v664: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:15:07.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:06 smithi092 ceph-mon[31190]: pgmap v664: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:15:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:06 smithi099 ceph-mon[34404]: pgmap v664: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-09-21T02:15:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:08 smithi136 ceph-mon[34754]: pgmap v665: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:15:09.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:08 smithi092 ceph-mon[31190]: pgmap v665: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:15:09.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:08 smithi099 ceph-mon[34404]: pgmap v665: 97 pgs: 97 active+clean; 3.6 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-21T02:15:11.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:10 smithi136 ceph-mon[34754]: pgmap v666: 97 pgs: 97 active+clean; 3.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:15:11.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:10 smithi092 ceph-mon[31190]: pgmap v666: 97 pgs: 97 active+clean; 3.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:15:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:10 smithi099 ceph-mon[34404]: pgmap v666: 97 pgs: 97 active+clean; 3.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 22 op/s 2024-09-21T02:15:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:12 smithi136 ceph-mon[34754]: pgmap v667: 97 pgs: 97 active+clean; 3.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:15:13.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:12 smithi092 ceph-mon[31190]: pgmap v667: 97 pgs: 97 active+clean; 3.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:15:13.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:12 smithi099 ceph-mon[34404]: pgmap v667: 97 pgs: 97 active+clean; 3.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-21T02:15:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 -- 172.21.15.92:0/1849476815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103fe0 msgr2=0x7f0a4410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 --2- 172.21.15.92:0/1849476815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103fe0 0x7f0a4410a980 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009ca0 tx=0x7f0a3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 -- 172.21.15.92:0/1849476815 shutdown_connections 2024-09-21T02:15:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 --2- 172.21.15.92:0/1849476815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103fe0 0x7f0a4410a980 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 --2- 172.21.15.92:0/1849476815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103640 0x7f0a44103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 --2- 172.21.15.92:0/1849476815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a441025c0 0x7f0a44102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 -- 172.21.15.92:0/1849476815 >> 172.21.15.92:0/1849476815 conn(0x7f0a440fdf00 msgr2=0x7f0a441003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 -- 172.21.15.92:0/1849476815 shutdown_connections 2024-09-21T02:15:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.136+0000 7f0a4b913640 1 -- 172.21.15.92:0/1849476815 wait complete. 2024-09-21T02:15:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 Processor -- start 2024-09-21T02:15:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 -- start start 2024-09-21T02:15:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a441025c0 0x7f0a44198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 0x7f0a44199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103fe0 0x7f0a4419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410e210 con 0x7f0a44103640 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410e090 con 0x7f0a44103fe0 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a4b913640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4410e390 con 0x7f0a441025c0 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a48eae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 0x7f0a44199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a48eae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 0x7f0a44199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59200/0 (socket says 172.21.15.92:59200) 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.137+0000 7f0a48eae640 1 -- 172.21.15.92:0/2483966270 learned_addr learned my addr 172.21.15.92:0/2483966270 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a496af640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a441025c0 0x7f0a44198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a48eae640 1 -- 172.21.15.92:0/2483966270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a441025c0 msgr2=0x7f0a44198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103fe0 0x7f0a4419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a48eae640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a441025c0 0x7f0a44198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a48eae640 1 -- 172.21.15.92:0/2483966270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103fe0 msgr2=0x7f0a4419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a48eae640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103fe0 0x7f0a4419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a48eae640 1 -- 172.21.15.92:0/2483966270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4419a840 con 0x7f0a44103640 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103fe0 0x7f0a4419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a48eae640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 0x7f0a44199470 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c890 tx=0x7f0a4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a367fc640 1 -- 172.21.15.92:0/2483966270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004220 con 0x7f0a44103640 2024-09-21T02:15:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a367fc640 1 -- 172.21.15.92:0/2483966270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a400043c0 con 0x7f0a44103640 2024-09-21T02:15:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4419ab30 con 0x7f0a44103640 2024-09-21T02:15:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a367fc640 1 -- 172.21.15.92:0/2483966270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004a00 con 0x7f0a44103640 2024-09-21T02:15:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.138+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a441a1220 con 0x7f0a44103640 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 Processor -- start 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 -- start start 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a367fc640 1 -- 172.21.15.92:0/2483966270 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f0a400079b0 con 0x7f0a44103640 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced935e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced5652f0 0x55eced9364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced56d4d0 0x55eced937010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eced8a86f0 con 0x55eced58e1b0 2024-09-21T02:15:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55eced897b20 con 0x55eced56d4d0 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a52882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eced89b930 con 0x55eced5652f0 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a49eb0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced935e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a49eb0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced935e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59214/0 (socket says 172.21.15.92:59214) 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a49eb0640 1 -- 172.21.15.92:0/339702538 learned_addr learned my addr 172.21.15.92:0/339702538 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a496af640 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced5652f0 0x55eced9364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.141+0000 7f0a48eae640 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced56d4d0 0x55eced937010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a49eb0640 1 -- 172.21.15.92:0/339702538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced5652f0 msgr2=0x55eced9364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced5652f0 0x55eced9364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a49eb0640 1 -- 172.21.15.92:0/339702538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced56d4d0 msgr2=0x55eced937010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced56d4d0 0x55eced937010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a49eb0640 1 -- 172.21.15.92:0/339702538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eced85d530 con 0x55eced58e1b0 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a367fc640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0a300b6590 0x7f0a300b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced935e50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c031a90 tx=0x7f0a3c02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710da62729f9b87b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a496af640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0a300b6590 0x7f0a300b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a367fc640 1 -- 172.21.15.92:0/2483966270 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0a400e7010 con 0x7f0a44103640 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a2f7fe640 1 -- 172.21.15.92:0/339702538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c031650 con 0x55eced58e1b0 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a2f7fe640 1 -- 172.21.15.92:0/339702538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a3c036070 con 0x55eced58e1b0 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.142+0000 7f0a2f7fe640 1 -- 172.21.15.92:0/339702538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c004650 con 0x55eced58e1b0 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 -- 172.21.15.92:0/339702538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 msgr2=0x55eced935e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a496af640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0a300b6590 0x7f0a300b8aa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009cf0 tx=0x7f0a38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced935e50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c031a90 tx=0x7f0a3c02f420 comp rx=0 tx=0).stop 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 -- 172.21.15.92:0/339702538 shutdown_connections 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced56d4d0 0x55eced937010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced5652f0 0x55eced9364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 --2- 172.21.15.92:0/339702538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced935e50 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 -- 172.21.15.92:0/339702538 >> 172.21.15.92:0/339702538 conn(0x55eced582300 msgr2=0x55eced9318e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 -- 172.21.15.92:0/339702538 shutdown_connections 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.143+0000 7f0a52882740 1 -- 172.21.15.92:0/339702538 wait complete. 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 -- start start 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 Processor -- start 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced5652f0 0x55eced93f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced93fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced940740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eced7cfab0 con 0x55eced58e1b0 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55eced83d730 con 0x55eced5652f0 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eced937ad0 con 0x55eced56d4d0 2024-09-21T02:15:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a52882740 10 client.0 _ll_register_callbacks cb 0x55eced8916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a49eb0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced940740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a49eb0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced940740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59224/0 (socket says 172.21.15.92:59224) 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a49eb0640 1 -- 172.21.15.92:0/4027632686 learned_addr learned my addr 172.21.15.92:0/4027632686 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a496af640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced5652f0 0x55eced93f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a48eae640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced93fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a49eb0640 1 -- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 msgr2=0x55eced93fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.144+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced93fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a49eb0640 1 -- 172.21.15.92:0/4027632686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced5652f0 msgr2=0x55eced93f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced5652f0 0x55eced93f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a49eb0640 1 -- 172.21.15.92:0/4027632686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eced5f9790 con 0x55eced58e1b0 2024-09-21T02:15:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a496af640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced5652f0 0x55eced93f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced940740 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c005c90 tx=0x7f0a3c0314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74149898183fb1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a2cff9640 1 -- 172.21.15.92:0/4027632686 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c051020 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a2cff9640 1 -- 172.21.15.92:0/4027632686 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0a3c0043d0 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3c036070 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a2cff9640 1 -- 172.21.15.92:0/4027632686 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a3c05c050 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a52882740 10 client.18471 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eced672620 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a2cff9640 1 -- 172.21.15.92:0/4027632686 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a3c036070 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.145+0000 7f0a2cff9640 1 -- 172.21.15.92:0/4027632686 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a3c04c030 con 0x55eced58e1b0 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a52882740 10 client.18471 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a52882740 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a52882740 20 client.18471 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a52882740 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced54cd80 0x55eced94c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a52882740 4 client.18471 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eced8aa640 con 0x55eced54cd80 2024-09-21T02:15:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a496af640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced54cd80 0x55eced94c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a2cff9640 10 client.18471 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.146+0000 7f0a496af640 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced54cd80 0x55eced94c820 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fb125c7ef8ed0d server_cookie=275d707831942d8 in_seq=0 out_seq=0 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a2cff9640 1 -- 172.21.15.92:0/4027632686 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eced8aa640 con 0x55eced54cd80 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a2cff9640 10 client.18471 handle_command_reply: tid=0 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 client.18471 shutdown 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 msgr2=0x55eced940740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced940740 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c005c90 tx=0x7f0a3c0314d0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 shutdown_connections 2024-09-21T02:15:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced54cd80 0x55eced94c820 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced58e1b0 0x55eced940740 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced93fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a2cff9640 0 client.18471 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 --2- 172.21.15.92:0/4027632686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced5652f0 0x55eced93f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 >> 172.21.15.92:0/4027632686 conn(0x55eced582300 msgr2=0x55eced92fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 shutdown_connections 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.147+0000 7f0a52882740 1 -- 172.21.15.92:0/4027632686 wait complete. 2024-09-21T02:15:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.148+0000 7f0a52882740 20 client.18471 trim_cache size 0 max 16384 2024-09-21T02:15:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 Processor -- start 2024-09-21T02:15:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 -- start start 2024-09-21T02:15:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced931220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced947620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced93daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441a9f00 con 0x55eced5652f0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441aba40 con 0x55eced58e1b0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a52882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441aa5d0 con 0x55eced56d4d0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a496af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced931220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a496af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced931220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59238/0 (socket says 172.21.15.92:59238) 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.162+0000 7f0a496af640 1 -- 172.21.15.92:0/737520018 learned_addr learned my addr 172.21.15.92:0/737520018 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced93daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a48eae640 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced947620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a496af640 1 -- 172.21.15.92:0/737520018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 msgr2=0x55eced947620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a496af640 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced947620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a496af640 1 -- 172.21.15.92:0/737520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 msgr2=0x55eced93daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a496af640 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced93daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a496af640 1 -- 172.21.15.92:0/737520018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eced9367b0 con 0x55eced5652f0 2024-09-21T02:15:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced93daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a496af640 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced931220 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009fd0 tx=0x7f0a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ed65eee5cf45e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a2effd640 1 -- 172.21.15.92:0/737520018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38038070 con 0x55eced5652f0 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a2effd640 1 -- 172.21.15.92:0/737520018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a38023cb0 con 0x55eced5652f0 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.163+0000 7f0a2effd640 1 -- 172.21.15.92:0/737520018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3802f040 con 0x55eced5652f0 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 -- 172.21.15.92:0/737520018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 msgr2=0x55eced931220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced931220 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009fd0 tx=0x7f0a380046c0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 -- 172.21.15.92:0/737520018 shutdown_connections 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced93daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced947620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 --2- 172.21.15.92:0/737520018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced931220 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 -- 172.21.15.92:0/737520018 >> 172.21.15.92:0/737520018 conn(0x55eced582300 msgr2=0x55eced936ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 -- 172.21.15.92:0/737520018 shutdown_connections 2024-09-21T02:15:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 -- 172.21.15.92:0/737520018 wait complete. 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 -- start start 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.164+0000 7f0a52882740 1 Processor -- start 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced9473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced946080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced9466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441ab370 con 0x55eced5652f0 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441aaca0 con 0x55eced58e1b0 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a441a7ee0 con 0x55eced56d4d0 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a52882740 10 client.0 _ll_register_callbacks cb 0x7f0a38009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced9473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced9473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59242/0 (socket says 172.21.15.92:59242) 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 -- 172.21.15.92:0/2154586499 learned_addr learned my addr 172.21.15.92:0/2154586499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a48eae640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced946080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced9466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 -- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 msgr2=0x55eced946080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced946080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 -- 172.21.15.92:0/2154586499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 msgr2=0x55eced9466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced9466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 -- 172.21.15.92:0/2154586499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eced946e40 con 0x55eced5652f0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a49eb0640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced9466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a496af640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced9473c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f0a38019040 tx=0x7f0a38005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5720d8a700a394 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a2cff9640 1 -- 172.21.15.92:0/2154586499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38041060 con 0x55eced5652f0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a38023cb0 con 0x55eced5652f0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.165+0000 7f0a2cff9640 1 -- 172.21.15.92:0/2154586499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0a38004730 con 0x55eced5652f0 2024-09-21T02:15:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a2cff9640 1 -- 172.21.15.92:0/2154586499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a38038070 con 0x55eced5652f0 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 10 client.18477 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eced9b7200 con 0x55eced5652f0 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a2cff9640 1 -- 172.21.15.92:0/2154586499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a38023cb0 con 0x55eced5652f0 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a2cff9640 1 -- 172.21.15.92:0/2154586499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a3802dc40 con 0x55eced5652f0 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 10 client.18477 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 20 client.18477 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced5221c0 0x55eced9b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 4 client.18477 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eced8a60f0 con 0x55eced5221c0 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.166+0000 7f0a48eae640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced5221c0 0x55eced9b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.167+0000 7f0a2cff9640 10 client.18477 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.167+0000 7f0a48eae640 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced5221c0 0x55eced9b9c20 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b3c2d70aae1901 server_cookie=3af08ed822a2d3d7 in_seq=0 out_seq=0 2024-09-21T02:15:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a2cff9640 1 -- 172.21.15.92:0/2154586499 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eced8a60f0 con 0x55eced5221c0 2024-09-21T02:15:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a2cff9640 10 client.18477 handle_command_reply: tid=0 2024-09-21T02:15:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 client.18477 shutdown 2024-09-21T02:15:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 msgr2=0x55eced9473c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced9473c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f0a38019040 tx=0x7f0a38005230 comp rx=0 tx=0).stop 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 shutdown_connections 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55eced5221c0 0x55eced9b9c20 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eced58e1b0 0x55eced9466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a2cff9640 0 client.18477 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eced56d4d0 0x55eced946080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 --2- 172.21.15.92:0/2154586499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eced5652f0 0x55eced9473c0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 >> 172.21.15.92:0/2154586499 conn(0x55eced582300 msgr2=0x55eced93da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 shutdown_connections 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 1 -- 172.21.15.92:0/2154586499 wait complete. 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.183+0000 7f0a52882740 20 client.18477 trim_cache size 0 max 16384 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:15:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:15:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:15:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:15:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0a300b6590 msgr2=0x7f0a300b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0a300b6590 0x7f0a300b8aa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009cf0 tx=0x7f0a38005f90 comp rx=0 tx=0).stop 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 msgr2=0x7f0a44199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 0x7f0a44199470 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c890 tx=0x7f0a4000cd50 comp rx=0 tx=0).stop 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 shutdown_connections 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0a300b6590 0x7f0a300b8aa0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a44103fe0 0x7f0a4419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a44103640 0x7f0a44199470 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 --2- 172.21.15.92:0/2483966270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a441025c0 0x7f0a44198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 >> 172.21.15.92:0/2483966270 conn(0x7f0a440fdf00 msgr2=0x7f0a44106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 shutdown_connections 2024-09-21T02:15:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:13.184+0000 7f0a4b913640 1 -- 172.21.15.92:0/2483966270 wait complete. 2024-09-21T02:15:15.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:14 smithi136 ceph-mon[34754]: pgmap v668: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:15:15.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:14 smithi092 ceph-mon[31190]: pgmap v668: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:15:15.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:14 smithi099 ceph-mon[34404]: pgmap v668: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-09-21T02:15:17.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:16 smithi136 ceph-mon[34754]: pgmap v669: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:15:17.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:16 smithi092 ceph-mon[31190]: pgmap v669: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:15:17.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:16 smithi099 ceph-mon[34404]: pgmap v669: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-21T02:15:19.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:18 smithi136 ceph-mon[34754]: pgmap v670: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:15:19.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:18 smithi092 ceph-mon[31190]: pgmap v670: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:15:19.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:18 smithi099 ceph-mon[34404]: pgmap v670: 97 pgs: 97 active+clean; 3.8 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T02:15:21.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:20 smithi136 ceph-mon[34754]: pgmap v671: 97 pgs: 97 active+clean; 3.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:15:21.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:20 smithi092 ceph-mon[31190]: pgmap v671: 97 pgs: 97 active+clean; 3.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:15:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:20 smithi099 ceph-mon[34404]: pgmap v671: 97 pgs: 97 active+clean; 3.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-09-21T02:15:23.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:22 smithi136 ceph-mon[34754]: pgmap v672: 97 pgs: 97 active+clean; 3.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:15:23.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:22 smithi092 ceph-mon[31190]: pgmap v672: 97 pgs: 97 active+clean; 3.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:15:23.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:22 smithi099 ceph-mon[34404]: pgmap v672: 97 pgs: 97 active+clean; 3.8 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:15:25.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:24 smithi136 ceph-mon[34754]: pgmap v673: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:25.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:24 smithi092 ceph-mon[31190]: pgmap v673: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:25.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:24 smithi099 ceph-mon[34404]: pgmap v673: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 -- 172.21.15.92:0/2469527030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 msgr2=0x7f9c7410b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/2469527030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7410b920 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009900 tx=0x7f9c6c02a410 comp rx=0 tx=0).stop 2024-09-21T02:15:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 -- 172.21.15.92:0/2469527030 shutdown_connections 2024-09-21T02:15:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/2469527030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7410b920 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/2469527030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c74106b20 0x7f9c74108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/2469527030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c74101a90 0x7f9c74101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 -- 172.21.15.92:0/2469527030 >> 172.21.15.92:0/2469527030 conn(0x7f9c740fd690 msgr2=0x7f9c740ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 -- 172.21.15.92:0/2469527030 shutdown_connections 2024-09-21T02:15:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.291+0000 7f9c78e5c640 1 -- 172.21.15.92:0/2469527030 wait complete. 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 Processor -- start 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 -- start start 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c74101a90 0x7f9c74199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c74106b20 0x7f9c7419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7410f300 con 0x7f9c74109480 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7410f180 con 0x7f9c74101a90 2024-09-21T02:15:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c78e5c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7410f480 con 0x7f9c74106b20 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.292+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57334/0 (socket says 172.21.15.92:57334) 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 -- 172.21.15.92:0/62358248 learned_addr learned my addr 172.21.15.92:0/62358248 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c74106b20 0x7f9c7419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c7259c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c74101a90 0x7f9c74199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 -- 172.21.15.92:0/62358248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c74106b20 msgr2=0x7f9c7419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c74106b20 0x7f9c7419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 -- 172.21.15.92:0/62358248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c74101a90 msgr2=0x7f9c74199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c74101a90 0x7f9c74199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 -- 172.21.15.92:0/62358248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7419b930 con 0x7f9c74109480 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7419b110 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009230 tx=0x7f9c6c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c6b7fe640 1 -- 172.21.15.92:0/62358248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c03f030 con 0x7f9c74109480 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c6b7fe640 1 -- 172.21.15.92:0/62358248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6c003aa0 con 0x7f9c74109480 2024-09-21T02:15:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7419bbc0 con 0x7f9c74109480 2024-09-21T02:15:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c6b7fe640 1 -- 172.21.15.92:0/62358248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c035070 con 0x7f9c74109480 2024-09-21T02:15:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.293+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7419bed0 con 0x7f9c74109480 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.296+0000 7f9c7fdcc740 1 Processor -- start 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.296+0000 7f9c6b7fe640 1 -- 172.21.15.92:0/62358248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f9c6c042030 con 0x7f9c74109480 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.296+0000 7f9c7fdcc740 1 -- start start 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb14e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56304ca876f0 con 0x56304c76d1b0 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56304ca76b20 con 0x56304c74c4d0 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56304ca7a930 con 0x56304c7442f0 2024-09-21T02:15:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c71d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb14e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c71d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb14e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57340/0 (socket says 172.21.15.92:57340) 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c71d9b640 1 -- 172.21.15.92:0/1094197726 learned_addr learned my addr 172.21.15.92:0/1094197726 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7259c640 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7259c640 1 -- 172.21.15.92:0/1094197726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 msgr2=0x56304cb16010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7259c640 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb16010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7259c640 1 -- 172.21.15.92:0/1094197726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 msgr2=0x56304cb14e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7259c640 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb14e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c7259c640 1 -- 172.21.15.92:0/1094197726 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304ca3c530 con 0x56304c7442f0 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c6b7fe640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c600b6510 0x7f9c600b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.297+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c600b6510 0x7f9c600b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.298+0000 7f9c6b7fe640 1 -- 172.21.15.92:0/62358248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c6c107cb0 con 0x7f9c74109480 2024-09-21T02:15:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.298+0000 7f9c7259c640 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb154c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009ec0 tx=0x7f9c64010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed77dc4cfa9f113 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.298+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c600b6510 0x7f9c600b8a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c012910 tx=0x7f9c5c00ba40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.298+0000 7f9c68ff9640 1 -- 172.21.15.92:0/1094197726 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64016070 con 0x56304c7442f0 2024-09-21T02:15:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.298+0000 7f9c68ff9640 1 -- 172.21.15.92:0/1094197726 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6401bb10 con 0x56304c7442f0 2024-09-21T02:15:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.298+0000 7f9c68ff9640 1 -- 172.21.15.92:0/1094197726 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64012be0 con 0x56304c7442f0 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/1094197726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 msgr2=0x56304cb154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb154c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009ec0 tx=0x7f9c64010740 comp rx=0 tx=0).stop 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/1094197726 shutdown_connections 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb16010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb154c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/1094197726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb14e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/1094197726 >> 172.21.15.92:0/1094197726 conn(0x56304c761300 msgr2=0x56304cb108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/1094197726 shutdown_connections 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/1094197726 wait complete. 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 -- start start 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.299+0000 7f9c7fdcc740 1 Processor -- start 2024-09-21T02:15:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 0x56304cb1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c76d1b0 0x56304cb1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56304c9aeab0 con 0x56304c74c4d0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56304ca1c730 con 0x56304c7442f0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56304cb16b30 con 0x56304c76d1b0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 10 client.0 _ll_register_callbacks cb 0x56304ca706b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 0x56304cb1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 0x56304cb1ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57346/0 (socket says 172.21.15.92:57346) 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 -- 172.21.15.92:0/21615352 learned_addr learned my addr 172.21.15.92:0/21615352 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c76d1b0 0x56304cb1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7259c640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 -- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c76d1b0 msgr2=0x56304cb1f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c76d1b0 0x56304cb1f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 -- 172.21.15.92:0/21615352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 msgr2=0x56304cb1e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 -- 172.21.15.92:0/21615352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304c7d8790 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 0x56304cb1ec90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005fd0 tx=0x7f9c5c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4911f8b4d1779339 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c59ffb640 1 -- 172.21.15.92:0/21615352 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0107b0 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c59ffb640 1 -- 172.21.15.92:0/21615352 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c5c01b070 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.300+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c6401bb10 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c59ffb640 1 -- 172.21.15.92:0/21615352 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c5c010950 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 10 client.18489 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56304c851620 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c59ffb640 1 -- 172.21.15.92:0/21615352 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6401bb10 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c59ffb640 1 -- 172.21.15.92:0/21615352 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c5c020420 con 0x56304c74c4d0 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 10 client.18489 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 20 client.18489 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c72bd80 0x56304cb2bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 4 client.18489 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.301+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56304ca89640 con 0x56304c72bd80 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.302+0000 7f9c7259c640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c72bd80 0x56304cb2bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.302+0000 7f9c59ffb640 10 client.18489 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.302+0000 7f9c7259c640 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c72bd80 0x56304cb2bd30 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e683a5077f844941 server_cookie=2fe6b702e858244d in_seq=0 out_seq=0 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c59ffb640 1 -- 172.21.15.92:0/21615352 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56304ca89640 con 0x56304c72bd80 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c59ffb640 10 client.18489 handle_command_reply: tid=0 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 client.18489 shutdown 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 msgr2=0x56304cb1ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 0x56304cb1ec90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005fd0 tx=0x7f9c5c00b410 comp rx=0 tx=0).stop 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 shutdown_connections 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c72bd80 0x56304cb2bd30 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c76d1b0 0x56304cb1f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c74c4d0 0x56304cb1ec90 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c59ffb640 0 client.18489 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/21615352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 >> 172.21.15.92:0/21615352 conn(0x56304c761300 msgr2=0x56304cb0eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 shutdown_connections 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/21615352 wait complete. 2024-09-21T02:15:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.303+0000 7f9c7fdcc740 20 client.18489 trim_cache size 0 max 16384 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 Processor -- start 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 -- start start 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c74c4d0 0x56304cb0b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb1cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741aafd0 con 0x56304c76d1b0 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741acb10 con 0x56304c7442f0 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741ab6a0 con 0x56304c74c4d0 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb1cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb1cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57356/0 (socket says 172.21.15.92:57356) 2024-09-21T02:15:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 -- 172.21.15.92:0/2650622843 learned_addr learned my addr 172.21.15.92:0/2650622843 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c7259c640 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c74c4d0 0x56304cb0b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 -- 172.21.15.92:0/2650622843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c74c4d0 msgr2=0x56304cb0b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c74c4d0 0x56304cb0b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 -- 172.21.15.92:0/2650622843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 msgr2=0x56304cb1d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 -- 172.21.15.92:0/2650622843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304cb163f0 con 0x56304c76d1b0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb1cae0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c105790 tx=0x7f9c6c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfdffbefb4c6241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.318+0000 7f9c5bfff640 1 -- 172.21.15.92:0/2650622843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c049070 con 0x56304c76d1b0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c5bfff640 1 -- 172.21.15.92:0/2650622843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6c0cd930 con 0x56304c76d1b0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c5bfff640 1 -- 172.21.15.92:0/2650622843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c0b50c0 con 0x56304c76d1b0 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/2650622843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 msgr2=0x56304cb1cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb1cae0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c105790 tx=0x7f9c6c009260 comp rx=0 tx=0).stop 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/2650622843 shutdown_connections 2024-09-21T02:15:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb1cae0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c74c4d0 0x56304cb0b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/2650622843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c7442f0 0x56304cb1d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/2650622843 >> 172.21.15.92:0/2650622843 conn(0x56304c761300 msgr2=0x56304cb15ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/2650622843 shutdown_connections 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.319+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/2650622843 wait complete. 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 -- start start 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 Processor -- start 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb25e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb24b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb25060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741ac440 con 0x56304c76d1b0 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741abd70 con 0x56304c74c4d0 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741a8fb0 con 0x56304c7442f0 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7fdcc740 10 client.0 _ll_register_callbacks cb 0x7f9c5c012f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb25060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb25060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57358/0 (socket says 172.21.15.92:57358) 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 -- 172.21.15.92:0/3846828969 learned_addr learned my addr 172.21.15.92:0/3846828969 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb24b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c7259c640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb25e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 -- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 msgr2=0x56304cb25e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb25e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 -- 172.21.15.92:0/3846828969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 msgr2=0x56304cb24b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb24b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 -- 172.21.15.92:0/3846828969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304cb257d0 con 0x56304c76d1b0 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c71d9b640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb24b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c72d9d640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb25060 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c005940 tx=0x7f9c6c0cd870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e516be3a5bee67e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.320+0000 7f9c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c59ffb640 1 -- 172.21.15.92:0/3846828969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c049040 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c59ffb640 1 -- 172.21.15.92:0/3846828969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f9c6c0d75b0 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c6c0cd930 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c59ffb640 1 -- 172.21.15.92:0/3846828969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9c6c0b91b0 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 10 client.18495 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56304cb96730 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c59ffb640 1 -- 172.21.15.92:0/3846828969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c6c0cd930 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c59ffb640 1 -- 172.21.15.92:0/3846828969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9c6c0cd080 con 0x56304c76d1b0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 10 client.18495 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 20 client.18495 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c7011c0 0x56304cb99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 4 client.18495 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.321+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56304ca850f0 con 0x56304c7011c0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.322+0000 7f9c7259c640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c7011c0 0x56304cb99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.322+0000 7f9c59ffb640 10 client.18495 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.322+0000 7f9c7259c640 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c7011c0 0x56304cb99020 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302294281bd0003c server_cookie=9595982b4a1ea40e in_seq=0 out_seq=0 2024-09-21T02:15:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c59ffb640 1 -- 172.21.15.92:0/3846828969 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56304ca850f0 con 0x56304c7011c0 2024-09-21T02:15:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c59ffb640 10 client.18495 handle_command_reply: tid=0 2024-09-21T02:15:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 client.18495 shutdown 2024-09-21T02:15:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 msgr2=0x56304cb25060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb25060 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c005940 tx=0x7f9c6c0cd870 comp rx=0 tx=0).stop 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 shutdown_connections 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x56304c7011c0 0x56304cb99020 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56304c76d1b0 0x56304cb25060 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c59ffb640 0 client.18495 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56304c74c4d0 0x56304cb24b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 --2- 172.21.15.92:0/3846828969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56304c7442f0 0x56304cb25e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 >> 172.21.15.92:0/3846828969 conn(0x56304c761300 msgr2=0x56304cb15ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 shutdown_connections 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 1 -- 172.21.15.92:0/3846828969 wait complete. 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.338+0000 7f9c7fdcc740 20 client.18495 trim_cache size 0 max 16384 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-09-21T02:15:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-09-21T02:15:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-09-21T02:15:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-09-21T02:15:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c600b6510 msgr2=0x7f9c600b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c600b6510 0x7f9c600b8a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c012910 tx=0x7f9c5c00ba40 comp rx=0 tx=0).stop 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 msgr2=0x7f9c7419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7419b110 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009230 tx=0x7f9c6c031a50 comp rx=0 tx=0).stop 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 shutdown_connections 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f9c600b6510 0x7f9c600b8a20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c74109480 0x7f9c7419b110 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c74106b20 0x7f9c7419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 --2- 172.21.15.92:0/62358248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c74101a90 0x7f9c74199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 >> 172.21.15.92:0/62358248 conn(0x7f9c740fd690 msgr2=0x7f9c741078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 shutdown_connections 2024-09-21T02:15:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-09-21T02:15:26.339+0000 7f9c78e5c640 1 -- 172.21.15.92:0/62358248 wait complete. 2024-09-21T02:15:27.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:26 smithi136 ceph-mon[34754]: pgmap v674: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:15:27.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:26 smithi092 ceph-mon[31190]: pgmap v674: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:15:27.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:26 smithi099 ceph-mon[34404]: pgmap v674: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:15:29.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:28 smithi136 ceph-mon[34754]: pgmap v675: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:15:29.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:28 smithi092 ceph-mon[31190]: pgmap v675: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:15:29.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:28 smithi099 ceph-mon[34404]: pgmap v675: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-21T02:15:31.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:30 smithi136 ceph-mon[34754]: pgmap v676: 97 pgs: 97 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:15:31.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:30 smithi092 ceph-mon[31190]: pgmap v676: 97 pgs: 97 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:15:31.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:30 smithi099 ceph-mon[34404]: pgmap v676: 97 pgs: 97 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T02:15:31.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:15:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 -- 172.21.15.92:0/3798023471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c105380 msgr2=0x7f5d2c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 --2- 172.21.15.92:0/3798023471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c105380 0x7f5d2c10d8d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009900 tx=0x7f5d2002a410 comp rx=0 tx=0).stop 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 -- 172.21.15.92:0/3798023471 shutdown_connections 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 --2- 172.21.15.92:0/3798023471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c10de60 0x7f5d2c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 --2- 172.21.15.92:0/3798023471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c105380 0x7f5d2c10d8d0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 --2- 172.21.15.92:0/3798023471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d2c104a30 0x7f5d2c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 -- 172.21.15.92:0/3798023471 >> 172.21.15.92:0/3798023471 conn(0x7f5d2c079760 msgr2=0x7f5d2c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 -- 172.21.15.92:0/3798023471 shutdown_connections 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.482+0000 7f5d3196c640 1 -- 172.21.15.92:0/3798023471 wait complete. 2024-09-21T02:15:32.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 Processor -- start 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 -- start start 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d2c104a30 0x7f5d2c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c105380 0x7f5d2c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 0x7f5d2c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c113aa0 con 0x7f5d2c10de60 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c113920 con 0x7f5d2c105380 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d3196c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c113c20 con 0x7f5d2c104a30 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d2b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 0x7f5d2c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d2b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 0x7f5d2c19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57382/0 (socket says 172.21.15.92:57382) 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d2b7fe640 1 -- 172.21.15.92:0/376182451 learned_addr learned my addr 172.21.15.92:0/376182451 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c105380 0x7f5d2c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.483+0000 7f5d2affd640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d2c104a30 0x7f5d2c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d2b7fe640 1 -- 172.21.15.92:0/376182451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d2c104a30 msgr2=0x7f5d2c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d2b7fe640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d2c104a30 0x7f5d2c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d2b7fe640 1 -- 172.21.15.92:0/376182451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c105380 msgr2=0x7f5d2c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d2b7fe640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c105380 0x7f5d2c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d2b7fe640 1 -- 172.21.15.92:0/376182451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c1a0460 con 0x7f5d2c10de60 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d2b7fe640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 0x7f5d2c19fd30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400d750 tx=0x7f5d2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d3096a640 1 -- 172.21.15.92:0/376182451 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24017070 con 0x7f5d2c10de60 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d3096a640 1 -- 172.21.15.92:0/376182451 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d240054d0 con 0x7f5d2c10de60 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d3096a640 1 -- 172.21.15.92:0/376182451 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24004c60 con 0x7f5d2c10de60 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c1a0750 con 0x7f5d2c10de60 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.484+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c1a6eb0 con 0x7f5d2c10de60 2024-09-21T02:15:32.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.487+0000 7f5d37ee0740 1 Processor -- start 2024-09-21T02:15:32.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.487+0000 7f5d3096a640 1 -- 172.21.15.92:0/376182451 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5d24009bd0 con 0x7f5d2c10de60 2024-09-21T02:15:32.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.487+0000 7f5d37ee0740 1 -- start start 2024-09-21T02:15:32.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec55070 0x55a3deffc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3deffc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3deffccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a3def72e50 con 0x55a3dec2c1b0 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a3def62280 con 0x55a3dec34390 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a3def66090 con 0x55a3dec55070 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3deffc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3deffc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57386/0 (socket says 172.21.15.92:57386) 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 -- 172.21.15.92:0/3768082986 learned_addr learned my addr 172.21.15.92:0/3768082986 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3deffccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec55070 0x55a3deffc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d3096a640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d140b6510 0x7f5d140b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 -- 172.21.15.92:0/3768082986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec55070 msgr2=0x55a3deffc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec55070 0x55a3deffc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 -- 172.21.15.92:0/3768082986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 msgr2=0x55a3deffccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3deffccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2b7fe640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d140b6510 0x7f5d140b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2affd640 1 -- 172.21.15.92:0/3768082986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3decc0650 con 0x55a3dec2c1b0 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.488+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec55070 0x55a3deffc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.489+0000 7f5d3096a640 1 -- 172.21.15.92:0/376182451 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5d240e6c50 con 0x7f5d2c10de60 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.489+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3deffc7b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0103d0 tx=0x7f5d1c010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b10ecae7dba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.489+0000 7f5d11ffb640 1 -- 172.21.15.92:0/3768082986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c011700 con 0x55a3dec2c1b0 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.489+0000 7f5d11ffb640 1 -- 172.21.15.92:0/3768082986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c011d50 con 0x55a3dec2c1b0 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.489+0000 7f5d11ffb640 1 -- 172.21.15.92:0/3768082986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00e3c0 con 0x55a3dec2c1b0 2024-09-21T02:15:32.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.489+0000 7f5d2b7fe640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d140b6510 0x7f5d140b8a20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5d240aa030 tx=0x7f5d2407a0f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3768082986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 msgr2=0x55a3deffc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3deffc7b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0103d0 tx=0x7f5d1c010890 comp rx=0 tx=0).stop 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3768082986 shutdown_connections 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3deffccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3deffc7b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3768082986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec55070 0x55a3deffc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3768082986 >> 172.21.15.92:0/3768082986 conn(0x55a3dec49460 msgr2=0x55a3deffa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3768082986 shutdown_connections 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3768082986 wait complete. 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 -- start start 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.490+0000 7f5d37ee0740 1 Processor -- start 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df007980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 0x55a3df007ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3df013d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a3dedc3700 con 0x55a3dec34390 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a3deefd110 con 0x55a3dec55070 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a3df00aa50 con 0x55a3dec2c1b0 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 0x55a3df007ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d37ee0740 10 client.0 _ll_register_callbacks cb 0x55a3def16990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 0x55a3df007ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57388/0 (socket says 172.21.15.92:57388) 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 -- 172.21.15.92:0/3445568211 learned_addr learned my addr 172.21.15.92:0/3445568211 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3df013d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df007980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 -- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 msgr2=0x55a3df007980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df007980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 -- 172.21.15.92:0/3445568211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 msgr2=0x55a3df013d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3df013d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.491+0000 7f5d2affd640 1 -- 172.21.15.92:0/3445568211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3def2ac40 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 0x55a3df007ec0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004ca0 tx=0x7f5d1c004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c33c5f48576ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d0affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d0affd640 1 -- 172.21.15.92:0/3445568211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c02b060 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d0affd640 1 -- 172.21.15.92:0/3445568211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5d1c011710 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c011d50 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d0affd640 1 -- 172.21.15.92:0/3445568211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c011910 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 10 client.18507 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3df0167c0 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d0affd640 1 -- 172.21.15.92:0/3445568211 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c011d50 con 0x55a3dec34390 2024-09-21T02:15:32.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d0affd640 1 -- 172.21.15.92:0/3445568211 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d1c0056b0 con 0x55a3dec34390 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 10 client.18507 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 20 client.18507 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3dec139f0 0x55a3df019bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 4 client.18507 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.492+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3def74da0 con 0x55a3dec139f0 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.493+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3dec139f0 0x55a3df019bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.493+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3dec139f0 0x55a3df019bd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a3a2c29396da22 server_cookie=e8ad6bd200516182 in_seq=0 out_seq=0 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.493+0000 7f5d0affd640 10 client.18507 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d0affd640 1 -- 172.21.15.92:0/3445568211 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a3def74da0 con 0x55a3dec139f0 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d0affd640 10 client.18507 handle_command_reply: tid=0 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 client.18507 shutdown 2024-09-21T02:15:32.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 msgr2=0x55a3df007ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 0x55a3df007ec0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c004ca0 tx=0x7f5d1c004cd0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 shutdown_connections 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3dec139f0 0x55a3df019bd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3df013d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec34390 0x55a3df007ec0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3445568211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df007980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 >> 172.21.15.92:0/3445568211 conn(0x55a3dec49460 msgr2=0x55a3defff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d0affd640 0 client.18507 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 shutdown_connections 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3445568211 wait complete. 2024-09-21T02:15:32.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.494+0000 7f5d37ee0740 20 client.18507 trim_cache size 0 max 16384 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 Processor -- start 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 -- start start 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3defffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec34390 0x55a3df013cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3deff9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1af490 con 0x55a3dec2c1b0 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b0fd0 con 0x55a3dec55070 2024-09-21T02:15:32.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.512+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1afb60 con 0x55a3dec34390 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3defffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3defffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57404/0 (socket says 172.21.15.92:57404) 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/3361637344 learned_addr learned my addr 172.21.15.92:0/3361637344 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2affd640 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec34390 0x55a3df013cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3deff9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/3361637344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec34390 msgr2=0x55a3df013cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec34390 0x55a3df013cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/3361637344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 msgr2=0x55a3deff9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3deff9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/3361637344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3dec13c20 con 0x55a3dec2c1b0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3defffa80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f5d20000c00 tx=0x7f5d20031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e07418b64a3b547 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d117fa640 1 -- 172.21.15.92:0/3361637344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20035040 con 0x55a3dec2c1b0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d117fa640 1 -- 172.21.15.92:0/3361637344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2004e070 con 0x55a3dec2c1b0 2024-09-21T02:15:32.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.513+0000 7f5d117fa640 1 -- 172.21.15.92:0/3361637344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200049b0 con 0x55a3dec2c1b0 2024-09-21T02:15:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3361637344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 msgr2=0x55a3defffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3defffa80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f5d20000c00 tx=0x7f5d20031400 comp rx=0 tx=0).stop 2024-09-21T02:15:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3361637344 shutdown_connections 2024-09-21T02:15:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec55070 0x55a3deff9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec34390 0x55a3df013cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/3361637344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec2c1b0 0x55a3defffa80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3361637344 >> 172.21.15.92:0/3361637344 conn(0x55a3dec49460 msgr2=0x55a3defff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3361637344 shutdown_connections 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.514+0000 7f5d37ee0740 1 -- 172.21.15.92:0/3361637344 wait complete. 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 -- start start 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 Processor -- start 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df01d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3df01a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 0x55a3df01af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b0900 con 0x55a3dec55070 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1b0230 con 0x55a3dec34390 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1ad470 con 0x55a3dec2c1b0 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d37ee0740 10 client.0 _ll_register_callbacks cb 0x7f5d20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d2a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 0x55a3df01af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d2a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 0x55a3df01af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57410/0 (socket says 172.21.15.92:57410) 2024-09-21T02:15:32.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.515+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/2890857159 learned_addr learned my addr 172.21.15.92:0/2890857159 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2affd640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3df01a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df01d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 msgr2=0x55a3df01d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df01d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/2890857159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 msgr2=0x55a3df01a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3df01a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 -- 172.21.15.92:0/2890857159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3df01b6d0 con 0x55a3dec55070 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d2a7fc640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 0x55a3df01af40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5d20032fd0 tx=0x7f5d200058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc61f4033446fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d0affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d0affd640 1 -- 172.21.15.92:0/2890857159 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20035070 con 0x55a3dec55070 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d0affd640 1 -- 172.21.15.92:0/2890857159 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5d200324f0 con 0x55a3dec55070 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d2004e070 con 0x55a3dec55070 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d0affd640 1 -- 172.21.15.92:0/2890857159 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d200326f0 con 0x55a3dec55070 2024-09-21T02:15:32.521 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d37ee0740 10 client.18513 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.516+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3df084340 con 0x55a3dec55070 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d0affd640 1 -- 172.21.15.92:0/2890857159 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2004e070 con 0x55a3dec55070 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d0affd640 1 -- 172.21.15.92:0/2890857159 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d20004cb0 con 0x55a3dec55070 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d37ee0740 10 client.18513 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d37ee0740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d37ee0740 20 client.18513 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3debe8da0 0x55a3df0869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d37ee0740 4 client.18513 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3def70850 con 0x55a3debe8da0 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d2affd640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3debe8da0 0x55a3df0869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.517+0000 7f5d0affd640 10 client.18513 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d2affd640 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3debe8da0 0x55a3df0869a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd4b4998be97202 server_cookie=5651a6af3be75c5f in_seq=0 out_seq=0 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d0affd640 1 -- 172.21.15.92:0/2890857159 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55a3def70850 con 0x55a3debe8da0 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d0affd640 10 client.18513 handle_command_reply: tid=0 2024-09-21T02:15:32.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 client.18513 shutdown 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 msgr2=0x55a3df01af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 0x55a3df01af40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5d20032fd0 tx=0x7f5d200058d0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 shutdown_connections 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x55a3debe8da0 0x55a3df0869a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a3dec55070 0x55a3df01af40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3dec34390 0x55a3df01a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 --2- 172.21.15.92:0/2890857159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a3dec2c1b0 0x55a3df01d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 >> 172.21.15.92:0/2890857159 conn(0x55a3dec49460 msgr2=0x55a3deff99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d0affd640 0 client.18513 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 shutdown_connections 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 1 -- 172.21.15.92:0/2890857159 wait complete. 2024-09-21T02:15:32.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.518+0000 7f5d37ee0740 20 client.18513 trim_cache size 0 max 16384 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {}, 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_stats": { 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "622c5fe2-42e9-4b30-bbdb-aacf7dfdfd9a": { 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:14:14.805000+0000", 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "80a7a045-f3bc-45a8-ac20-fc7e8853b101": { 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:04:29.535380+0000", 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "07bdbdd7-0ab6-4ee5-b51b-00c4036a0f84": { 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:46.299169+0000", 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "6efa3f01-9ec1-473a-908e-a63bfd3275d7": { 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:00.267321+0000", 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "c97f0c49-9c5d-499d-abe3-b0479f78dccd": { 2024-09-21T02:15:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:06:45.249298+0000", 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "1c962d51-81e7-4a2e-bb15-e30d91c9dc6a": { 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:05:15.343341+0000", 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "c273971d-f2a7-4d64-b784-1321103a1652": { 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:11:15.780116+0000", 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "43fb3d77-e7be-4f74-9a1f-0ad7813d2be4": { 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:07:30.481269+0000", 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "7ad514e7-6d04-4006-a154-862de6f99cc0": { 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:10:31.024143+0000", 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "bed9224a-80ce-4634-bff3-0a2e25dd6f67": { 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:13:30.072031+0000", 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "c7050547-640c-4032-ac18-f3db3ac2d2d9": { 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:08:15.830237+0000", 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "3b26b928-2f6b-4636-9658-0e6fe939ca98": { 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:09:01.124203+0000", 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "7f554d8f-2849-42ec-8c87-5032945b82d0": { 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:14:59.505964+0000", 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "0b7521ce-80b1-4138-a4f9-de46c10a80b9": { 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:00.894081+0000", 2024-09-21T02:15:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "4091c28f-03b5-4162-8d4f-dbcbba169f12": { 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "start_time": "2024-09-21T02:12:45.266061+0000", 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_passed": 0, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_failed": 0, 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_skipped": 0 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:32.532 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d140b6510 msgr2=0x7f5d140b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d140b6510 0x7f5d140b8a20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5d240aa030 tx=0x7f5d2407a0f0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 msgr2=0x7f5d2c19fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 0x7f5d2c19fd30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400d750 tx=0x7f5d2400dc10 comp rx=0 tx=0).stop 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 shutdown_connections 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5d140b6510 0x7f5d140b8a20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d2c10de60 0x7f5d2c19fd30 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c105380 0x7f5d2c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 --2- 172.21.15.92:0/376182451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d2c104a30 0x7f5d2c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 >> 172.21.15.92:0/376182451 conn(0x7f5d2c079760 msgr2=0x7f5d2c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 shutdown_connections 2024-09-21T02:15:32.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:32.519+0000 7f5d3196c640 1 -- 172.21.15.92:0/376182451 wait complete. 2024-09-21T02:15:33.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:32 smithi136 ceph-mon[34754]: pgmap v677: 97 pgs: 97 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:15:33.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:32 smithi092 ceph-mon[31190]: pgmap v677: 97 pgs: 97 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:15:33.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:32 smithi099 ceph-mon[34404]: pgmap v677: 97 pgs: 97 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-21T02:15:33.338 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:15:33.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:15:33.587 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:34.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 -- 172.21.15.92:0/1714390674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8102ae0 msgr2=0x7f2aa8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 --2- 172.21.15.92:0/1714390674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8102ae0 0x7f2aa8102ef0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009900 tx=0x7f2a9802a410 comp rx=0 tx=0).stop 2024-09-21T02:15:34.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 -- 172.21.15.92:0/1714390674 shutdown_connections 2024-09-21T02:15:34.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 --2- 172.21.15.92:0/1714390674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 0x7f2aa810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 --2- 172.21.15.92:0/1714390674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa8108420 0x7f2aa8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 --2- 172.21.15.92:0/1714390674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8102ae0 0x7f2aa8102ef0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 -- 172.21.15.92:0/1714390674 >> 172.21.15.92:0/1714390674 conn(0x7f2aa8079770 msgr2=0x7f2aa8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 -- 172.21.15.92:0/1714390674 shutdown_connections 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.083+0000 7f2aacdab640 1 -- 172.21.15.92:0/1714390674 wait complete. 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.084+0000 7f2aacdab640 1 Processor -- start 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.084+0000 7f2aacdab640 1 -- start start 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aacdab640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa8102ae0 0x7f2aa819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aacdab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 0x7f2aa819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aacdab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 0x7f2aa819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aacdab640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8112a30 con 0x7f2aa8108420 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aacdab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81128b0 con 0x7f2aa8108d70 2024-09-21T02:15:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aacdab640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8112bb0 con 0x7f2aa8102ae0 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 0x7f2aa819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 0x7f2aa819b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34254/0 (socket says 172.21.15.92:34254) 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 -- 172.21.15.92:0/37485765 learned_addr learned my addr 172.21.15.92:0/37485765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aa659c640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa8102ae0 0x7f2aa819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aa6d9d640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 0x7f2aa819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 -- 172.21.15.92:0/37485765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa8102ae0 msgr2=0x7f2aa819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa8102ae0 0x7f2aa819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 -- 172.21.15.92:0/37485765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 msgr2=0x7f2aa819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 0x7f2aa819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 -- 172.21.15.92:0/37485765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa81a65d0 con 0x7f2aa8108420 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2aa6d9d640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 0x7f2aa819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.085+0000 7f2a9dd9b640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 0x7f2aa819b800 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c970 tx=0x7f2a9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.086+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90007bb0 con 0x7f2aa8108420 2024-09-21T02:15:34.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.086+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a90007d50 con 0x7f2aa8108420 2024-09-21T02:15:34.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.086+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000f450 con 0x7f2aa8108420 2024-09-21T02:15:34.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.086+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa81a68c0 con 0x7f2aa8108420 2024-09-21T02:15:34.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.086+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa81a6db0 con 0x7f2aa8108420 2024-09-21T02:15:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.087+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa8103d60 con 0x7f2aa8108420 2024-09-21T02:15:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.089+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2a90004eb0 con 0x7f2aa8108420 2024-09-21T02:15:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.089+0000 7f2a9ffff640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a940b6510 0x7f2a940b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.089+0000 7f2aa659c640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a940b6510 0x7f2a940b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.089+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2a900e7410 con 0x7f2aa8108420 2024-09-21T02:15:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.090+0000 7f2aa659c640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a940b6510 0x7f2a940b8a20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a98000c00 tx=0x7f2a98031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:34.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.093+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a90014030 con 0x7f2aa8108420 2024-09-21T02:15:34.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.304+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aa806bfd0 con 0x7f2aa8108420 2024-09-21T02:15:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.305+0000 7f2a9ffff640 1 -- 172.21.15.92:0/37485765 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f2aa806bfd0 con 0x7f2aa8108420 2024-09-21T02:15:34.306 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:15:34.306 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:15:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a940b6510 msgr2=0x7f2a940b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a940b6510 0x7f2a940b8a20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a98000c00 tx=0x7f2a98031400 comp rx=0 tx=0).stop 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 msgr2=0x7f2aa819b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 0x7f2aa819b800 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c970 tx=0x7f2a9000ce30 comp rx=0 tx=0).stop 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 shutdown_connections 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f2a940b6510 0x7f2a940b8a20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa8108d70 0x7f2aa819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa8108420 0x7f2aa819b800 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 --2- 172.21.15.92:0/37485765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa8102ae0 0x7f2aa819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 >> 172.21.15.92:0/37485765 conn(0x7f2aa8079770 msgr2=0x7f2aa81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 shutdown_connections 2024-09-21T02:15:34.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:34.307+0000 7f2aacdab640 1 -- 172.21.15.92:0/37485765 wait complete. 2024-09-21T02:15:34.908 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:34 smithi092 ceph-mon[31190]: pgmap v678: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:34.908 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:34 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/37485765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:15:34.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:15:35.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:34 smithi136 ceph-mon[34754]: pgmap v678: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:35.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:34 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/37485765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:15:35.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:34 smithi099 ceph-mon[34404]: pgmap v678: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 19 op/s 2024-09-21T02:15:35.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:34 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/37485765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:15:35.183 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 -- 172.21.15.92:0/3491294418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd684107b80 msgr2=0x7fd684107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 --2- 172.21.15.92:0/3491294418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd684107b80 0x7fd684107f90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00aa10 tx=0x7fd66c02b720 comp rx=0 tx=0).stop 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 -- 172.21.15.92:0/3491294418 shutdown_connections 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 --2- 172.21.15.92:0/3491294418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6841084d0 0x7fd68410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 --2- 172.21.15.92:0/3491294418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd684107b80 0x7fd684107f90 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 --2- 172.21.15.92:0/3491294418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd684102240 0x7fd684102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 -- 172.21.15.92:0/3491294418 >> 172.21.15.92:0/3491294418 conn(0x7fd684079f90 msgr2=0x7fd68407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 -- 172.21.15.92:0/3491294418 shutdown_connections 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 -- 172.21.15.92:0/3491294418 wait complete. 2024-09-21T02:15:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 Processor -- start 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.627+0000 7fd68a615640 1 -- start start 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd68a615640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd684102240 0x7fd68419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd68a615640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd684107b80 0x7fd68419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd68a615640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 0x7fd68419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd68a615640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd684111950 con 0x7fd6841084d0 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd68a615640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841117d0 con 0x7fd684102240 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd68a615640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd684111ad0 con 0x7fd684107b80 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 0x7fd68419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 0x7fd68419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34270/0 (socket says 172.21.15.92:34270) 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 -- 172.21.15.92:0/3742451518 learned_addr learned my addr 172.21.15.92:0/3742451518 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd67bfff640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd684107b80 0x7fd68419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd683fff640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd684102240 0x7fd68419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 -- 172.21.15.92:0/3742451518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd684107b80 msgr2=0x7fd68419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd684107b80 0x7fd68419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 -- 172.21.15.92:0/3742451518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd684102240 msgr2=0x7fd68419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd684102240 0x7fd68419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 -- 172.21.15.92:0/3742451518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6841a04e0 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd683fff640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd684102240 0x7fd68419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd688bb2640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 0x7fd68419fdb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00b970 tx=0x7fd67c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.628+0000 7fd681ffb640 1 -- 172.21.15.92:0/3742451518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c019070 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.629+0000 7fd681ffb640 1 -- 172.21.15.92:0/3742451518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c014440 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.629+0000 7fd681ffb640 1 -- 172.21.15.92:0/3742451518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0075d0 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.629+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6841a07d0 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.629+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841a6f30 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.631+0000 7fd690b89740 1 Processor -- start 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.631+0000 7fd681ffb640 1 -- 172.21.15.92:0/3742451518 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fd67c01c070 con 0x7fd6841084d0 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.631+0000 7fd690b89740 1 -- start start 2024-09-21T02:15:35.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd690b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df76f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd690b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df76fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd690b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 0x55a6df770120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd690b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a6df6e5ea0 con 0x55a6df3a7390 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd690b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a6df6d52d0 con 0x55a6df39f1b0 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd690b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6df6d90e0 con 0x55a6df3c8070 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 0x55a6df770120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd683fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df76fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 0x55a6df770120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34274/0 (socket says 172.21.15.92:34274) 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 -- 172.21.15.92:0/1427406528 learned_addr learned my addr 172.21.15.92:0/1427406528 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df76f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 -- 172.21.15.92:0/1427406528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 msgr2=0x55a6df76f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df76f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 -- 172.21.15.92:0/1427406528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 msgr2=0x55a6df76fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df76fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd67bfff640 1 -- 172.21.15.92:0/1427406528 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6df433650 con 0x55a6df3a7390 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd683fff640 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df76fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.632+0000 7fd681ffb640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6700b6510 0x7fd6700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.633+0000 7fd67bfff640 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 0x55a6df770120 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0057c0 tx=0x7fd66c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4a39bcbd03dc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.633+0000 7fd683fff640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6700b6510 0x7fd6700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.633+0000 7fd681ffb640 1 -- 172.21.15.92:0/3742451518 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd67c0e67d0 con 0x7fd6841084d0 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.633+0000 7fd67a7fc640 1 -- 172.21.15.92:0/1427406528 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c036040 con 0x55a6df3a7390 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.633+0000 7fd67a7fc640 1 -- 172.21.15.92:0/1427406528 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd66c005030 con 0x55a6df3a7390 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.633+0000 7fd67a7fc640 1 -- 172.21.15.92:0/1427406528 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004950 con 0x55a6df3a7390 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 -- 172.21.15.92:0/1427406528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 msgr2=0x55a6df770120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd683fff640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6700b6510 0x7fd6700b8a20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55a6df76e320 tx=0x7fd67400b5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 0x55a6df770120 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0057c0 tx=0x7fd66c000bc0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 -- 172.21.15.92:0/1427406528 shutdown_connections 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3a7390 0x55a6df770120 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df76fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 --2- 172.21.15.92:0/1427406528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df76f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 -- 172.21.15.92:0/1427406528 >> 172.21.15.92:0/1427406528 conn(0x55a6df3bc460 msgr2=0x55a6df76d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 -- 172.21.15.92:0/1427406528 shutdown_connections 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 -- 172.21.15.92:0/1427406528 wait complete. 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 -- start start 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.634+0000 7fd690b89740 1 Processor -- start 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df77ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3c8070 0x55a6df787140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a6df536700 con 0x55a6df39f1b0 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a6df670160 con 0x55a6df3c8070 2024-09-21T02:15:35.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a6df77de20 con 0x55a6df3a7390 2024-09-21T02:15:35.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd690b89740 10 client.0 _ll_register_callbacks cb 0x55a6df6899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:35.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df77ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df77ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34278/0 (socket says 172.21.15.92:34278) 2024-09-21T02:15:35.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 -- 172.21.15.92:0/2536339631 learned_addr learned my addr 172.21.15.92:0/2536339631 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd67bfff640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd688bb2640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3c8070 0x55a6df787140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 -- 172.21.15.92:0/2536339631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 msgr2=0x55a6df77b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 -- 172.21.15.92:0/2536339631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3c8070 msgr2=0x55a6df787140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3c8070 0x55a6df787140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 -- 172.21.15.92:0/2536339631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6df69dc90 con 0x55a6df39f1b0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd683fff640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df77ad20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd6740108b0 tx=0x7fd6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242f2fbe1b12af56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd6637fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.635+0000 7fd6637fe640 1 -- 172.21.15.92:0/2536339631 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67400fac0 con 0x55a6df39f1b0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd6637fe640 1 -- 172.21.15.92:0/2536339631 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd67400fc60 con 0x55a6df39f1b0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66c005030 con 0x55a6df39f1b0 2024-09-21T02:15:35.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd6637fe640 1 -- 172.21.15.92:0/2536339631 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd67400fe60 con 0x55a6df39f1b0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 10 client.18531 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6df789b80 con 0x55a6df39f1b0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd6637fe640 1 -- 172.21.15.92:0/2536339631 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd66c005030 con 0x55a6df39f1b0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd6637fe640 1 -- 172.21.15.92:0/2536339631 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd67401d450 con 0x55a6df39f1b0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 10 client.18531 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 10 client.18531 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 20 client.18531 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df3869f0 0x55a6df78d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 4 client.18531 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6df6e7df0 con 0x55a6df3869f0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.636+0000 7fd67bfff640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df3869f0 0x55a6df78d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.637+0000 7fd6637fe640 10 client.18531 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.637+0000 7fd67bfff640 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df3869f0 0x55a6df78d0b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d71ec16a6be0d56f server_cookie=bd2040a22c68a5a9 in_seq=0 out_seq=0 2024-09-21T02:15:35.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.637+0000 7fd6637fe640 1 -- 172.21.15.92:0/2536339631 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6df6e7df0 con 0x55a6df3869f0 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.637+0000 7fd6637fe640 10 client.18531 handle_command_reply: tid=0 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.637+0000 7fd690b89740 1 client.18531 shutdown 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 msgr2=0x55a6df77ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df77ad20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd6740108b0 tx=0x7fd6740046c0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 shutdown_connections 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df3869f0 0x55a6df78d0b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3c8070 0x55a6df787140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 --2- 172.21.15.92:0/2536339631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df77ad20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd6637fe640 0 client.18531 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 >> 172.21.15.92:0/2536339631 conn(0x55a6df3bc460 msgr2=0x55a6df772830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 shutdown_connections 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 1 -- 172.21.15.92:0/2536339631 wait complete. 2024-09-21T02:15:35.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.638+0000 7fd690b89740 20 client.18531 trim_cache size 0 max 16384 2024-09-21T02:15:35.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 Processor -- start 2024-09-21T02:15:35.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 -- start start 2024-09-21T02:15:35.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df779c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3a7390 0x55a6df77c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df787040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841af5b0 con 0x55a6df39f1b0 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b10f0 con 0x55a6df3a7390 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd690b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841afc80 con 0x55a6df3c8070 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd683fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df779c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd683fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df779c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34280/0 (socket says 172.21.15.92:34280) 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd683fff640 1 -- 172.21.15.92:0/1120414086 learned_addr learned my addr 172.21.15.92:0/1120414086 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.651+0000 7fd688bb2640 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df787040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd67bfff640 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3a7390 0x55a6df77c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd683fff640 1 -- 172.21.15.92:0/1120414086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 msgr2=0x55a6df787040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd683fff640 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df787040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd683fff640 1 -- 172.21.15.92:0/1120414086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3a7390 msgr2=0x55a6df77c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd683fff640 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3a7390 0x55a6df77c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd683fff640 1 -- 172.21.15.92:0/1120414086 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6df779220 con 0x55a6df39f1b0 2024-09-21T02:15:35.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd683fff640 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df779c10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd67400bdc0 tx=0x7fd67401df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dec35b545c53060 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd679ffb640 1 -- 172.21.15.92:0/1120414086 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67401dba0 con 0x55a6df39f1b0 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd679ffb640 1 -- 172.21.15.92:0/1120414086 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67400fc40 con 0x55a6df39f1b0 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.652+0000 7fd679ffb640 1 -- 172.21.15.92:0/1120414086 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674032790 con 0x55a6df39f1b0 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 -- 172.21.15.92:0/1120414086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 msgr2=0x55a6df779c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df779c10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd67400bdc0 tx=0x7fd67401df30 comp rx=0 tx=0).stop 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 -- 172.21.15.92:0/1120414086 shutdown_connections 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3c8070 0x55a6df787040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df3a7390 0x55a6df77c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 --2- 172.21.15.92:0/1120414086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df39f1b0 0x55a6df779c10 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 -- 172.21.15.92:0/1120414086 >> 172.21.15.92:0/1120414086 conn(0x55a6df3bc460 msgr2=0x55a6df77c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 -- 172.21.15.92:0/1120414086 shutdown_connections 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 -- 172.21.15.92:0/1120414086 wait complete. 2024-09-21T02:15:35.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.653+0000 7fd690b89740 1 -- start start 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 Processor -- start 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df77f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 0x55a6df7f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b0a20 con 0x55a6df3c8070 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b0350 con 0x55a6df39f1b0 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841ad590 con 0x55a6df3a7390 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd690b89740 10 client.0 _ll_register_callbacks cb 0x7fd67400bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 0x55a6df7f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 0x55a6df7f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34296/0 (socket says 172.21.15.92:34296) 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 -- 172.21.15.92:0/1429917153 learned_addr learned my addr 172.21.15.92:0/1429917153 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd67bfff640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd683fff640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df77f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 -- 172.21.15.92:0/1429917153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 msgr2=0x55a6df77fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 -- 172.21.15.92:0/1429917153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 msgr2=0x55a6df77f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df77f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.654+0000 7fd688bb2640 1 -- 172.21.15.92:0/1429917153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6df7f6fb0 con 0x55a6df3c8070 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd688bb2640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 0x55a6df7f6a70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0ac6f0 tx=0x7fd67c014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19bacef2081a12b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd6637fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd6637fe640 1 -- 172.21.15.92:0/1429917153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0b1420 con 0x55a6df3c8070 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd6637fe640 1 -- 172.21.15.92:0/1429917153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd67c00cb40 con 0x55a6df3c8070 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67400fc40 con 0x55a6df3c8070 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd6637fe640 1 -- 172.21.15.92:0/1429917153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd67c0b6360 con 0x55a6df3c8070 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 10 client.18537 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:35.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6df7f8190 con 0x55a6df3c8070 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd6637fe640 1 -- 172.21.15.92:0/1429917153 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67400fc40 con 0x55a6df3c8070 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd6637fe640 1 -- 172.21.15.92:0/1429917153 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd67c0a8340 con 0x55a6df3c8070 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 10 client.18537 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 10 client.18537 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 20 client.18537 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df35bda0 0x55a6df7fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 4 client.18537 mds_command: new command op to 14799 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.655+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 --> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6df6e38a0 con 0x55a6df35bda0 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd683fff640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df35bda0 0x55a6df7fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd6637fe640 10 client.18537 ms_handle_connect on v2:172.21.15.92:6840/2220276047 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd683fff640 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df35bda0 0x55a6df7fa9a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36ff0300e4411d00 server_cookie=b1b3bf1b58c313eb in_seq=0 out_seq=0 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd6637fe640 1 -- 172.21.15.92:0/1429917153 <== mds.4 v2:172.21.15.92:6840/2220276047 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6df6e38a0 con 0x55a6df35bda0 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd6637fe640 10 client.18537 handle_command_reply: tid=0 2024-09-21T02:15:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 client.18537 shutdown 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 msgr2=0x55a6df7f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 0x55a6df7f6a70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0ac6f0 tx=0x7fd67c014d00 comp rx=0 tx=0).stop 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 shutdown_connections 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:6840/2220276047,v1:172.21.15.92:6841/2220276047] conn(0x55a6df35bda0 0x55a6df7fa9a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a6df3c8070 0x55a6df7f6a70 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a6df3a7390 0x55a6df77fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd6637fe640 0 client.18537 ms_handle_reset on v2:172.21.15.92:6840/2220276047 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 --2- 172.21.15.92:0/1429917153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6df39f1b0 0x55a6df77f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.656+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 >> 172.21.15.92:0/1429917153 conn(0x55a6df3bc460 msgr2=0x55a6df77c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 shutdown_connections 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd690b89740 1 -- 172.21.15.92:0/1429917153 wait complete. 2024-09-21T02:15:35.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd690b89740 20 client.18537 trim_cache size 0 max 16384 2024-09-21T02:15:35.668 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:15:35.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6700b6510 msgr2=0x7fd6700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6700b6510 0x7fd6700b8a20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55a6df76e320 tx=0x7fd67400b5a0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 msgr2=0x7fd68419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 0x7fd68419fdb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00b970 tx=0x7fd67c00be30 comp rx=0 tx=0).stop 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 shutdown_connections 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fd6700b6510 0x7fd6700b8a20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6841084d0 0x7fd68419fdb0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd684107b80 0x7fd68419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 --2- 172.21.15.92:0/3742451518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd684102240 0x7fd68419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 >> 172.21.15.92:0/3742451518 conn(0x7fd684079f90 msgr2=0x7fd684103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 shutdown_connections 2024-09-21T02:15:35.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:35.657+0000 7fd68a615640 1 -- 172.21.15.92:0/3742451518 wait complete. 2024-09-21T02:15:36.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:15:36.538 INFO:tasks.workunit.client.0.smithi092.stdout:561/600: open fd = 3 2024-09-21T02:15:36.538 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.538 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.538 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.538 INFO:tasks.workunit.client.0.smithi092.stdout:562/600: open fd = 3 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:563/600: open fd = 3 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:564/600: open fd = 3 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:565/600: open fd = 3 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.539 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:566/600: open fd = 3 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:567/600: open fd = 3 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:568/600: open fd = 3 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:569/600: open fd = 3 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.540 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:570/600: open fd = 3 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:571/600: open fd = 3 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:572/600: open fd = 3 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:573/600: open fd = 3 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.541 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:574/600: open fd = 3 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:575/600: open fd = 3 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:576/600: open fd = 3 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:577/600: open fd = 3 2024-09-21T02:15:36.542 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:578/600: open fd = 3 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:579/600: open fd = 3 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:580/600: open fd = 3 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.543 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:581/600: open fd = 3 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:582/600: open fd = 3 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:583/600: open fd = 3 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:584/600: open fd = 3 2024-09-21T02:15:36.544 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:585/600: open fd = 3 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:586/600: open fd = 3 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:587/600: open fd = 3 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.545 INFO:tasks.workunit.client.0.smithi092.stdout:588/600: open fd = 3 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:589/600: open fd = 3 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:590/600: open fd = 3 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:591/600: open fd = 3 2024-09-21T02:15:36.546 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:592/600: open fd = 3 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:593/600: open fd = 3 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:594/600: open fd = 3 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.547 INFO:tasks.workunit.client.0.smithi092.stdout:595/600: open fd = 3 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:596/600: open fd = 3 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:597/600: open fd = 3 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:598/600: open fd = 3 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.548 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:599/600: open fd = 3 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:600/600: open fd = 3 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-09-21T02:15:36.549 INFO:tasks.workunit.client.0.smithi092.stderr:+ echo OK 2024-09-21T02:15:36.550 INFO:tasks.workunit.client.0.smithi092.stdout:OK 2024-09-21T02:15:36.550 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T02:15:36.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:15:36.587 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-21T02:15:36.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T02:15:36.644 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:36.930 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:36 smithi092 ceph-mon[31190]: pgmap v679: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:15:36.931 DEBUG:teuthology.parallel:result is None 2024-09-21T02:15:36.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T02:15:37.007 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T02:15:37.007 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T02:15:37.017 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:15:37.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:36 smithi136 ceph-mon[34754]: pgmap v679: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:15:37.056 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T02:15:37.064 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T02:15:37.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:36 smithi099 ceph-mon[34404]: pgmap v679: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-09-21T02:15:37.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 -- 172.21.15.92:0/2381744852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00102ae0 msgr2=0x7f5e00102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 --2- 172.21.15.92:0/2381744852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00102ae0 0x7f5e00102ef0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009900 tx=0x7f5df002a410 comp rx=0 tx=0).stop 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 -- 172.21.15.92:0/2381744852 shutdown_connections 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 --2- 172.21.15.92:0/2381744852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00108d70 0x7f5e0010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 --2- 172.21.15.92:0/2381744852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e00108420 0x7f5e00108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 --2- 172.21.15.92:0/2381744852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00102ae0 0x7f5e00102ef0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 -- 172.21.15.92:0/2381744852 >> 172.21.15.92:0/2381744852 conn(0x7f5e00079770 msgr2=0x7f5e00079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 -- 172.21.15.92:0/2381744852 shutdown_connections 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.190+0000 7f5e0830e640 1 -- 172.21.15.92:0/2381744852 wait complete. 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 Processor -- start 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 -- start start 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00102ae0 0x7f5e0019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e00108420 0x7f5e0019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 0x7f5e0019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00112af0 con 0x7f5e00108d70 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00112970 con 0x7f5e00108420 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e0830e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00112c70 con 0x7f5e00102ae0 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e068ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 0x7f5e0019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e068ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 0x7f5e0019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34314/0 (socket says 172.21.15.92:34314) 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e068ab640 1 -- 172.21.15.92:0/4182743877 learned_addr learned my addr 172.21.15.92:0/4182743877 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.191+0000 7f5e060aa640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00102ae0 0x7f5e0019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e058a9640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e00108420 0x7f5e0019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e068ab640 1 -- 172.21.15.92:0/4182743877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00102ae0 msgr2=0x7f5e0019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e068ab640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00102ae0 0x7f5e0019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e068ab640 1 -- 172.21.15.92:0/4182743877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e00108420 msgr2=0x7f5e0019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e068ab640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e00108420 0x7f5e0019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e068ab640 1 -- 172.21.15.92:0/4182743877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001a66d0 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e060aa640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00102ae0 0x7f5e0019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e068ab640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 0x7f5e0019bec0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b970 tx=0x7f5df800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5df77fe640 1 -- 172.21.15.92:0/4182743877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8019070 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e001a69c0 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5df77fe640 1 -- 172.21.15.92:0/4182743877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df8014440 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5df77fe640 1 -- 172.21.15.92:0/4182743877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80075d0 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.192+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001a6f30 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 Processor -- start 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5df77fe640 1 -- 172.21.15.92:0/4182743877 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5df801c070 con 0x7f5e00108d70 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 -- start start 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b5e070 0x55fc65f056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc65e7bea0 con 0x55fc65b351b0 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fc65e6b2d0 con 0x55fc65b3d390 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e0e882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc65e6f0e0 con 0x55fc65b5e070 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e060aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e060aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f05be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34324/0 (socket says 172.21.15.92:34324) 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e060aa640 1 -- 172.21.15.92:0/2695148500 learned_addr learned my addr 172.21.15.92:0/2695148500 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.195+0000 7f5e058a9640 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e058a9640 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b5e070 0x55fc65f056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5df77fe640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5dec0b6540 0x7f5dec0b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e060aa640 1 -- 172.21.15.92:0/2695148500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b5e070 msgr2=0x55fc65f056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e060aa640 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b5e070 0x55fc65f056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e060aa640 1 -- 172.21.15.92:0/2695148500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 msgr2=0x55fc65f06120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e060aa640 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f06120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e060aa640 1 -- 172.21.15.92:0/2695148500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc65bc9650 con 0x55fc65b351b0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e068ab640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5dec0b6540 0x7f5dec0b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5df77fe640 1 -- 172.21.15.92:0/4182743877 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5df80e6860 con 0x7f5e00108d70 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5e060aa640 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f05be0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5df0031440 tx=0x7f5df0031e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822bed651b2f9f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5df4d9a640 1 -- 172.21.15.92:0/2695148500 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df003f070 con 0x55fc65b351b0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5df4d9a640 1 -- 172.21.15.92:0/2695148500 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df00316a0 con 0x55fc65b351b0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.196+0000 7f5df4d9a640 1 -- 172.21.15.92:0/2695148500 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0035070 con 0x55fc65b351b0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 -- 172.21.15.92:0/2695148500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 msgr2=0x55fc65f05be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e068ab640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5dec0b6540 0x7f5dec0b8a50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5df80a8fa0 tx=0x7f5df8079d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f05be0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5df0031440 tx=0x7f5df0031e50 comp rx=0 tx=0).stop 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 -- 172.21.15.92:0/2695148500 shutdown_connections 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f05be0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 --2- 172.21.15.92:0/2695148500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b5e070 0x55fc65f056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 -- 172.21.15.92:0/2695148500 >> 172.21.15.92:0/2695148500 conn(0x55fc65b52460 msgr2=0x55fc65f03590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 -- 172.21.15.92:0/2695148500 shutdown_connections 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.197+0000 7f5e0e882740 1 -- 172.21.15.92:0/2695148500 wait complete. 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 -- start start 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 Processor -- start 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f150e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f15620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 0x55fc65f15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc65ccc700 con 0x55fc65b5e070 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fc65e06160 con 0x55fc65b3d390 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc65f18180 con 0x55fc65b351b0 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e0e882740 10 client.0 _ll_register_callbacks cb 0x55fc65e1f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e058a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 0x55fc65f15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e058a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 0x55fc65f15b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34326/0 (socket says 172.21.15.92:34326) 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e058a9640 1 -- 172.21.15.92:0/321923450 learned_addr learned my addr 172.21.15.92:0/321923450 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e058a9640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.198+0000 7f5e060aa640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f15620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e058a9640 1 -- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 msgr2=0x55fc65f150e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e058a9640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f150e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e058a9640 1 -- 172.21.15.92:0/321923450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 msgr2=0x55fc65f15620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e058a9640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f15620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e058a9640 1 -- 172.21.15.92:0/321923450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc65e33c90 con 0x55fc65b5e070 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e058a9640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 0x55fc65f15b60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006b9c0 tx=0x7f5dfc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f1ed5c106713e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5de9ffb640 1 -- 172.21.15.92:0/321923450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc019030 con 0x55fc65b5e070 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df00316a0 con 0x55fc65b5e070 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.199+0000 7f5de9ffb640 1 -- 172.21.15.92:0/321923450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc0191d0 con 0x55fc65b5e070 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5de9ffb640 1 -- 172.21.15.92:0/321923450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dfc018450 con 0x55fc65b5e070 2024-09-21T02:15:37.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 10 client.18549 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc65f23ea0 con 0x55fc65b5e070 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5de9ffb640 1 -- 172.21.15.92:0/321923450 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc018aa0 con 0x55fc65b5e070 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5de9ffb640 1 -- 172.21.15.92:0/321923450 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dfc023030 con 0x55fc65b5e070 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 10 client.18549 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 10 client.18549 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 20 client.18549 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65b1c9f0 0x55fc65f27370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 4 client.18549 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5df00316a0 con 0x55fc65b1c9f0 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.200+0000 7f5e060aa640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65b1c9f0 0x55fc65f27370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.201+0000 7f5e060aa640 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65b1c9f0 0x55fc65f27370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44c72a6fcc4a9fe0 server_cookie=fea7a83c67916e56 in_seq=0 out_seq=0 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.201+0000 7f5de9ffb640 10 client.18549 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:15:37.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5de9ffb640 1 -- 172.21.15.92:0/321923450 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5df00316a0 con 0x55fc65b1c9f0 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5de9ffb640 10 client.18549 handle_command_reply: tid=0 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 client.18549 shutdown 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 msgr2=0x55fc65f15b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 0x55fc65f15b60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006b9c0 tx=0x7f5dfc0047d0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 shutdown_connections 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b5e070 0x55fc65f15b60 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65b1c9f0 0x55fc65f27370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b3d390 0x55fc65f15620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 --2- 172.21.15.92:0/321923450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 >> 172.21.15.92:0/321923450 conn(0x55fc65b52460 msgr2=0x55fc65f01bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5de9ffb640 0 client.18549 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 shutdown_connections 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 1 -- 172.21.15.92:0/321923450 wait complete. 2024-09-21T02:15:37.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.202+0000 7f5e0e882740 20 client.18549 trim_cache size 0 max 16384 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 Processor -- start 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 -- start start 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f0af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 0x55fc65f2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f2d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ad430 con 0x55fc65b3d390 2024-09-21T02:15:37.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001aef70 con 0x55fc65b5e070 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.217+0000 7f5e0e882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001adb00 con 0x55fc65b351b0 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 0x55fc65f2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 0x55fc65f2cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34330/0 (socket says 172.21.15.92:34330) 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 -- 172.21.15.92:0/3397163130 learned_addr learned my addr 172.21.15.92:0/3397163130 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e058a9640 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f2d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e058a9640 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f0af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 -- 172.21.15.92:0/3397163130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 msgr2=0x55fc65f0af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f0af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 -- 172.21.15.92:0/3397163130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 msgr2=0x55fc65f2d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f2d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 -- 172.21.15.92:0/3397163130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc65f15ea0 con 0x55fc65b3d390 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e058a9640 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f0af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:37.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5e060aa640 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 0x55fc65f2cb30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5df002a930 tx=0x7f5df0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c010d51121e89f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5debfff640 1 -- 172.21.15.92:0/3397163130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0035040 con 0x55fc65b3d390 2024-09-21T02:15:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5debfff640 1 -- 172.21.15.92:0/3397163130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df0009780 con 0x55fc65b3d390 2024-09-21T02:15:37.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.218+0000 7f5debfff640 1 -- 172.21.15.92:0/3397163130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df003f040 con 0x55fc65b3d390 2024-09-21T02:15:37.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 -- 172.21.15.92:0/3397163130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 msgr2=0x55fc65f2cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 0x55fc65f2cb30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5df002a930 tx=0x7f5df0005280 comp rx=0 tx=0).stop 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 -- 172.21.15.92:0/3397163130 shutdown_connections 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f2d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b3d390 0x55fc65f2cb30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 --2- 172.21.15.92:0/3397163130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b351b0 0x55fc65f0af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 -- 172.21.15.92:0/3397163130 >> 172.21.15.92:0/3397163130 conn(0x55fc65b52460 msgr2=0x55fc65f0a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 -- 172.21.15.92:0/3397163130 shutdown_connections 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.219+0000 7f5e0e882740 1 -- 172.21.15.92:0/3397163130 wait complete. 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 -- start start 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 Processor -- start 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f19440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b3d390 0x55fc65f19980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f19ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ae8a0 con 0x55fc65b351b0 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ae1d0 con 0x55fc65b5e070 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ab410 con 0x55fc65b3d390 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e0e882740 10 client.0 _ll_register_callbacks cb 0x7f5dfc00aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e058a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f19440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e058a9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f19440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34338/0 (socket says 172.21.15.92:34338) 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.220+0000 7f5e058a9640 1 -- 172.21.15.92:0/125966486 learned_addr learned my addr 172.21.15.92:0/125966486 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e060aa640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b3d390 0x55fc65f19980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e058a9640 1 -- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b3d390 msgr2=0x55fc65f19980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e058a9640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b3d390 0x55fc65f19980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e058a9640 1 -- 172.21.15.92:0/125966486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 msgr2=0x55fc65f19ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e058a9640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f19ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e058a9640 1 -- 172.21.15.92:0/125966486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc65f90d60 con 0x55fc65b351b0 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e060aa640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b3d390 0x55fc65f19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:37.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e058a9640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f19440 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006b9c0 tx=0x7f5dfc016970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc69db7ea229395 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5de9ffb640 1 -- 172.21.15.92:0/125966486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc032060 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5de9ffb640 1 -- 172.21.15.92:0/125966486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc016be0 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df0009780 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5de9ffb640 1 -- 172.21.15.92:0/125966486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dfc00a910 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e0e882740 10 client.18555 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc65f91f80 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.221+0000 7f5de9ffb640 1 -- 172.21.15.92:0/125966486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df0009780 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5de9ffb640 1 -- 172.21.15.92:0/125966486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dfc023050 con 0x55fc65b351b0 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e0e882740 10 client.18555 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e0e882740 10 client.18555 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e0e882740 20 client.18555 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:37.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e0e882740 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65af1da0 0x55fc65f947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e0e882740 4 client.18555 mds_command: new command op to 24610 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 --> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc65e7ddf0 con 0x55fc65af1da0 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e060aa640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65af1da0 0x55fc65f947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5e060aa640 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65af1da0 0x55fc65f947f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18687a2c84b8710d server_cookie=88c321b3e2a931dc in_seq=0 out_seq=0 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.222+0000 7f5de9ffb640 10 client.18555 ms_handle_connect on v2:172.21.15.136:6834/2933531534 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5de9ffb640 1 -- 172.21.15.92:0/125966486 <== mds.3 v2:172.21.15.136:6834/2933531534 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc65e7ddf0 con 0x55fc65af1da0 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5de9ffb640 10 client.18555 handle_command_reply: tid=0 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 client.18555 shutdown 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 msgr2=0x55fc65f19440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f19440 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006b9c0 tx=0x7f5dfc016970 comp rx=0 tx=0).stop 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 shutdown_connections 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:6834/2933531534,v1:172.21.15.136:6835/2933531534] conn(0x55fc65af1da0 0x55fc65f947f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc65b5e070 0x55fc65f19ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc65b3d390 0x55fc65f19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 --2- 172.21.15.92:0/125966486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc65b351b0 0x55fc65f19440 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 >> 172.21.15.92:0/125966486 conn(0x55fc65b52460 msgr2=0x55fc65f0a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5de9ffb640 0 client.18555 ms_handle_reset on v2:172.21.15.136:6834/2933531534 2024-09-21T02:15:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 shutdown_connections 2024-09-21T02:15:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 1 -- 172.21.15.92:0/125966486 wait complete. 2024-09-21T02:15:37.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.223+0000 7f5e0e882740 20 client.18555 trim_cache size 0 max 16384 2024-09-21T02:15:37.229 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5dec0b6540 msgr2=0x7f5dec0b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5dec0b6540 0x7f5dec0b8a50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5df80a8fa0 tx=0x7f5df8079d00 comp rx=0 tx=0).stop 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 msgr2=0x7f5e0019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 0x7f5e0019bec0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b970 tx=0x7f5df800be30 comp rx=0 tx=0).stop 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 shutdown_connections 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5dec0b6540 0x7f5dec0b8a50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e00108d70 0x7f5e0019bec0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e00108420 0x7f5e0019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 --2- 172.21.15.92:0/4182743877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e00102ae0 0x7f5e0019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 >> 172.21.15.92:0/4182743877 conn(0x7f5e00079770 msgr2=0x7f5e00106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:37.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 shutdown_connections 2024-09-21T02:15:37.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:37.224+0000 7f5e0830e640 1 -- 172.21.15.92:0/4182743877 wait complete. 2024-09-21T02:15:37.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:15:38.177 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:38.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 -- 172.21.15.92:0/2194190237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881051c0 msgr2=0x7f43881055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 --2- 172.21.15.92:0/2194190237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881051c0 0x7f43881055d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f437c009900 tx=0x7f437c02a410 comp rx=0 tx=0).stop 2024-09-21T02:15:38.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 -- 172.21.15.92:0/2194190237 shutdown_connections 2024-09-21T02:15:38.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 --2- 172.21.15.92:0/2194190237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4388105b10 0x7f438810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 --2- 172.21.15.92:0/2194190237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881051c0 0x7f43881055d0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 --2- 172.21.15.92:0/2194190237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43881031f0 0x7f4388103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 -- 172.21.15.92:0/2194190237 >> 172.21.15.92:0/2194190237 conn(0x7f4388079e40 msgr2=0x7f438807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 -- 172.21.15.92:0/2194190237 shutdown_connections 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.650+0000 7f438f642640 1 -- 172.21.15.92:0/2194190237 wait complete. 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 Processor -- start 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 -- start start 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 0x7f438819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43881051c0 0x7f438819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4388105b10 0x7f438819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43881129e0 con 0x7f43881031f0 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4388112860 con 0x7f4388105b10 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438f642640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4388112b60 con 0x7f43881051c0 2024-09-21T02:15:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 0x7f438819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 0x7f438819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34362/0 (socket says 172.21.15.92:34362) 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 -- 172.21.15.92:0/416337360 learned_addr learned my addr 172.21.15.92:0/416337360 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438dbdf640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4388105b10 0x7f438819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438cbdd640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43881051c0 0x7f438819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 -- 172.21.15.92:0/416337360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43881051c0 msgr2=0x7f438819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43881051c0 0x7f438819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 -- 172.21.15.92:0/416337360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4388105b10 msgr2=0x7f438819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.651+0000 7f438d3de640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4388105b10 0x7f438819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f438d3de640 1 -- 172.21.15.92:0/416337360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43881a0430 con 0x7f43881031f0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f438d3de640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 0x7f438819a4c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f437800c970 tx=0x7f437800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f43867fc640 1 -- 172.21.15.92:0/416337360 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378004220 con 0x7f43881031f0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f43867fc640 1 -- 172.21.15.92:0/416337360 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43780043c0 con 0x7f43881031f0 2024-09-21T02:15:38.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43881a0720 con 0x7f43881031f0 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f43867fc640 1 -- 172.21.15.92:0/416337360 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437800f450 con 0x7f43881031f0 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.652+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43881a6e80 con 0x7f43881031f0 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 Processor -- start 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f43867fc640 1 -- 172.21.15.92:0/416337360 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4378003610 con 0x7f43881031f0 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 -- start start 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a295800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a295d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b01a20bea0 con 0x55b019eee070 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b01a1fb2d0 con 0x55b019ecd390 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f4395bb6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b01a1ff0e0 con 0x55b019ec51b0 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f438dbdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f438dbdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34374/0 (socket says 172.21.15.92:34374) 2024-09-21T02:15:38.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.655+0000 7f438dbdf640 1 -- 172.21.15.92:0/2320225517 learned_addr learned my addr 172.21.15.92:0/2320225517 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438d3de640 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a295800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438cbdd640 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a295d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438dbdf640 1 -- 172.21.15.92:0/2320225517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 msgr2=0x55b01a295800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438dbdf640 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a295800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438dbdf640 1 -- 172.21.15.92:0/2320225517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 msgr2=0x55b01a295d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438dbdf640 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a295d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438dbdf640 1 -- 172.21.15.92:0/2320225517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b019f59650 con 0x55b019eee070 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438cbdd640 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a295d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f43867fc640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f43700b6590 0x7f43700b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438dbdf640 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2952c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4380010370 tx=0x7f4380010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79aa27e93dc6d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f438d3de640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f43700b6590 0x7f43700b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f436f7fe640 1 -- 172.21.15.92:0/2320225517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380016070 con 0x55b019eee070 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f43867fc640 1 -- 172.21.15.92:0/416337360 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f43780e6fc0 con 0x7f43881031f0 2024-09-21T02:15:38.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f436f7fe640 1 -- 172.21.15.92:0/2320225517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4380004d20 con 0x55b019eee070 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.656+0000 7f436f7fe640 1 -- 172.21.15.92:0/2320225517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380005390 con 0x55b019eee070 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 -- 172.21.15.92:0/2320225517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 msgr2=0x55b01a2952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f438d3de640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f43700b6590 0x7f43700b8aa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f43780e4b40 tx=0x7f4378000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2952c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4380010370 tx=0x7f4380010830 comp rx=0 tx=0).stop 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 -- 172.21.15.92:0/2320225517 shutdown_connections 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a295d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a295800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 --2- 172.21.15.92:0/2320225517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2952c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 -- 172.21.15.92:0/2320225517 >> 172.21.15.92:0/2320225517 conn(0x55b019ee2460 msgr2=0x55b01a2931b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 -- 172.21.15.92:0/2320225517 shutdown_connections 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.657+0000 7f4395bb6740 1 -- 172.21.15.92:0/2320225517 wait complete. 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 -- start start 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 Processor -- start 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ec51b0 0x55b01a2a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ecd390 0x55b01a2a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b01a05c700 con 0x55b019eee070 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b01a196160 con 0x55b019ec51b0 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b01a2a3aa0 con 0x55b019ecd390 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.658+0000 7f4395bb6740 10 client.0 _ll_register_callbacks cb 0x55b01a1af9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34378/0 (socket says 172.21.15.92:34378) 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 -- 172.21.15.92:0/1630902239 learned_addr learned my addr 172.21.15.92:0/1630902239 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ecd390 0x55b01a2a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ec51b0 0x55b01a2a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 -- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ecd390 msgr2=0x55b01a2a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ecd390 0x55b01a2a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 -- 172.21.15.92:0/1630902239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ec51b0 msgr2=0x55b01a2a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ec51b0 0x55b01a2a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 -- 172.21.15.92:0/1630902239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a1c3c90 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2acdc0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4380004ff0 tx=0x7f4380012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f3e43904497eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f436cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f436cff9640 1 -- 172.21.15.92:0/1630902239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438001e030 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4380004d20 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f436cff9640 1 -- 172.21.15.92:0/1630902239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4380023070 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f436cff9640 1 -- 172.21.15.92:0/1630902239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4380030050 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f4395bb6740 10 client.18567 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.659+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b01a2af7a0 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f436cff9640 1 -- 172.21.15.92:0/1630902239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4380004d20 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f436cff9640 1 -- 172.21.15.92:0/1630902239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f438000f040 con 0x55b019eee070 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f4395bb6740 10 client.18567 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f4395bb6740 10 client.18567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:15:38.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f4395bb6740 20 client.18567 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eac9f0 0x55b01a2b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f4395bb6740 4 client.18567 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b01a20ddf0 con 0x55b019eac9f0 2024-09-21T02:15:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.660+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eac9f0 0x55b01a2b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.661+0000 7f436cff9640 10 client.18567 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:15:38.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.661+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eac9f0 0x55b01a2b2b60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fab69987e974a5f server_cookie=ea8bf909ece9dd9d in_seq=0 out_seq=0 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.661+0000 7f436cff9640 1 -- 172.21.15.92:0/1630902239 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b01a20ddf0 con 0x55b019eac9f0 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.661+0000 7f436cff9640 10 client.18567 handle_command_reply: tid=0 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.661+0000 7f4395bb6740 1 client.18567 shutdown 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 msgr2=0x55b01a2acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2acdc0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4380004ff0 tx=0x7f4380012ea0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 shutdown_connections 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eac9f0 0x55b01a2b2b60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019eee070 0x55b01a2acdc0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ecd390 0x55b01a2a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f436cff9640 0 client.18567 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:15:38.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1630902239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ec51b0 0x55b01a2a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 >> 172.21.15.92:0/1630902239 conn(0x55b019ee2460 msgr2=0x55b01a298470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:38.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 shutdown_connections 2024-09-21T02:15:38.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 1 -- 172.21.15.92:0/1630902239 wait complete. 2024-09-21T02:15:38.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.662+0000 7f4395bb6740 20 client.18567 trim_cache size 0 max 16384 2024-09-21T02:15:38.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 Processor -- start 2024-09-21T02:15:38.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 -- start start 2024-09-21T02:15:38.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019eac9f0 0x55b01a293200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 0x55b01a2accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a2983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43881af560 con 0x55b019ec51b0 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b10a0 con 0x55b019ecd390 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f4395bb6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43881afc30 con 0x55b019eac9f0 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f438cbdd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 0x55b01a2accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f438cbdd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 0x55b01a2accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34390/0 (socket says 172.21.15.92:34390) 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.675+0000 7f438cbdd640 1 -- 172.21.15.92:0/503722694 learned_addr learned my addr 172.21.15.92:0/503722694 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438dbdf640 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019eac9f0 0x55b01a293200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438cbdd640 1 -- 172.21.15.92:0/503722694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019eac9f0 msgr2=0x55b01a293200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438cbdd640 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019eac9f0 0x55b01a293200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438cbdd640 1 -- 172.21.15.92:0/503722694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 msgr2=0x55b01a2983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438cbdd640 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a2983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438cbdd640 1 -- 172.21.15.92:0/503722694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01a2977e0 con 0x55b019ec51b0 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f438cbdd640 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 0x55b01a2accc0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f437c009900 tx=0x7f437c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283f371ffb7bfbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f436effd640 1 -- 172.21.15.92:0/503722694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c048070 con 0x55b019ec51b0 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f436effd640 1 -- 172.21.15.92:0/503722694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f437c0043e0 con 0x55b019ec51b0 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.676+0000 7f436effd640 1 -- 172.21.15.92:0/503722694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c035040 con 0x55b019ec51b0 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 -- 172.21.15.92:0/503722694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 msgr2=0x55b01a2accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 0x55b01a2accc0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f437c009900 tx=0x7f437c000bc0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 -- 172.21.15.92:0/503722694 shutdown_connections 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019ecd390 0x55b01a2983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ec51b0 0x55b01a2accc0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 --2- 172.21.15.92:0/503722694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019eac9f0 0x55b01a293200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 -- 172.21.15.92:0/503722694 >> 172.21.15.92:0/503722694 conn(0x55b019ee2460 msgr2=0x55b01a292af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 -- 172.21.15.92:0/503722694 shutdown_connections 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 -- 172.21.15.92:0/503722694 wait complete. 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 -- start start 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.677+0000 7f4395bb6740 1 Processor -- start 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019eac9f0 0x55b01a2a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a2a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 0x55b01a2a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b09d0 con 0x55b019ecd390 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43881b0300 con 0x55b019eac9f0 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43881ad540 con 0x55b019ec51b0 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f4395bb6740 10 client.0 _ll_register_callbacks cb 0x7f4380009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 0x55b01a2a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 0x55b01a2a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34404/0 (socket says 172.21.15.92:34404) 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 -- 172.21.15.92:0/1972132208 learned_addr learned my addr 172.21.15.92:0/1972132208 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:38.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a2a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019eac9f0 0x55b01a2a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 -- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 msgr2=0x55b01a2a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a2a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 -- 172.21.15.92:0/1972132208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019eac9f0 msgr2=0x55b01a2a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019eac9f0 0x55b01a2a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 -- 172.21.15.92:0/1972132208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a2a5340 con 0x55b019ecd390 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a2a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.678+0000 7f438dbdf640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 0x55b01a2a4d50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4380009ac0 tx=0x7f4380009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b2f3ecec8d2378 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f436cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f436cff9640 1 -- 172.21.15.92:0/1972132208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438001e040 con 0x55b019ecd390 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f436cff9640 1 -- 172.21.15.92:0/1972132208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f43800057c0 con 0x55b019ecd390 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437c0043e0 con 0x55b019ecd390 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f436cff9640 1 -- 172.21.15.92:0/1972132208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4380016070 con 0x55b019ecd390 2024-09-21T02:15:38.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 10 client.18573 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b01a31d190 con 0x55b019ecd390 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f436cff9640 1 -- 172.21.15.92:0/1972132208 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f437c0043e0 con 0x55b019ecd390 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f436cff9640 1 -- 172.21.15.92:0/1972132208 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4380003b30 con 0x55b019ecd390 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 10 client.18573 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 10 client.18573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 20 client.18573 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eee070 0x55b01a31f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 4 client.18573 mds_command: new command op to 24619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.679+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 --> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b01a2098a0 con 0x55b019eee070 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.680+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eee070 0x55b01a31f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.680+0000 7f436cff9640 10 client.18573 ms_handle_connect on v2:172.21.15.136:6836/3734933147 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.680+0000 7f438cbdd640 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eee070 0x55b01a31f890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bda0110638fe0f14 server_cookie=674a93c5052e3824 in_seq=0 out_seq=0 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.680+0000 7f436cff9640 1 -- 172.21.15.92:0/1972132208 <== mds.2 v2:172.21.15.136:6836/3734933147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b01a2098a0 con 0x55b019eee070 2024-09-21T02:15:38.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.680+0000 7f436cff9640 10 client.18573 handle_command_reply: tid=0 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.680+0000 7f4395bb6740 1 client.18573 shutdown 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 msgr2=0x55b01a2a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 0x55b01a2a4d50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4380009ac0 tx=0x7f4380009600 comp rx=0 tx=0).stop 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 shutdown_connections 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:6836/3734933147,v1:172.21.15.136:6837/3734933147] conn(0x55b019eee070 0x55b01a31f890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b019ecd390 0x55b01a2a4d50 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b019ec51b0 0x55b01a2a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 --2- 172.21.15.92:0/1972132208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b019eac9f0 0x55b01a2a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f436cff9640 0 client.18573 ms_handle_reset on v2:172.21.15.136:6836/3734933147 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 >> 172.21.15.92:0/1972132208 conn(0x55b019ee2460 msgr2=0x55b01a2983c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 shutdown_connections 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 1 -- 172.21.15.92:0/1972132208 wait complete. 2024-09-21T02:15:38.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.681+0000 7f4395bb6740 20 client.18573 trim_cache size 0 max 16384 2024-09-21T02:15:38.688 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:15:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f43700b6590 msgr2=0x7f43700b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f43700b6590 0x7f43700b8aa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f43780e4b40 tx=0x7f4378000bc0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 msgr2=0x7f438819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 0x7f438819a4c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f437800c970 tx=0x7f437800ce30 comp rx=0 tx=0).stop 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 shutdown_connections 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f43700b6590 0x7f43700b8aa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4388105b10 0x7f438819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43881051c0 0x7f438819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 --2- 172.21.15.92:0/416337360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43881031f0 0x7f438819a4c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 >> 172.21.15.92:0/416337360 conn(0x7f4388079e40 msgr2=0x7f43881064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 shutdown_connections 2024-09-21T02:15:38.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:38.682+0000 7f438f642640 1 -- 172.21.15.92:0/416337360 wait complete. 2024-09-21T02:15:39.004 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:38 smithi092 ceph-mon[31190]: pgmap v680: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:15:39.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:38 smithi136 ceph-mon[34754]: pgmap v680: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:15:39.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:38 smithi099 ceph-mon[34404]: pgmap v680: 97 pgs: 97 active+clean; 4.0 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-21T02:15:39.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 -- 172.21.15.92:0/1274560847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28109df0 msgr2=0x7fed28110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 --2- 172.21.15.92:0/1274560847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28109df0 0x7fed28110620 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fed20009ca0 tx=0x7fed2002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 -- 172.21.15.92:0/1274560847 shutdown_connections 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 --2- 172.21.15.92:0/1274560847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed28109df0 0x7fed28110620 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 --2- 172.21.15.92:0/1274560847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed281094a0 0x7fed281098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 --2- 172.21.15.92:0/1274560847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28103b60 0x7fed28103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 -- 172.21.15.92:0/1274560847 >> 172.21.15.92:0/1274560847 conn(0x7fed2807b850 msgr2=0x7fed2807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.176+0000 7fed2e777640 1 -- 172.21.15.92:0/1274560847 shutdown_connections 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.177+0000 7fed2e777640 1 -- 172.21.15.92:0/1274560847 wait complete. 2024-09-21T02:15:40.179 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.177+0000 7fed2e777640 1 Processor -- start 2024-09-21T02:15:40.180 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.177+0000 7fed2e777640 1 -- start start 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2e777640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28103b60 0x7fed2819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2e777640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 0x7fed2819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2e777640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed28109df0 0x7fed2819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2e777640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fed28113ab0 con 0x7fed281094a0 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2e777640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed28113930 con 0x7fed28103b60 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2e777640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed28113c30 con 0x7fed28109df0 2024-09-21T02:15:40.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 0x7fed2819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 0x7fed2819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34430/0 (socket says 172.21.15.92:34430) 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 -- 172.21.15.92:0/4173417169 learned_addr learned my addr 172.21.15.92:0/4173417169 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed27fff640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28103b60 0x7fed2819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2cd14640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed28109df0 0x7fed2819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 -- 172.21.15.92:0/4173417169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed28109df0 msgr2=0x7fed2819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed28109df0 0x7fed2819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 -- 172.21.15.92:0/4173417169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28103b60 msgr2=0x7fed2819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28103b60 0x7fed2819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 -- 172.21.15.92:0/4173417169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed281a0370 con 0x7fed281094a0 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed2cd14640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed28109df0 0x7fed2819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.178+0000 7fed277fe640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 0x7fed2819a9a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00d740 tx=0x7fed1c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.179+0000 7fed257fa640 1 -- 172.21.15.92:0/4173417169 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c017070 con 0x7fed281094a0 2024-09-21T02:15:40.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.179+0000 7fed257fa640 1 -- 172.21.15.92:0/4173417169 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c00de50 con 0x7fed281094a0 2024-09-21T02:15:40.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.179+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed281a0660 con 0x7fed281094a0 2024-09-21T02:15:40.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.179+0000 7fed257fa640 1 -- 172.21.15.92:0/4173417169 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00f620 con 0x7fed281094a0 2024-09-21T02:15:40.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.179+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed281a6dc0 con 0x7fed281094a0 2024-09-21T02:15:40.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.181+0000 7fed34ceb740 1 Processor -- start 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.181+0000 7fed257fa640 1 -- 172.21.15.92:0/4173417169 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fed1c01a070 con 0x7fed281094a0 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.181+0000 7fed34ceb740 1 -- start start 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d5b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d5b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d5bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed34ceb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564ab4cd1ea0 con 0x564ab4993390 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed34ceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564ab4cc12d0 con 0x564ab49b4070 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed34ceb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ab4cc50e0 con 0x564ab498b1b0 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d5bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d5bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34440/0 (socket says 172.21.15.92:34440) 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 -- 172.21.15.92:0/3284423637 learned_addr learned my addr 172.21.15.92:0/3284423637 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:40.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed2cd14640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d5b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed27fff640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d5b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 -- 172.21.15.92:0/3284423637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 msgr2=0x564ab4d5b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d5b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 -- 172.21.15.92:0/3284423637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 msgr2=0x564ab4d5b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d5b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 -- 172.21.15.92:0/3284423637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab4a1f650 con 0x564ab4993390 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed27fff640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d5b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed2cd14640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d5b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed257fa640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fed100b6590 0x7fed100b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed2cd14640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fed100b6590 0x7fed100b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.182+0000 7fed277fe640 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d5bd40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0eae30 tx=0x7fed1c0ef760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d409479a0efa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.183+0000 7fed257fa640 1 -- 172.21.15.92:0/4173417169 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fed1c0e6c90 con 0x7fed281094a0 2024-09-21T02:15:40.186 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.183+0000 7fed0e7fc640 1 -- 172.21.15.92:0/3284423637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0a9450 con 0x564ab4993390 2024-09-21T02:15:40.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.183+0000 7fed0e7fc640 1 -- 172.21.15.92:0/3284423637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c0ac910 con 0x564ab4993390 2024-09-21T02:15:40.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.183+0000 7fed0e7fc640 1 -- 172.21.15.92:0/3284423637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0ac570 con 0x564ab4993390 2024-09-21T02:15:40.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 -- 172.21.15.92:0/3284423637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 msgr2=0x564ab4d5bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d5bd40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0eae30 tx=0x7fed1c0ef760 comp rx=0 tx=0).stop 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed2cd14640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fed100b6590 0x7fed100b8aa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fed20004af0 tx=0x7fed20005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 -- 172.21.15.92:0/3284423637 shutdown_connections 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d5bd40 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d5b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3284423637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d5b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 -- 172.21.15.92:0/3284423637 >> 172.21.15.92:0/3284423637 conn(0x564ab49a8460 msgr2=0x564ab4d591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 -- 172.21.15.92:0/3284423637 shutdown_connections 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 -- 172.21.15.92:0/3284423637 wait complete. 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 -- start start 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.184+0000 7fed34ceb740 1 Processor -- start 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d66910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d66e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d72df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564ab4b22700 con 0x564ab4993390 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564ab4c5c160 con 0x564ab49b4070 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564ab4d69aa0 con 0x564ab498b1b0 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed34ceb740 10 client.0 _ll_register_callbacks cb 0x564ab4c759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d66e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d66e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34456/0 (socket says 172.21.15.92:34456) 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 -- 172.21.15.92:0/2514983818 learned_addr learned my addr 172.21.15.92:0/2514983818 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed27fff640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d66910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed2cd14640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d72df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 -- 172.21.15.92:0/2514983818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 msgr2=0x564ab4d66910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d66910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 -- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 msgr2=0x564ab4d72df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d72df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 -- 172.21.15.92:0/2514983818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab4c89c90 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed277fe640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d66e50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0e47c0 tx=0x7fed1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570786eb8238e271 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.185+0000 7fed077fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed077fe640 1 -- 172.21.15.92:0/2514983818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c021040 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed077fe640 1 -- 172.21.15.92:0/2514983818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fed1c009c30 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1c0ac910 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed077fe640 1 -- 172.21.15.92:0/2514983818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed1c0ac750 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 10 client.18585 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ab4d757b0 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed077fe640 1 -- 172.21.15.92:0/2514983818 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c0ac910 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed077fe640 1 -- 172.21.15.92:0/2514983818 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fed1c0ec050 con 0x564ab4993390 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 10 client.18585 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 10 client.18585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 20 client.18585 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:40.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab49729f0 0x564ab4d78b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 4 client.18585 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.186+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab4cd3df0 con 0x564ab49729f0 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.187+0000 7fed27fff640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab49729f0 0x564ab4d78b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.187+0000 7fed077fe640 10 client.18585 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.187+0000 7fed27fff640 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab49729f0 0x564ab4d78b10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9194d76ebf4673f6 server_cookie=daf6181d397d821e in_seq=0 out_seq=0 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed077fe640 1 -- 172.21.15.92:0/2514983818 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ab4cd3df0 con 0x564ab49729f0 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed077fe640 10 client.18585 handle_command_reply: tid=0 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 client.18585 shutdown 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 msgr2=0x564ab4d66e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d66e50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0e47c0 tx=0x7fed1c000bc0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 shutdown_connections 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab49b4070 0x564ab4d72df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab49729f0 0x564ab4d78b10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab4993390 0x564ab4d66e50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 --2- 172.21.15.92:0/2514983818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d66910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 >> 172.21.15.92:0/2514983818 conn(0x564ab49a8460 msgr2=0x564ab4d5e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed077fe640 0 client.18585 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:15:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 shutdown_connections 2024-09-21T02:15:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 1 -- 172.21.15.92:0/2514983818 wait complete. 2024-09-21T02:15:40.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.188+0000 7fed34ceb740 20 client.18585 trim_cache size 0 max 16384 2024-09-21T02:15:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.201+0000 7fed34ceb740 1 Processor -- start 2024-09-21T02:15:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.201+0000 7fed34ceb740 1 -- start start 2024-09-21T02:15:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab498b1b0 0x564ab4d66dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab4993390 0x564ab4d72d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d63b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed34ceb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fed281af490 con 0x564ab49b4070 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed34ceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b0fd0 con 0x564ab498b1b0 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed34ceb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed281afb60 con 0x564ab4993390 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d63b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d63b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34464/0 (socket says 172.21.15.92:34464) 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 -- 172.21.15.92:0/1248875288 learned_addr learned my addr 172.21.15.92:0/1248875288 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed27fff640 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab498b1b0 0x564ab4d66dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed277fe640 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab4993390 0x564ab4d72d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 -- 172.21.15.92:0/1248875288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab4993390 msgr2=0x564ab4d72d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab4993390 0x564ab4d72d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 -- 172.21.15.92:0/1248875288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab498b1b0 msgr2=0x564ab4d66dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab498b1b0 0x564ab4d66dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 -- 172.21.15.92:0/1248875288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab4d53460 con 0x564ab49b4070 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed2cd14640 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d63b00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fed20009ca0 tx=0x7fed20041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1363957f9316cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed0dffb640 1 -- 172.21.15.92:0/1248875288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20003970 con 0x564ab49b4070 2024-09-21T02:15:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed0dffb640 1 -- 172.21.15.92:0/1248875288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed20036040 con 0x564ab49b4070 2024-09-21T02:15:40.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.202+0000 7fed0dffb640 1 -- 172.21.15.92:0/1248875288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed200047f0 con 0x564ab49b4070 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 -- 172.21.15.92:0/1248875288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 msgr2=0x564ab4d63b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d63b00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fed20009ca0 tx=0x7fed20041000 comp rx=0 tx=0).stop 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 -- 172.21.15.92:0/1248875288 shutdown_connections 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d63b00 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab4993390 0x564ab4d72d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 --2- 172.21.15.92:0/1248875288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab498b1b0 0x564ab4d66dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 -- 172.21.15.92:0/1248875288 >> 172.21.15.92:0/1248875288 conn(0x564ab49a8460 msgr2=0x564ab4d666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:40.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 -- 172.21.15.92:0/1248875288 shutdown_connections 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.203+0000 7fed34ceb740 1 -- 172.21.15.92:0/1248875288 wait complete. 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 -- start start 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 Processor -- start 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d7c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab4993390 0x564ab4d79720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d79c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b0900 con 0x564ab49b4070 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed281b0230 con 0x564ab4993390 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed281ad470 con 0x564ab498b1b0 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed34ceb740 10 client.0 _ll_register_callbacks cb 0x7fed20009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed2cd14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d79c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed2cd14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d79c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34476/0 (socket says 172.21.15.92:34476) 2024-09-21T02:15:40.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed2cd14640 1 -- 172.21.15.92:0/3183820572 learned_addr learned my addr 172.21.15.92:0/3183820572 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed277fe640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab4993390 0x564ab4d79720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.204+0000 7fed27fff640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d7c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed2cd14640 1 -- 172.21.15.92:0/3183820572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 msgr2=0x564ab4d7c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed2cd14640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d7c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed2cd14640 1 -- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab4993390 msgr2=0x564ab4d79720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed2cd14640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab4993390 0x564ab4d79720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed2cd14640 1 -- 172.21.15.92:0/3183820572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab4d7a350 con 0x564ab49b4070 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed2cd14640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d79c60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fed2002f500 tx=0x7fed20003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40c7dfc70186cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed077fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed077fe640 1 -- 172.21.15.92:0/3183820572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20058070 con 0x564ab49b4070 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed077fe640 1 -- 172.21.15.92:0/3183820572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fed20042040 con 0x564ab49b4070 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed20036040 con 0x564ab49b4070 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed077fe640 1 -- 172.21.15.92:0/3183820572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed20004060 con 0x564ab49b4070 2024-09-21T02:15:40.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed34ceb740 10 client.18591 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ab4de7500 con 0x564ab49b4070 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.205+0000 7fed077fe640 1 -- 172.21.15.92:0/3183820572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed20036040 con 0x564ab49b4070 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed077fe640 1 -- 172.21.15.92:0/3183820572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fed2002fa70 con 0x564ab49b4070 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed34ceb740 10 client.18591 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed34ceb740 10 client.18591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed34ceb740 20 client.18591 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab4947da0 0x564ab4de9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed34ceb740 4 client.18591 mds_command: new command op to 24623 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 --> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ab4ccf8a0 con 0x564ab4947da0 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.206+0000 7fed27fff640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab4947da0 0x564ab4de9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed077fe640 10 client.18591 ms_handle_connect on v2:172.21.15.99:6838/2904178656 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed27fff640 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab4947da0 0x564ab4de9c90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9167f8aa3c50882 server_cookie=48ede8dbd2eef35a in_seq=0 out_seq=0 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed077fe640 1 -- 172.21.15.92:0/3183820572 <== mds.1 v2:172.21.15.99:6838/2904178656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ab4ccf8a0 con 0x564ab4947da0 2024-09-21T02:15:40.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed077fe640 10 client.18591 handle_command_reply: tid=0 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 client.18591 shutdown 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 msgr2=0x564ab4d79c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d79c60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fed2002f500 tx=0x7fed20003e70 comp rx=0 tx=0).stop 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 shutdown_connections 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:6838/2904178656,v1:172.21.15.99:6839/2904178656] conn(0x564ab4947da0 0x564ab4de9c90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed077fe640 0 client.18591 ms_handle_reset on v2:172.21.15.99:6838/2904178656 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564ab49b4070 0x564ab4d79c60 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ab4993390 0x564ab4d79720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 --2- 172.21.15.92:0/3183820572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564ab498b1b0 0x564ab4d7c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 >> 172.21.15.92:0/3183820572 conn(0x564ab49a8460 msgr2=0x564ab4d5c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 shutdown_connections 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 1 -- 172.21.15.92:0/3183820572 wait complete. 2024-09-21T02:15:40.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.207+0000 7fed34ceb740 20 client.18591 trim_cache size 0 max 16384 2024-09-21T02:15:40.214 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:15:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fed100b6590 msgr2=0x7fed100b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fed100b6590 0x7fed100b8aa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fed20004af0 tx=0x7fed20005c30 comp rx=0 tx=0).stop 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 msgr2=0x7fed2819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 0x7fed2819a9a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00d740 tx=0x7fed1c00dc00 comp rx=0 tx=0).stop 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 shutdown_connections 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fed100b6590 0x7fed100b8aa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed28109df0 0x7fed2819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed281094a0 0x7fed2819a9a0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 --2- 172.21.15.92:0/4173417169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28103b60 0x7fed2819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 >> 172.21.15.92:0/4173417169 conn(0x7fed2807b850 msgr2=0x7fed28107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 shutdown_connections 2024-09-21T02:15:40.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:40.208+0000 7fed2e777640 1 -- 172.21.15.92:0/4173417169 wait complete. 2024-09-21T02:15:40.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:15:40.946 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:40 smithi092 ceph-mon[31190]: pgmap v681: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:15:41.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:40 smithi136 ceph-mon[34754]: pgmap v681: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:15:41.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:40 smithi099 ceph-mon[34404]: pgmap v681: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 21 op/s 2024-09-21T02:15:41.174 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 -- 172.21.15.92:0/3047429795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 msgr2=0x7f6d2c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 --2- 172.21.15.92:0/3047429795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c102e70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009900 tx=0x7f6d2802a410 comp rx=0 tx=0).stop 2024-09-21T02:15:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 -- 172.21.15.92:0/3047429795 shutdown_connections 2024-09-21T02:15:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 --2- 172.21.15.92:0/3047429795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c108cf0 0x7f6d2c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 --2- 172.21.15.92:0/3047429795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2c1083a0 0x7f6d2c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 --2- 172.21.15.92:0/3047429795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c102e70 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 -- 172.21.15.92:0/3047429795 >> 172.21.15.92:0/3047429795 conn(0x7f6d2c079730 msgr2=0x7f6d2c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 -- 172.21.15.92:0/3047429795 shutdown_connections 2024-09-21T02:15:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.699+0000 7f6d34495640 1 -- 172.21.15.92:0/3047429795 wait complete. 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 Processor -- start 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 -- start start 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c1083a0 0x7f6d2c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2c108cf0 0x7f6d2c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c113a80 con 0x7f6d2c102a60 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c113900 con 0x7f6d2c108cf0 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d34495640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c113c00 con 0x7f6d2c1083a0 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d32231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d32231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34492/0 (socket says 172.21.15.92:34492) 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.700+0000 7f6d32231640 1 -- 172.21.15.92:0/1206168202 learned_addr learned my addr 172.21.15.92:0/1206168202 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d31a30640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c1083a0 0x7f6d2c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d32231640 1 -- 172.21.15.92:0/1206168202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c1083a0 msgr2=0x7f6d2c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d32231640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c1083a0 0x7f6d2c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d32231640 1 -- 172.21.15.92:0/1206168202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2c108cf0 msgr2=0x7f6d2c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d32231640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2c108cf0 0x7f6d2c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d32231640 1 -- 172.21.15.92:0/1206168202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c1a03c0 con 0x7f6d2c102a60 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d31a30640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c1083a0 0x7f6d2c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d32231640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c19a450 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6d28005940 tx=0x7f6d280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d237fe640 1 -- 172.21.15.92:0/1206168202 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2803f030 con 0x7f6d2c102a60 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d237fe640 1 -- 172.21.15.92:0/1206168202 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d28003cb0 con 0x7f6d2c102a60 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2c1a0650 con 0x7f6d2c102a60 2024-09-21T02:15:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d237fe640 1 -- 172.21.15.92:0/1206168202 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28035070 con 0x7f6d2c102a60 2024-09-21T02:15:41.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.701+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2c1a6e10 con 0x7f6d2c102a60 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d237fe640 1 -- 172.21.15.92:0/1206168202 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f6d28042030 con 0x7f6d2c102a60 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 Processor -- start 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 -- start start 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b6b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b6bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b6c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556d30ae1ea0 con 0x556d3079b1b0 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d30ad12d0 con 0x556d307a3390 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556d30ad50e0 con 0x556d307c4070 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.704+0000 7f6d32231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b6bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b6bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34504/0 (socket says 172.21.15.92:34504) 2024-09-21T02:15:41.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 -- 172.21.15.92:0/3844809838 learned_addr learned my addr 172.21.15.92:0/3844809838 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d31a30640 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b6c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32a32640 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b6b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d237fe640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d180b6590 0x7f6d180b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 -- 172.21.15.92:0/3844809838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 msgr2=0x556d30b6b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b6b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 -- 172.21.15.92:0/3844809838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 msgr2=0x556d30b6c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b6c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 -- 172.21.15.92:0/3844809838 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d3082f650 con 0x556d3079b1b0 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32a32640 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b6b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32a32640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d180b6590 0x7f6d180b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d237fe640 1 -- 172.21.15.92:0/1206168202 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6d28107bd0 con 0x7f6d2c102a60 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d32231640 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b6bbe0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6d28038ed0 tx=0x7f6d28041ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabee2df09497463 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.705+0000 7f6d20d9a640 1 -- 172.21.15.92:0/3844809838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d280cd4b0 con 0x556d3079b1b0 2024-09-21T02:15:41.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d32a32640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d180b6590 0x7f6d180b8aa0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009270 tx=0x7f6d24004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:41.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d20d9a640 1 -- 172.21.15.92:0/3844809838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d280372d0 con 0x556d3079b1b0 2024-09-21T02:15:41.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d20d9a640 1 -- 172.21.15.92:0/3844809838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28003e60 con 0x556d3079b1b0 2024-09-21T02:15:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3844809838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 msgr2=0x556d30b6bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b6bbe0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6d28038ed0 tx=0x7f6d28041ab0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3844809838 shutdown_connections 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b6c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b6bbe0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3844809838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b6b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.706+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3844809838 >> 172.21.15.92:0/3844809838 conn(0x556d307b8460 msgr2=0x556d30b69590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3844809838 shutdown_connections 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3844809838 wait complete. 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 -- start start 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 Processor -- start 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556d30932700 con 0x556d3079b1b0 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d30a6c160 con 0x556d307a3390 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556d30ae1ea0 con 0x556d307c4070 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d32231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d32231640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34520/0 (socket says 172.21.15.92:34520) 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d32231640 1 -- 172.21.15.92:0/2381484128 learned_addr learned my addr 172.21.15.92:0/2381484128 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.707+0000 7f6d3aa09740 10 client.0 _ll_register_callbacks cb 0x556d30a859e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32a32640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d31a30640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32231640 1 -- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 msgr2=0x556d30b871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32231640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b871b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32231640 1 -- 172.21.15.92:0/2381484128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 msgr2=0x556d30b7b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32231640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b7b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32231640 1 -- 172.21.15.92:0/2381484128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30a99c90 con 0x556d3079b1b0 2024-09-21T02:15:41.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d31a30640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d32231640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7b1e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6d280cafb0 tx=0x7f6d28032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35192b533a72fa67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d15ffb640 1 -- 172.21.15.92:0/2381484128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28049040 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d15ffb640 1 -- 172.21.15.92:0/2381484128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6d280c9fb0 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d30ae3df0 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d15ffb640 1 -- 172.21.15.92:0/2381484128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d28003e60 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d3aa09740 10 client.18603 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d30b89c30 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.708+0000 7f6d15ffb640 1 -- 172.21.15.92:0/2381484128 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d30ae3df0 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d15ffb640 1 -- 172.21.15.92:0/2381484128 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d280d72d0 con 0x556d3079b1b0 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d3aa09740 10 client.18603 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d3aa09740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d3aa09740 20 client.18603 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:41.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307829f0 0x556d30b8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d3aa09740 4 client.18603 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d30adf8a0 con 0x556d307829f0 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.709+0000 7f6d31a30640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307829f0 0x556d30b8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d31a30640 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307829f0 0x556d30b8cf90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea747b0160c8937 server_cookie=8de22e14c0f82c81 in_seq=0 out_seq=0 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d15ffb640 10 client.18603 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d15ffb640 1 -- 172.21.15.92:0/2381484128 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d30adf8a0 con 0x556d307829f0 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d15ffb640 10 client.18603 handle_command_reply: tid=0 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d3aa09740 1 client.18603 shutdown 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 msgr2=0x556d30b7b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.710+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7b1e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6d280cafb0 tx=0x7f6d28032f60 comp rx=0 tx=0).stop 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 shutdown_connections 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307829f0 0x556d30b8cf90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307c4070 0x556d30b871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307a3390 0x556d30b7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d15ffb640 0 client.18603 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/2381484128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7b1e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 >> 172.21.15.92:0/2381484128 conn(0x556d307b8460 msgr2=0x556d30b6d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 shutdown_connections 2024-09-21T02:15:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 1 -- 172.21.15.92:0/2381484128 wait complete. 2024-09-21T02:15:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.711+0000 7f6d3aa09740 20 client.18603 trim_cache size 0 max 16384 2024-09-21T02:15:41.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 Processor -- start 2024-09-21T02:15:41.726 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 -- start start 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d3079b1b0 0x556d30b783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 0x556d30b78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ad230 con 0x556d307a3390 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1aed70 con 0x556d307829f0 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ad900 con 0x556d3079b1b0 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d32a32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 0x556d30b78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d32a32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 0x556d30b78910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34522/0 (socket says 172.21.15.92:34522) 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d32a32640 1 -- 172.21.15.92:0/657486093 learned_addr learned my addr 172.21.15.92:0/657486093 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d32231640 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.724+0000 7f6d31a30640 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d3079b1b0 0x556d30b783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d32a32640 1 -- 172.21.15.92:0/657486093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d3079b1b0 msgr2=0x556d30b783d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.727 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d32a32640 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d3079b1b0 0x556d30b783d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d32a32640 1 -- 172.21.15.92:0/657486093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 msgr2=0x556d30b7b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d32a32640 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d32a32640 1 -- 172.21.15.92:0/657486093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d30b642d0 con 0x556d307a3390 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d32a32640 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 0x556d30b78910 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6d2401d740 tx=0x7f6d24013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66938bc9276aae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d17fff640 1 -- 172.21.15.92:0/657486093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24012530 con 0x556d307a3390 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d17fff640 1 -- 172.21.15.92:0/657486093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d24012b80 con 0x556d307a3390 2024-09-21T02:15:41.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.725+0000 7f6d17fff640 1 -- 172.21.15.92:0/657486093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2401bbb0 con 0x556d307a3390 2024-09-21T02:15:41.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 -- 172.21.15.92:0/657486093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 msgr2=0x556d30b78910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 0x556d30b78910 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6d2401d740 tx=0x7f6d24013000 comp rx=0 tx=0).stop 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 -- 172.21.15.92:0/657486093 shutdown_connections 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d307a3390 0x556d30b78910 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d3079b1b0 0x556d30b783d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/657486093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 -- 172.21.15.92:0/657486093 >> 172.21.15.92:0/657486093 conn(0x556d307b8460 msgr2=0x556d30b7af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 -- 172.21.15.92:0/657486093 shutdown_connections 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 -- 172.21.15.92:0/657486093 wait complete. 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 -- start start 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.726+0000 7f6d3aa09740 1 Processor -- start 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.732 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307a3390 0x556d30bf6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ae6a0 con 0x556d3079b1b0 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1adfd0 con 0x556d307829f0 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ab210 con 0x556d307a3390 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 10 client.0 _ll_register_callbacks cb 0x7f6d1c00b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34528/0 (socket says 172.21.15.92:34528) 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 -- 172.21.15.92:0/3449689067 learned_addr learned my addr 172.21.15.92:0/3449689067 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d32231640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d32a32640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307a3390 0x556d30bf6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 -- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307a3390 msgr2=0x556d30bf6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307a3390 0x556d30bf6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 -- 172.21.15.92:0/3449689067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 msgr2=0x556d30b7f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.733 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 -- 172.21.15.92:0/3449689067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30bf6db0 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d32a32640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307a3390 0x556d30bf6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d31a30640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7fa10 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005ea0 tx=0x7f6d1c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea353ea572768eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d15ffb640 1 -- 172.21.15.92:0/3449689067 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c010ae0 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d15ffb640 1 -- 172.21.15.92:0/3449689067 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f6d1c010c80 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.727+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d24012b80 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d15ffb640 1 -- 172.21.15.92:0/3449689067 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d1c010e80 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 10 client.18609 fetch_fsmap learned FSMap version 29 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d30bf7ee0 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d15ffb640 1 -- 172.21.15.92:0/3449689067 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d24012b80 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d15ffb640 1 -- 172.21.15.92:0/3449689067 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6d1c022420 con 0x556d3079b1b0 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 10 client.18609 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T02:15:41.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 20 client.18609 populate_metadata read hostname 'smithi092' 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307c4070 0x556d30bfa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 4 client.18609 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 --> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d30b70f50 con 0x556d307c4070 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.728+0000 7f6d32231640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307c4070 0x556d30bfa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d15ffb640 10 client.18609 ms_handle_connect on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d32231640 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307c4070 0x556d30bfa6d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cbe85f86dbc81a server_cookie=da0e6dda6a6600d2 in_seq=0 out_seq=0 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d15ffb640 1 -- 172.21.15.92:0/3449689067 <== mds.0 v2:172.21.15.136:6838/419135468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d30b70f50 con 0x556d307c4070 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d15ffb640 10 client.18609 handle_command_reply: tid=0 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 client.18609 shutdown 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 msgr2=0x556d30b7fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7fa10 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c005ea0 tx=0x7f6d1c011b50 comp rx=0 tx=0).stop 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 shutdown_connections 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:6838/419135468,v1:172.21.15.136:6839/419135468] conn(0x556d307c4070 0x556d30bfa6d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556d307a3390 0x556d30bf6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.735 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d15ffb640 0 client.18609 ms_handle_reset on v2:172.21.15.136:6838/419135468 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d3079b1b0 0x556d30b7fa10 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 --2- 172.21.15.92:0/3449689067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d307829f0 0x556d30b7f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 >> 172.21.15.92:0/3449689067 conn(0x556d307b8460 msgr2=0x556d30b6c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 shutdown_connections 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 1 -- 172.21.15.92:0/3449689067 wait complete. 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.729+0000 7f6d3aa09740 20 client.18609 trim_cache size 0 max 16384 2024-09-21T02:15:41.736 INFO:teuthology.orchestra.run.smithi092.stdout:[] 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d180b6590 msgr2=0x7f6d180b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d180b6590 0x7f6d180b8aa0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009270 tx=0x7f6d24004a40 comp rx=0 tx=0).stop 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 msgr2=0x7f6d2c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c19a450 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6d28005940 tx=0x7f6d280042f0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 shutdown_connections 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f6d180b6590 0x7f6d180b8aa0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2c108cf0 0x7f6d2c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d2c1083a0 0x7f6d2c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 --2- 172.21.15.92:0/1206168202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c102a60 0x7f6d2c19a450 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 >> 172.21.15.92:0/1206168202 conn(0x7f6d2c079730 msgr2=0x7f6d2c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:41.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 shutdown_connections 2024-09-21T02:15:41.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:41.730+0000 7f6d34495640 1 -- 172.21.15.92:0/1206168202 wait complete. 2024-09-21T02:15:42.458 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T02:15:42.459 INFO:tasks.fwd_scrub:done joining 2024-09-21T02:15:42.459 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T02:15:42.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:15:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 -- 172.21.15.92:0/3559617629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103fc0 msgr2=0x7f839010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 --2- 172.21.15.92:0/3559617629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103fc0 0x7f839010a960 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8388009ca0 tx=0x7f838802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 -- 172.21.15.92:0/3559617629 shutdown_connections 2024-09-21T02:15:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 --2- 172.21.15.92:0/3559617629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103fc0 0x7f839010a960 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 --2- 172.21.15.92:0/3559617629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8390103620 0x7f8390103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 --2- 172.21.15.92:0/3559617629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83901025a0 0x7f8390102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 -- 172.21.15.92:0/3559617629 >> 172.21.15.92:0/3559617629 conn(0x7f83900fdee0 msgr2=0x7f8390100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 -- 172.21.15.92:0/3559617629 shutdown_connections 2024-09-21T02:15:42.611 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.608+0000 7f839647e640 1 -- 172.21.15.92:0/3559617629 wait complete. 2024-09-21T02:15:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 Processor -- start 2024-09-21T02:15:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 -- start start 2024-09-21T02:15:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83901025a0 0x7f839006c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 0x7f839019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8390103fc0 0x7f83901a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f839010e1e0 con 0x7f8390103620 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f839010e060 con 0x7f8390103fc0 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.609+0000 7f839647e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f839010e360 con 0x7f83901025a0 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 0x7f839019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 0x7f839019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34544/0 (socket says 172.21.15.92:34544) 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 -- 172.21.15.92:0/348811468 learned_addr learned my addr 172.21.15.92:0/348811468 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838ffff640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83901025a0 0x7f839006c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f8394a1b640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8390103fc0 0x7f83901a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 -- 172.21.15.92:0/348811468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83901025a0 msgr2=0x7f839006c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83901025a0 0x7f839006c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 -- 172.21.15.92:0/348811468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8390103fc0 msgr2=0x7f83901a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8390103fc0 0x7f83901a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 -- 172.21.15.92:0/348811468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83901a1930 con 0x7f8390103620 2024-09-21T02:15:42.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838f7fe640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 0x7f839019ebe0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f837800d950 tx=0x7f837800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:42.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378017070 con 0x7f8390103620 2024-09-21T02:15:42.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8378009c30 con 0x7f8390103620 2024-09-21T02:15:42.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83901a1c20 con 0x7f8390103620 2024-09-21T02:15:42.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837800f620 con 0x7f8390103620 2024-09-21T02:15:42.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.610+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83901a2170 con 0x7f8390103620 2024-09-21T02:15:42.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.611+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390109750 con 0x7f8390103620 2024-09-21T02:15:42.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.613+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f8378010460 con 0x7f8390103620 2024-09-21T02:15:42.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.614+0000 7f838d7fa640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f837c0b6510 0x7f837c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.614+0000 7f838ffff640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f837c0b6510 0x7f837c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.614+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f83780e6f40 con 0x7f8390103620 2024-09-21T02:15:42.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.614+0000 7f838ffff640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f837c0b6510 0x7f837c0b8a20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f838000bbe0 tx=0x7f838000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:42.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.617+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83780ad7b0 con 0x7f8390103620 2024-09-21T02:15:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.796+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f839002e810 con 0x7f8390103620 2024-09-21T02:15:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.798+0000 7f838d7fa640 1 -- 172.21.15.92:0/348811468 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22797 (secure 0 0 0) 0x7f8378009de0 con 0x7f8390103620 2024-09-21T02:15:42.800 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:15:42.800 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:03:12.414418+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f837c0b6510 msgr2=0x7f837c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f837c0b6510 0x7f837c0b8a20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f838000bbe0 tx=0x7f838000b390 comp rx=0 tx=0).stop 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 msgr2=0x7f839019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 0x7f839019ebe0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f837800d950 tx=0x7f837800de10 comp rx=0 tx=0).stop 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 shutdown_connections 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f837c0b6510 0x7f837c0b8a20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8390103fc0 0x7f83901a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8390103620 0x7f839019ebe0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 --2- 172.21.15.92:0/348811468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83901025a0 0x7f839006c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 >> 172.21.15.92:0/348811468 conn(0x7f83900fdee0 msgr2=0x7f8390106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 shutdown_connections 2024-09-21T02:15:42.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.800+0000 7f839647e640 1 -- 172.21.15.92:0/348811468 wait complete. 2024-09-21T02:15:42.814 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:57:59.243065+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:02:27.085163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:02:29.579582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:15:42.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 -- 172.21.15.92:0/3615805384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104560 msgr2=0x7f66a0104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 --2- 172.21.15.92:0/3615805384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104560 0x7f66a0104970 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6688009900 tx=0x7f668802a410 comp rx=0 tx=0).stop 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 -- 172.21.15.92:0/3615805384 shutdown_connections 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 --2- 172.21.15.92:0/3615805384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104f00 0x7f66a010b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 --2- 172.21.15.92:0/3615805384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104560 0x7f66a0104970 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 --2- 172.21.15.92:0/3615805384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 0x7f66a0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 -- 172.21.15.92:0/3615805384 >> 172.21.15.92:0/3615805384 conn(0x7f66a00fee20 msgr2=0x7f66a01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:42.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 -- 172.21.15.92:0/3615805384 shutdown_connections 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 -- 172.21.15.92:0/3615805384 wait complete. 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.926+0000 7f66a53ff640 1 Processor -- start 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 -- start start 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 0x7f66a0199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104560 0x7f66a019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 0x7f66a019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010f100 con 0x7f66a0104f00 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010ef80 con 0x7f66a0104560 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f66a53ff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010f280 con 0x7f66a01034e0 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 0x7f66a019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 0x7f66a019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34570/0 (socket says 172.21.15.92:34570) 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 -- 172.21.15.92:0/1418724758 learned_addr learned my addr 172.21.15.92:0/1418724758 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669e7fc640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104560 0x7f66a019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669effd640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 0x7f66a0199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 -- 172.21.15.92:0/1418724758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 msgr2=0x7f66a0199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 0x7f66a0199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 -- 172.21.15.92:0/1418724758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104560 msgr2=0x7f66a019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104560 0x7f66a019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 -- 172.21.15.92:0/1418724758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a019b7e0 con 0x7f66a0104f00 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669effd640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 0x7f66a0199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669e7fc640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104560 0x7f66a019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.927+0000 7f669f7fe640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 0x7f66a019afc0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f669800eaa0 tx=0x7f669800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.928+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669800cdb0 con 0x7f66a0104f00 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.928+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6698005020 con 0x7f66a0104f00 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.928+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a019bad0 con 0x7f66a0104f00 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.928+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980116d0 con 0x7f66a0104f00 2024-09-21T02:15:42.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.928+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a21c0 con 0x7f66a0104f00 2024-09-21T02:15:42.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.929+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a019bd30 con 0x7f66a0104f00 2024-09-21T02:15:42.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.931+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f66980051d0 con 0x7f66a0104f00 2024-09-21T02:15:42.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.931+0000 7f6697fff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f668c0b6510 0x7f668c0b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:42.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.932+0000 7f669effd640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f668c0b6510 0x7f668c0b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:42.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.932+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f66980e7970 con 0x7f66a0104f00 2024-09-21T02:15:42.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.932+0000 7f669effd640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f668c0b6510 0x7f668c0b8a20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6690009b40 tx=0x7f6690005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:42.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:42.934+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6698016030 con 0x7f66a0104f00 2024-09-21T02:15:43.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:42 smithi136 ceph-mon[34754]: pgmap v682: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:15:43.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:42 smithi092 ceph-mon[31190]: pgmap v682: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:15:43.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:42 smithi099 ceph-mon[34404]: pgmap v682: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-21T02:15:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.116+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66a0063fc0 con 0x7f66a0104f00 2024-09-21T02:15:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.117+0000 7f6697fff640 1 -- 172.21.15.92:0/1418724758 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f66980ae1e0 con 0x7f66a0104f00 2024-09-21T02:15:43.119 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f668c0b6510 msgr2=0x7f668c0b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f668c0b6510 0x7f668c0b8a20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6690009b40 tx=0x7f6690005e50 comp rx=0 tx=0).stop 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 msgr2=0x7f66a019afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 0x7f66a019afc0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f669800eaa0 tx=0x7f669800ef60 comp rx=0 tx=0).stop 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 shutdown_connections 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f668c0b6510 0x7f668c0b8a20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a0104f00 0x7f66a019afc0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66a0104560 0x7f66a019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 --2- 172.21.15.92:0/1418724758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66a01034e0 0x7f66a0199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 >> 172.21.15.92:0/1418724758 conn(0x7f66a00fee20 msgr2=0x7f66a0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 shutdown_connections 2024-09-21T02:15:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.119+0000 7f66a53ff640 1 -- 172.21.15.92:0/1418724758 wait complete. 2024-09-21T02:15:43.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 -- 172.21.15.92:0/3303396841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 msgr2=0x7f4e80103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 --2- 172.21.15.92:0/3303396841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80103a30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009900 tx=0x7f4e7402a410 comp rx=0 tx=0).stop 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 -- 172.21.15.92:0/3303396841 shutdown_connections 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 --2- 172.21.15.92:0/3303396841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80103fc0 0x7f4e8010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 --2- 172.21.15.92:0/3303396841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80103a30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 --2- 172.21.15.92:0/3303396841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e801025a0 0x7f4e80102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 -- 172.21.15.92:0/3303396841 >> 172.21.15.92:0/3303396841 conn(0x7f4e800fdee0 msgr2=0x7f4e80100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 -- 172.21.15.92:0/3303396841 shutdown_connections 2024-09-21T02:15:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 -- 172.21.15.92:0/3303396841 wait complete. 2024-09-21T02:15:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 Processor -- start 2024-09-21T02:15:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.243+0000 7f4e868d3640 1 -- start start 2024-09-21T02:15:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e868d3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e801025a0 0x7f4e80198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e868d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e868d3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80103fc0 0x7f4e8019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e868d3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010e1e0 con 0x7f4e80103620 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e868d3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010e060 con 0x7f4e801025a0 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e868d3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010e360 con 0x7f4e80103fc0 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46012/0 (socket says 172.21.15.92:46012) 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/2346389373 learned_addr learned my addr 172.21.15.92:0/2346389373 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e84e70640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80103fc0 0x7f4e8019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7ffff640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e801025a0 0x7f4e80198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/2346389373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80103fc0 msgr2=0x7f4e8019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80103fc0 0x7f4e8019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/2346389373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e801025a0 msgr2=0x7f4e80198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e801025a0 0x7f4e80198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 -- 172.21.15.92:0/2346389373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8019a8d0 con 0x7f4e80103620 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.244+0000 7f4e7f7fe640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80199500 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006010 tx=0x7f4e740043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.245+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7403f030 con 0x7f4e80103620 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.245+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e74003cb0 con 0x7f4e80103620 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.245+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8019ab60 con 0x7f4e80103620 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.245+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74035070 con 0x7f4e80103620 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.245+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8019ae70 con 0x7f4e80103620 2024-09-21T02:15:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.246+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e80109770 con 0x7f4e80103620 2024-09-21T02:15:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.247+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4e74042030 con 0x7f4e80103620 2024-09-21T02:15:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.248+0000 7f4e7d7fa640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6510 0x7f4e680b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.248+0000 7f4e7ffff640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6510 0x7f4e680b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.248+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4e741080c0 con 0x7f4e80103620 2024-09-21T02:15:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.248+0000 7f4e7ffff640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6510 0x7f4e680b8a20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4e700058c0 tx=0x7f4e7000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:43.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.252+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e740ce930 con 0x7f4e80103620 2024-09-21T02:15:43.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.429+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e80063fc0 con 0x7f4e80103620 2024-09-21T02:15:43.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.429+0000 7f4e7d7fa640 1 -- 172.21.15.92:0/2346389373 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f4e740d2d10 con 0x7f4e80103620 2024-09-21T02:15:43.431 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-09-21T02:15:43.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6510 msgr2=0x7f4e680b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6510 0x7f4e680b8a20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4e700058c0 tx=0x7f4e7000a4c0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 msgr2=0x7f4e80199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80199500 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006010 tx=0x7f4e740043b0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.433 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 shutdown_connections 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f4e680b6510 0x7f4e680b8a20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e80103fc0 0x7f4e8019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e80103620 0x7f4e80199500 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 --2- 172.21.15.92:0/2346389373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e801025a0 0x7f4e80198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 >> 172.21.15.92:0/2346389373 conn(0x7f4e800fdee0 msgr2=0x7f4e80106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 shutdown_connections 2024-09-21T02:15:43.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.431+0000 7f4e868d3640 1 -- 172.21.15.92:0/2346389373 wait complete. 2024-09-21T02:15:43.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:15:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 -- 172.21.15.92:0/1980293692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984102c50 msgr2=0x7fa984105070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 --2- 172.21.15.92:0/1980293692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984102c50 0x7fa984105070 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa974009900 tx=0x7fa97402a410 comp rx=0 tx=0).stop 2024-09-21T02:15:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 -- 172.21.15.92:0/1980293692 shutdown_connections 2024-09-21T02:15:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 --2- 172.21.15.92:0/1980293692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9841082d0 0x7fa98410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 --2- 172.21.15.92:0/1980293692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9841056b0 0x7fa984107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 --2- 172.21.15.92:0/1980293692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa984102c50 0x7fa984105070 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 -- 172.21.15.92:0/1980293692 >> 172.21.15.92:0/1980293692 conn(0x7fa9840fd660 msgr2=0x7fa9840ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 -- 172.21.15.92:0/1980293692 shutdown_connections 2024-09-21T02:15:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.555+0000 7fa988c69640 1 -- 172.21.15.92:0/1980293692 wait complete. 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 Processor -- start 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 -- start start 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa984102c50 0x7fa984199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9841056b0 0x7fa98419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 0x7fa98419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa98410f300 con 0x7fa9841082d0 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa98410f180 con 0x7fa984102c50 2024-09-21T02:15:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa988c69640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa98410f480 con 0x7fa9841056b0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa982d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 0x7fa98419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.556+0000 7fa982d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 0x7fa98419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46030/0 (socket says 172.21.15.92:46030) 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 -- 172.21.15.92:0/1301326522 learned_addr learned my addr 172.21.15.92:0/1301326522 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa981d9b640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9841056b0 0x7fa98419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa98259c640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa984102c50 0x7fa984199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 -- 172.21.15.92:0/1301326522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9841056b0 msgr2=0x7fa98419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9841056b0 0x7fa98419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 -- 172.21.15.92:0/1301326522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa984102c50 msgr2=0x7fa984199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa984102c50 0x7fa984199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 -- 172.21.15.92:0/1301326522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98419b910 con 0x7fa9841082d0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa982d9d640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 0x7fa98419b180 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00cce0 tx=0x7fa97c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00f450 con 0x7fa9841082d0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa97c00faa0 con 0x7fa9841082d0 2024-09-21T02:15:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98419bc00 con 0x7fa9841082d0 2024-09-21T02:15:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c018b70 con 0x7fa9841082d0 2024-09-21T02:15:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.557+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9841a21a0 con 0x7fa9841082d0 2024-09-21T02:15:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.558+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9840ff180 con 0x7fa9841082d0 2024-09-21T02:15:43.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.560+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fa97c018d10 con 0x7fa9841082d0 2024-09-21T02:15:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.560+0000 7fa97b7fe640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9700b6510 0x7fa9700b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.561+0000 7fa98259c640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9700b6510 0x7fa9700b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.561+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa97c0e78c0 con 0x7fa9841082d0 2024-09-21T02:15:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.561+0000 7fa98259c640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9700b6510 0x7fa9700b8a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa974031440 tx=0x7fa9740314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:43.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.564+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa97c014030 con 0x7fa9841082d0 2024-09-21T02:15:43.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.743+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa984063fc0 con 0x7fa9841082d0 2024-09-21T02:15:43.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.743+0000 7fa97b7fe640 1 -- 172.21.15.92:0/1301326522 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7fa97c0ae1c0 con 0x7fa9841082d0 2024-09-21T02:15:43.745 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9700b6510 msgr2=0x7fa9700b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9700b6510 0x7fa9700b8a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa974031440 tx=0x7fa9740314a0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 msgr2=0x7fa98419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 0x7fa98419b180 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00cce0 tx=0x7fa97c00a620 comp rx=0 tx=0).stop 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 shutdown_connections 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fa9700b6510 0x7fa9700b8a20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9841082d0 0x7fa98419b180 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9841056b0 0x7fa98419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 --2- 172.21.15.92:0/1301326522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa984102c50 0x7fa984199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 >> 172.21.15.92:0/1301326522 conn(0x7fa9840fd660 msgr2=0x7fa984103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 shutdown_connections 2024-09-21T02:15:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.746+0000 7fa988c69640 1 -- 172.21.15.92:0/1301326522 wait complete. 2024-09-21T02:15:43.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T02:15:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 -- 172.21.15.92:0/2766715769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48103680 msgr2=0x7f5b48103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/2766715769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48103680 0x7f5b48103af0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009900 tx=0x7f5b3802a410 comp rx=0 tx=0).stop 2024-09-21T02:15:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 -- 172.21.15.92:0/2766715769 shutdown_connections 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/2766715769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b481050a0 0x7f5b4810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/2766715769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b48104700 0x7f5b48104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/2766715769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b48103680 0x7f5b48103af0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 -- 172.21.15.92:0/2766715769 >> 172.21.15.92:0/2766715769 conn(0x7f5b480fefe0 msgr2=0x7f5b48101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 -- 172.21.15.92:0/2766715769 shutdown_connections 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.866+0000 7f5b4e65c640 1 -- 172.21.15.92:0/2766715769 wait complete. 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 Processor -- start 2024-09-21T02:15:43.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 -- start start 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b48103680 0x7f5b4819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b48104700 0x7f5b4819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 0x7f5b4819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810f3a0 con 0x7f5b481050a0 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810f220 con 0x7f5b48104700 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4e65c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810f520 con 0x7f5b48103680 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4cbf9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 0x7f5b4819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4cbf9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 0x7f5b4819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46044/0 (socket says 172.21.15.92:46044) 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.867+0000 7f5b4cbf9640 1 -- 172.21.15.92:0/26803920 learned_addr learned my addr 172.21.15.92:0/26803920 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:15:43.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4cbf9640 1 -- 172.21.15.92:0/26803920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b48103680 msgr2=0x7f5b4819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b47fff640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b48103680 0x7f5b4819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4cbf9640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b48103680 0x7f5b4819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4cbf9640 1 -- 172.21.15.92:0/26803920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b48104700 msgr2=0x7f5b4819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4cbf9640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b48104700 0x7f5b4819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4cbf9640 1 -- 172.21.15.92:0/26803920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4819ba90 con 0x7f5b481050a0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4cbf9640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 0x7f5b4819b270 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b970 tx=0x7f5b4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40019070 con 0x7f5b481050a0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4819bd80 con 0x7f5b481050a0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b40014440 con 0x7f5b481050a0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400075d0 con 0x7f5b481050a0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.868+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a2470 con 0x7f5b481050a0 2024-09-21T02:15:43.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.869+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4810a850 con 0x7f5b481050a0 2024-09-21T02:15:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.871+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f5b4001c070 con 0x7f5b481050a0 2024-09-21T02:15:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.871+0000 7f5b457fa640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5b300b6510 0x7f5b300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.871+0000 7f5b47fff640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5b300b6510 0x7f5b300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:43.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.871+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f5b400e6d20 con 0x7f5b481050a0 2024-09-21T02:15:43.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.872+0000 7f5b47fff640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5b300b6510 0x7f5b300b8a20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5b3802a930 tx=0x7f5b38005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:43.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:43.874+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b400ad590 con 0x7f5b481050a0 2024-09-21T02:15:44.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:43 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/348811468' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:15:44.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:43 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1418724758' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:15:44.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:43 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2346389373' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:15:44.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:43 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/348811468' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:15:44.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:43 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1418724758' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:15:44.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:43 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2346389373' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:15:44.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:43 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/348811468' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:15:44.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:43 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1418724758' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:15:44.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:43 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2346389373' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:15:44.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.104+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5b48063fc0 con 0x7f5b481050a0 2024-09-21T02:15:44.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.112+0000 7f5b457fa640 1 -- 172.21.15.92:0/26803920 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+833520 (secure 0 0 0) 0x7f5b400b1970 con 0x7f5b481050a0 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 20, 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "active_gid": 14150, 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "active_name": "x", 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "active_addrs": { 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1870152537 2024-09-21T02:15:44.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1870152537 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "active_addr": "172.21.15.92:6801/1870152537", 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "active_change": "2024-09-21T01:55:10.932523+0000", 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "available": true, 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "standbys": [ 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "gid": 24127, 2024-09-21T02:15:44.124 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "z", 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_features": 4540719139924082687, 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.125 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.127 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.130 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.132 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.134 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.135 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.136 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.137 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.139 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.140 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.141 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.143 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.147 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.149 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.150 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.151 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.152 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-09-21T02:15:44.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-21T02:15:44.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-09-21T02:15:44.155 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.156 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.159 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.162 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.163 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.164 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.165 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.167 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.168 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:15:44.169 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:15:44.170 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.171 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.172 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.175 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.176 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:15:44.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.178 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.179 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nginx": { 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nginx", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.180 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nginx container image", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.181 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.184 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba": { 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba_metrics": { 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.187 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.188 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.189 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.190 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-09-21T02:15:44.191 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.192 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.193 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:15:44.194 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "grafana_dashboards_path": { 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.198 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.199 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.200 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.203 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.205 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.206 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.207 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.208 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.209 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-09-21T02:15:44.210 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-09-21T02:15:44.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.212 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.213 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.214 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.216 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.217 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.219 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_keepalive_interval": { 2024-09-21T02:15:44.221 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-09-21T02:15:44.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.223 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.225 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.226 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.227 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.228 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.230 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.237 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.238 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:15:44.241 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.242 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.244 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.246 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.247 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.249 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.250 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:44.251 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.252 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.253 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.254 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "[]", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.256 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "{}", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.258 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.259 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.260 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.261 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.262 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.263 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.265 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.270 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:15:44.272 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.273 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.274 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.275 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.277 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.278 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.283 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.284 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.287 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.288 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.289 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:44.291 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.292 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-21T02:15:44.293 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-09-21T02:15:44.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "sso_oauth2": { 2024-09-21T02:15:44.295 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sso_oauth2", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-21T02:15:44.296 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.299 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.300 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.301 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.302 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.303 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.304 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-09-21T02:15:44.307 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-09-21T02:15:44.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.310 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.311 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.312 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.313 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.316 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.317 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.318 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.319 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.324 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.326 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.327 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.328 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.330 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.331 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.332 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.334 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.335 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.336 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.337 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.341 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.343 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.345 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.347 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-21T02:15:44.349 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-09-21T02:15:44.350 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.351 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.352 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.353 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.354 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.355 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.356 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-09-21T02:15:44.357 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.358 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.359 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.360 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.361 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.362 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.364 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.369 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.370 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.371 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.372 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.375 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.376 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.377 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.378 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.379 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.380 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.381 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.382 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.383 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.384 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.385 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.386 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.387 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:15:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.390 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.392 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.393 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-21T02:15:44.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.397 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.403 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.404 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.406 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.408 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.409 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.410 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.411 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.412 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.414 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.417 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.419 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.420 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.421 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.422 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.423 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-09-21T02:15:44.424 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.433 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.434 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.438 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-09-21T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.441 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.442 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.444 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.445 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.446 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.447 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.448 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.449 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.450 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.451 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.454 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.455 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.456 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.457 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.458 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.459 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.460 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.460 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.460 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.461 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.461 INFO:teuthology.orchestra.run.smithi092.stdout: "max_requests": { 2024-09-21T02:15:44.461 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_requests", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:44.462 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:44.463 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.466 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.467 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:15:44.471 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.472 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.476 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.477 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.478 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.479 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.480 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.481 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.482 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.483 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.485 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.486 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.487 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.488 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.490 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.491 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-09-21T02:15:44.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.493 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.494 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.495 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smb", 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.497 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "internal_store_backend": { 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "internal_store_backend", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.498 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.499 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.501 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.503 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "update_orchestration": { 2024-09-21T02:15:44.504 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "update_orchestration", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.506 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.507 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.508 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.511 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.513 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.514 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.515 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.516 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.517 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.518 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.519 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.520 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.521 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.522 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.523 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.524 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.527 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.528 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.529 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.530 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.532 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.533 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.534 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.535 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.536 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.537 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-09-21T02:15:44.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.539 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.540 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.544 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.545 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.549 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.550 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.551 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.552 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.553 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.554 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.555 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.556 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.557 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.558 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.560 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.563 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.564 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.567 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.568 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.570 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.571 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.572 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.573 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.574 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.575 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.576 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.577 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.578 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.580 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.581 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.585 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "gid": 24140, 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "y", 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_features": 4540719139924082687, 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.587 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.588 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.589 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.590 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.591 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.594 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.595 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-09-21T02:15:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-09-21T02:15:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-21T02:15:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-09-21T02:15:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-21T02:15:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:15:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-09-21T02:15:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-09-21T02:15:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-09-21T02:15:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-09-21T02:15:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-09-21T02:15:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:15:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-09-21T02:15:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-09-21T02:15:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-09-21T02:15:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:15:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:15:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nginx": { 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nginx", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nginx container image", 2024-09-21T02:15:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba": { 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba_metrics": { 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-09-21T02:15:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-09-21T02:15:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "grafana_dashboards_path": { 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-09-21T02:15:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-09-21T02:15:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-09-21T02:15:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:15:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:15:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:15:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-09-21T02:15:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-09-21T02:15:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-09-21T02:15:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-09-21T02:15:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-09-21T02:15:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-09-21T02:15:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:15:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-09-21T02:15:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:15:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_keepalive_interval": { 2024-09-21T02:15:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-09-21T02:15:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-09-21T02:15:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.706 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-09-21T02:15:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:15:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:15:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:15:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-09-21T02:15:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-09-21T02:15:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:15:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:15:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:15:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.739 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.740 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:15:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:15:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:15:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:15:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "[]", 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:15:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "{}", 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:15:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:15:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-09-21T02:15:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:15:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:15:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:15:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:15:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-09-21T02:15:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "sso_oauth2": { 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sso_oauth2", 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-09-21T02:15:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-09-21T02:15:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-09-21T02:15:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-21T02:15:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-09-21T02:15:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-09-21T02:15:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-09-21T02:15:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-09-21T02:15:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-09-21T02:15:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-09-21T02:15:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-21T02:15:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-09-21T02:15:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-09-21T02:15:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-21T02:15:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "max_requests": { 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_requests", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:15:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smb", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "internal_store_backend": { 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "internal_store_backend", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "update_orchestration": { 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "update_orchestration", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-09-21T02:15:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-09-21T02:15:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-09-21T02:15:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-09-21T02:15:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-09-21T02:15:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "modules": [ 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "dashboard", 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "iostat", 2024-09-21T02:15:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "nfs", 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "restful" 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-09-21T02:15:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-09-21T02:15:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-09-21T02:15:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-09-21T02:15:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-09-21T02:15:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-21T02:15:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:15:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:15:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-09-21T02:15:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-09-21T02:15:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-09-21T02:15:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:15:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:15:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nginx": { 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nginx", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nginx container image", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:15:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-09-21T02:15:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba": { 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba", 2024-09-21T02:15:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_samba_metrics": { 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:15:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:15:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "grafana_dashboards_path": { 2024-09-21T02:15:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-09-21T02:15:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-09-21T02:15:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:15:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_keepalive_interval": { 2024-09-21T02:15:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-09-21T02:15:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:15:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:15:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "[]", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "{}", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:15:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:15:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:15:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:15:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:15:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:15:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-09-21T02:15:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "sso_oauth2": { 2024-09-21T02:15:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sso_oauth2", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-09-21T02:15:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:15:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-09-21T02:15:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-09-21T02:15:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-09-21T02:15:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-09-21T02:15:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:15:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:15:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-09-21T02:15:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:15:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-09-21T02:15:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-09-21T02:15:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-09-21T02:15:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "max_requests": { 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_requests", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:15:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-09-21T02:15:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-09-21T02:15:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-09-21T02:15:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-09-21T02:15:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smb", 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "internal_store_backend": { 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "internal_store_backend", 2024-09-21T02:15:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "update_orchestration": { 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "update_orchestration", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-09-21T02:15:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-09-21T02:15:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "sqlite3_killpoint": { 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:15:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "dev", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-09-21T02:15:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "services": { 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "dashboard": "https://172.21.15.92:8443/" 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "always_on_modules": { 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "octopus": [ 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-21T02:15:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "pacific": [ 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-21T02:15:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "quincy": [ 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-21T02:15:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "reef": [ 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-21T02:15:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "squid": [ 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "last_failure_osd_epoch": 3, 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "active_clients": [ 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-09-21T02:15:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3402643159 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "libcephsqlite", 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 330833430 2024-09-21T02:15:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 910473048 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-21T02:15:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 966327342 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-21T02:15:45.586 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.117+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5b300b6510 msgr2=0x7f5b300b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.117+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5b300b6510 0x7f5b300b8a20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5b3802a930 tx=0x7f5b38005fd0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.117+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 msgr2=0x7f5b4819b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.117+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 0x7f5b4819b270 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b970 tx=0x7f5b4000be30 comp rx=0 tx=0).stop 2024-09-21T02:15:45.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.117+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 shutdown_connections 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.117+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f5b300b6510 0x7f5b300b8a20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.118+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b481050a0 0x7f5b4819b270 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.118+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b48104700 0x7f5b4819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.118+0000 7f5b4e65c640 1 --2- 172.21.15.92:0/26803920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b48103680 0x7f5b4819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.118+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 >> 172.21.15.92:0/26803920 conn(0x7f5b480fefe0 msgr2=0x7f5b48107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.118+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 shutdown_connections 2024-09-21T02:15:45.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:15:44.118+0000 7f5b4e65c640 1 -- 172.21.15.92:0/26803920 wait complete. 2024-09-21T02:15:45.626 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T02:15:45.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T02:15:45.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:44 smithi092 ceph-mon[31190]: pgmap v683: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:15:45.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1301326522' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:15:45.627 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:44 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/26803920' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T02:15:45.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:44 smithi099 ceph-mon[34404]: pgmap v683: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:15:45.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1301326522' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:15:45.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:44 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/26803920' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T02:15:45.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:44 smithi136 ceph-mon[34754]: pgmap v683: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-21T02:15:45.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1301326522' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:15:45.628 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:44 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/26803920' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T02:15:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:46.444 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1535, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1059, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1190711, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 439297, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.134624419, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.065258024, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.080042440, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.005167437, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1058, 2024-09-21T02:15:46.445 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.049941582, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000047203 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1959, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000407363, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000207 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 885248, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 139808 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 28, 2024-09-21T02:15:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 195, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 6989, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 393083, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.031126598, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.020000148, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.009138423, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000181151, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 193, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.025877150, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000134078 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:46.447 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000007016, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000200 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 4480, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 383296 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 58, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 58, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 13302, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 429715, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 40, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-21T02:15:46.448 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.019709661, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.001986429, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.015999506, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000537723, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 57, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.024460168, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000429125 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 34, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000004825, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000141 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.449 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 51, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001414222, 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000027729 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000022790, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000011395 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 3, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 2, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-21T02:15:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001584109, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000792054 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 1, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 11, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 14, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 14, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 14, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 2, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 2, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 2, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 3, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 1, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 1, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-09-21T02:15:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 5, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 298, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:46.454 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 2, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 3, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 2, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 1 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-21T02:15:46.455 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 2, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:46.456 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_write_failed": 0 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 5, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 5, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 2, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-21T02:15:46.458 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4195732, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.018683271, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004670817 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 17, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 17, 2024-09-21T02:15:46.459 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 0, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 15, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 16, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 1, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 14, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 14, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 0, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 2, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 2, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 117880, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 324816896 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 4, 2024-09-21T02:15:46.460 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 3, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 13, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001396762, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001396762 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.461 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:46.462 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000187347, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000187347 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:46.463 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.466 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.467 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.468 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.469 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.470 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-21T02:15:46.471 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.472 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 1, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 1, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 683, 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:46.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:46.474 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-21T02:15:46.476 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 254871, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 85, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:46.477 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 158, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 68913, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 320, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:46.478 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 41, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 12105, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 41, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 41, 2024-09-21T02:15:46.479 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.212993717, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005194968 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 41, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 65 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 41, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 0, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 41, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-21T02:15:46.480 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 4, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 6, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 6, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:46.481 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:46.482 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-21T02:15:46.483 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 77, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 2, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 9, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 9, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-21T02:15:46.484 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 18, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 1 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-21T02:15:46.485 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:46.486 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 1617, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1048315, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 1617, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1617, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1048315, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 41, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 12105, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 41, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 12105, 2024-09-21T02:15:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:46.489 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 41, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 41, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 41, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 41, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-21T02:15:46.491 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x56382b059520": { 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 5, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1428, 2024-09-21T02:15:46.492 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 5, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 4, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1428, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:46.493 INFO:teuthology.orchestra.run.smithi092.stdout: "trackedop": { 2024-09-21T02:15:46.494 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-09-21T02:15:46.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:46.494 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: pgmap v684: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.026 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:46 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: pgmap v684: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:46 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: pgmap v684: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-21T02:15:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:15:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:46 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:47.251 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T02:15:47.251 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T02:15:47.252 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T02:15:47.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T02:15:47.512 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:47.886 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:15:47.886 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:15:47.886 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:47 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:48.007 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:48.007 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:48.007 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 227, 2024-09-21T02:15:48.007 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 222, 2024-09-21T02:15:48.007 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 839248, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 136368, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.041141569, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.019869502, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.027647916, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 221, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.013358889, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000060447 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:48.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 839248, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 136368 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 169, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 160, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 109984, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.016394795, 2024-09-21T02:15:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.013491365, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001175231, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 167, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.009423389, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000056427 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 109984 2024-09-21T02:15:48.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 4, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 4, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2992, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 448, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.002101756, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000042936, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.002039450, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:48.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.003746144, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001248714 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-21T02:15:48.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:48.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 148945, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 29, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:48.015 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 968, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:48.016 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 228, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 807188, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 228, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 228, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 807188, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:48.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:48.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:48.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:48.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:48.018 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:48.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:15:48.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:15:48.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:47 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:48.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:15:48.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:15:48.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:47 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' 2024-09-21T02:15:48.774 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T02:15:48.774 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T02:15:48.774 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T02:15:48.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T02:15:49.025 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:49.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:48 smithi136 ceph-mon[34754]: pgmap v685: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-09-21T02:15:49.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:48 smithi092 ceph-mon[31190]: pgmap v685: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-09-21T02:15:49.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:48 smithi099 ceph-mon[34404]: pgmap v685: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 791, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 791, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 267369, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 250715, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.109753803, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.056152135, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.021510765, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.023244397, 2024-09-21T02:15:49.525 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 790, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.046080657, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000058329 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:49.526 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1489, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1654, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 380276, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 851081, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.127665970, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.057928312, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.026559988, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.034698136, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1652, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.065776872, 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000039816 2024-09-21T02:15:49.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 377840 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 236, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 229, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 869780, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 136613, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-21T02:15:49.528 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.039000087, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.019277112, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.026644301, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000071075, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 228, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.013835969, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000060684 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.529 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 868432, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 135648 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3824, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.099493930, 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000026018 2024-09-21T02:15:49.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.531 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:49.532 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 1, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-09-21T02:15:49.533 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:49.534 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:49.535 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-21T02:15:49.536 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_started": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "uninline_write_failed": 0 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-09-21T02:15:49.537 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 5, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4195732, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 4195424, 2024-09-21T02:15:49.538 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 5 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 3, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 0, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 1, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 1, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 0, 2024-09-21T02:15:49.539 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 118344, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 316420096 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-21T02:15:49.540 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.541 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.542 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.543 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.544 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-21T02:15:49.545 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.547 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.548 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.549 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.550 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1766, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:49.552 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:49.553 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:49.554 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 197480, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 55, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:49.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 158, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 8672, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 13, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-21T02:15:49.556 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 2278, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 2278, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2278, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.179965267, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000517982 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2278, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2279 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.557 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 2278, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 2278, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1483, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 793, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-09-21T02:15:49.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-21T02:15:49.559 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 77, 2024-09-21T02:15:49.560 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 2, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 6, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 6, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-21T02:15:49.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2512, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1276383, 2024-09-21T02:15:49.562 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2512, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2512, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1276383, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2278, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1433, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2278, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1433, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:49.564 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2278, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 2278, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2278, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2278, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.565 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.566 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55ecebc51450": { 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:49.567 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "trackedop": { 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:49.568 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:50.427 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T02:15:50.428 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T02:15:50.428 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T02:15:50.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T02:15:50.677 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:15:51.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:50 smithi136 ceph-mon[34754]: pgmap v686: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-09-21T02:15:51.038 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:50 smithi092 ceph-mon[31190]: pgmap v686: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-09-21T02:15:51.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:50 smithi099 ceph-mon[34404]: pgmap v686: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-09-21T02:15:51.160 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-21T02:15:51.160 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:51.160 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 0, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 0, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 0, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001139788, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000286992, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:51.161 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 170, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 160, 2024-09-21T02:15:51.162 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 110160, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.015370602, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.012431353, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001284566, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 168, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.010324603, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000061455 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:51.163 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 110160 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 233, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 226, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 842784, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 138112, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.040795908, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.019285710, 2024-09-21T02:15:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.028462685, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 226, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.015532784, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000068729 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 842784, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 138112 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.165 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:51.166 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:51.167 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 148984, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 30, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:51.168 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 968, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-09-21T02:15:51.169 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 230, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 807448, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 230, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-21T02:15:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 230, 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 807448, 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-21T02:15:51.171 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-21T02:15:51.832 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T02:15:51.832 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T02:15:51.832 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T02:15:51.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T02:15:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1130, 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 932, 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1105144, 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 298999, 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 7, 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.139737493, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.064831844, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.077486404, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.006104414, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 932, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.062326551, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000066873 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1465, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000360235, 2024-09-21T02:15:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000245 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 905616, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 140096 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 53, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 218, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 10392, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 760192, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 39, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 8, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.028878902, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.011585034, 2024-09-21T02:15:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.014402630, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000755985, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 216, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.023021691, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000106581 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 52, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000007310, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000140 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T02:15:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 385200 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 21, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 23, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 5276, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 63023, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 10, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.009263739, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.002294500, 2024-09-21T02:15:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.006116647, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000468814, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 21, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.033370654, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001589078 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000387, 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000096 2024-09-21T02:15:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 32, 2024-09-21T02:15:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001982431, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000061950 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000025686, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000012843 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 4, 2024-09-21T02:15:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 3, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.003121791, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001040597 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 1, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-21T02:15:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 12, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 12, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 11, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 2, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 8, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 6, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 2, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 5, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 298, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 3, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-21T02:15:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_started": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_write_failed": 0 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 6, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 6, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 2, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-21T02:15:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195956, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.010813152, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002162630 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-09-21T02:15:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 15, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 15, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 0, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 13, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 14, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 1, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 12, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 12, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 0, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 3, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 3, 2024-09-21T02:15:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 117664, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 324808704 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 9, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 4, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 8, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 1, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-21T02:15:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002018571, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002018571 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-21T02:15:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001103220, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000551610 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-21T02:15:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 0, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 2, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 0, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 2, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 2, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 75, 2024-09-21T02:15:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 247287, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 88, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 88, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 1, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 158, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 59712, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 237, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-21T02:15:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 43, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 12179, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 28, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 43, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 43, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.242994375, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005651031 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 43, 2024-09-21T02:15:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 67 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 43, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 43, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 5, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 7, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-21T02:15:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 6, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-21T02:15:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 77, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 2, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 9, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 9, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 17, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 2 2024-09-21T02:15:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 1200, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 995819, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 1200, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 1200, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 995819, 2024-09-21T02:15:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 43, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 12179, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 43, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 12179, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 43, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 43, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 43, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 43, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x56254ebf3380": { 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 6, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1652, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 6, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5, 2024-09-21T02:15:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1652, 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:52.668 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T02:15:52.770 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:52 smithi099 ceph-mon[34404]: pgmap v687: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-21T02:15:53.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:52 smithi136 ceph-mon[34754]: pgmap v687: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-21T02:15:53.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:52 smithi092 ceph-mon[31190]: pgmap v687: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-21T02:15:53.467 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T02:15:53.468 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T02:15:53.468 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T02:15:53.468 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T02:15:53.723 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:15:54.250 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 241, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 231, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 888178, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 138532, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.045851776, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.020852434, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.034130318, 2024-09-21T02:15:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000032062, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 230, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.016575662, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000072068 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 887344, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 138208 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 709, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 876, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 176358, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 606165, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.096851880, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.046363632, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.018462653, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.023597725, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 875, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.048503991, 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000055433 2024-09-21T02:15:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 382288 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1590, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1592, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 476619, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 505678, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-21T02:15:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.207595594, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.097162308, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.044471473, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.050194411, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1590, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.075491033, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000047478 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3810, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.115024379, 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000030190 2024-09-21T02:15:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-21T02:15:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:15:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-21T02:15:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_started": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_write_failed": 0 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 5, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-21T02:15:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195732, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 4195424, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 5 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 3, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 3, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 1, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-21T02:15:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 117784, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 316424192 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-21T02:15:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 2, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-21T02:15:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 1991, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 189400, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 51, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-21T02:15:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 158, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8672, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 13, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 2295, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 2295, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2295, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.175972607, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000512406 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2295, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2296 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 2295, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 2295, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 1494, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 799, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-09-21T02:15:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-21T02:15:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 77, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 2, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 5, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 5, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-09-21T02:15:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-21T02:15:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 2536, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1297618, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 2536, 2024-09-21T02:15:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2536, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1297618, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2295, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 1433, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2295, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1433, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2295, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 2295, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2295, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2295, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5636d12b75f0": { 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-09-21T02:15:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-21T02:15:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:54.294 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T02:15:55.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:54 smithi136 ceph-mon[34754]: pgmap v688: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-21T02:15:55.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:54 smithi092 ceph-mon[31190]: pgmap v688: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-21T02:15:55.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:54 smithi099 ceph-mon[34404]: pgmap v688: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-21T02:15:55.161 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T02:15:55.161 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T02:15:55.161 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T02:15:55.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T02:15:55.407 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1637, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1637, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4679021, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 519752, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.195324330, 2024-09-21T02:15:55.897 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.095643122, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.042045012, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.042706265, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1636, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.073014249, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000044629 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 5, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 173, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1816, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 383002, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.013829617, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.009806579, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.002039201, 2024-09-21T02:15:55.899 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000156805, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 171, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.016442589, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000096155 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 381664 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1609, 2024-09-21T02:15:55.900 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1599, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4893343, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 573071, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.177987110, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.072410114, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.066137951, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.039975905, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1598, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.063187560, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000039541 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:55.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 887232, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 137728 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5165, 2024-09-21T02:15:55.902 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.128543504, 2024-09-21T02:15:55.903 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000024887 2024-09-21T02:15:55.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.905 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:55.905 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.906 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 17, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 4, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 13, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:55.907 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 10, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 2, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-21T02:15:55.908 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 234, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:55.909 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-21T02:15:55.910 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:55.911 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_started": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_write_failed": 0 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-21T02:15:55.912 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 2420, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 52, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 11889513, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 12052994, 2024-09-21T02:15:55.913 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 12049572, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 2420 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 15, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 17, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 9, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 11, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 2, 2024-09-21T02:15:55.914 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 17, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 19, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 2, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 118544, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 317468672 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-21T02:15:55.915 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.921 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.922 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.923 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 3, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 3294, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:55.926 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:55.927 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:55.928 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 198301, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 58, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 158, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:55.929 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 57856, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 418, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 3006, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 26, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 4, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 3006, 2024-09-21T02:15:55.930 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3006, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.513964716, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000503647 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3006, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3007 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 3006, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3006, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 1579, 2024-09-21T02:15:55.931 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1424, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 2, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:55.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-21T02:15:55.933 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 77, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 2, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 6, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 6, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-21T02:15:55.934 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-21T02:15:55.935 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 3247, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 9266659, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 3247, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3247, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 9266659, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.936 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3006, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 7858700, 2024-09-21T02:15:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3006, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 7858700, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3006, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3006, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3006, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3006, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.939 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x557fccc21450": { 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:55.941 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:55.942 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T02:15:56.626 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T02:15:56.626 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T02:15:56.626 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T02:15:56.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T02:15:56.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:56 smithi099 ceph-mon[34404]: pgmap v689: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-21T02:15:56.870 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:15:57.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:56 smithi136 ceph-mon[34754]: pgmap v689: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-21T02:15:57.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:56 smithi092 ceph-mon[31190]: pgmap v689: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1038, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1028, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1155280, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 392938, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.166364528, 2024-09-21T02:15:57.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.081666690, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.057252516, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.027429104, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1027, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.061310851, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000059698 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 888160, 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 140128 2024-09-21T02:15:57.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 709, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 880, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 174980, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 607404, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.092790267, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.041529986, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.017913379, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.026053641, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 878, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.032561835, 2024-09-21T02:15:57.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000037086 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 382192 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 804, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 804, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 213689, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 254849, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-21T02:15:57.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.084576555, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.037718962, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.020137759, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.020147727, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 803, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.028774467, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000035833 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3771, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.115759695, 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000030697 2024-09-21T02:15:57.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.396 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:57.397 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-21T02:15:57.398 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:57.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:57.400 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:57.401 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_started": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "uninline_write_failed": 0 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-21T02:15:57.402 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 4, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195424, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 4195283, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.403 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 4 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 3, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 3, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 0, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 1, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 0, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 0, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 0, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 0, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-21T02:15:57.404 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 116624, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 316420096 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.406 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.412 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.413 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 0, 2024-09-21T02:15:57.415 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 1, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 1287, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:57.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:57.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 189383, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 50, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:57.418 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 158, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8672, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 13, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:57.419 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 2303, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 2303, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2303, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.240971088, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000538849 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.420 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2303, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2304 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 2303, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 2303, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 1500, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 801, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-21T02:15:57.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-21T02:15:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-21T02:15:57.423 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 77, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 2, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 5, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 5, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-09-21T02:15:57.424 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:57.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 2547, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1299266, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 2547, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2547, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1299266, 2024-09-21T02:15:57.426 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2303, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 1125, 2024-09-21T02:15:57.427 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2303, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1125, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:57.428 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2303, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 2303, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2303, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2303, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:57.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.430 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5565e5879380": { 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-21T02:15:57.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-21T02:15:57.432 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-09-21T02:15:57.432 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-21T02:15:57.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-21T02:15:57.432 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-21T02:15:58.051 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T02:15:58.051 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T02:15:58.052 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T02:15:58.052 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T02:15:58.312 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:15:58.901 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-21T02:15:58.901 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 42, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 42, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 11876, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 39089, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 12, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.020432198, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.003996971, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.011660958, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000791888, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 41, 2024-09-21T02:15:58.902 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.016922169, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000412735 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 30, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000007119, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000237 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 6000, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 3424 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 319, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 306, 2024-09-21T02:15:58.903 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 908955, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 549723, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 37, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.062221643, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.024052621, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.047479342, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000363188, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 305, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.030407113, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000099695 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 32, 2024-09-21T02:15:58.904 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000003011, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000094 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 900544, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 143008 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 437, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 696, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 96925, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 605973, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.093104860, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.051684057, 2024-09-21T02:15:58.905 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.034301854, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005212509, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 694, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.043491763, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000062668 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 948, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000270021, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000284 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:15:58.906 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 383488 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 47, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001392150, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029620 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-21T02:15:58.907 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000020933, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000010466 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 1, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 1, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000168135, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000168135 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-21T02:15:58.908 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 11, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 10, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 10, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 10, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 1, 2024-09-21T02:15:58.909 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 1, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-21T02:15:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 5, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 298, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:15:58.911 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 1, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-21T02:15:58.912 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:15:58.913 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_started": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_succeeded": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_write_failed": 0 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 5, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 5, 2024-09-21T02:15:58.914 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 2, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4195732, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.009803383, 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002450845 2024-09-21T02:15:58.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 13, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 13, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 0, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 11, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 12, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 1, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 10, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 10, 2024-09-21T02:15:58.916 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 1, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 1, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 117840, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 324812800 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 1, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 1, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 14, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.917 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000168135, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000168135 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.919 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-21T02:15:58.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.922 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.925 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-21T02:15:58.926 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-21T02:15:58.928 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 1, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 1, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 689, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:15:58.929 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:15:58.930 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 238503, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 79, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-21T02:15:58.931 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7688, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 158, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 51512, 2024-09-21T02:15:58.932 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 200, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 40, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 11730, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 40, 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-21T02:15:58.933 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 40, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.208995267, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005224881 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 40, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 62 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 40, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 0, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 40, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-09-21T02:15:58.934 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 10, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 4, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 7, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 6, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 20, 2024-09-21T02:15:58.935 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:15:58.936 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-21T02:15:58.937 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 77, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 2, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 8, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 8, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 14, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1 2024-09-21T02:15:58.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.939 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 794, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 928843, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 794, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 794, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 928843, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 40, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 11730, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 40, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 11730, 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:15:58.941 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 40, 2024-09-21T02:15:58.942 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 40, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 40, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 40, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:15:58.943 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x5563eba6b520": { 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:15:58.944 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 5, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1428, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 5, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1428, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:15:58.945 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:15:58.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:15:58.946 INFO:teuthology.orchestra.run.smithi136.stdout: "trackedop": { 2024-09-21T02:15:58.946 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-21T02:15:58.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:15:58.946 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-21T02:15:58.992 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:15:58 smithi136 ceph-mon[34754]: pgmap v690: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:15:59.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:15:58 smithi092 ceph-mon[31190]: pgmap v690: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:15:59.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:15:58 smithi099 ceph-mon[34404]: pgmap v690: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T02:15:59.651 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T02:15:59.651 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T02:15:59.651 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T02:15:59.652 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T02:15:59.917 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 328, 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 315, 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 908556, 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 547201, 2024-09-21T02:16:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 40, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.067837244, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.024099371, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.052725446, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000418506, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 315, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.031627070, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000100403 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 43, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000004941, 2024-09-21T02:16:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000114 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 900192, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 145536 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 15, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 184, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 3737, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 414110, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.015028890, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.009956386, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.003006144, 2024-09-21T02:16:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000286065, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 182, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.024332753, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000133696 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000178, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000089 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:16:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 383472 2024-09-21T02:16:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:16:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 662, 2024-09-21T02:16:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 607, 2024-09-21T02:16:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 155073, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 138104, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.095020121, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.049584682, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.042481602, 2024-09-21T02:16:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.004829075, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 605, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.047028839, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000077733 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1226, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000367415, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000299 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-21T02:16:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 48, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001644983, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000034270 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:16:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000023447, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000011723 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 2, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 1, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000217365, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000217365 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-21T02:16:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 1, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 11, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 11, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 11, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 10, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-21T02:16:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 4, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 2, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 1, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:16:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 5, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 298, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:16:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 1 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 2, 2024-09-21T02:16:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:16:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_started": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_succeeded": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_write_failed": 0 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 4, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-21T02:16:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 4, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 2, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4195424, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-21T02:16:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.006338224, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002112741 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 14, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 14, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 0, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 12, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 13, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 1, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 11, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 11, 2024-09-21T02:16:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 1, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 1, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 118272, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 324812800 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 3, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 2, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 4, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000217365, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000217365 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-21T02:16:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-21T02:16:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 0, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 1, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 0, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 1, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 1, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 85, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:16:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-21T02:16:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-21T02:16:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 238727, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 79, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-21T02:16:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7688, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 158, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 55416, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 213, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-21T02:16:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 38, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 11332, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 38, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 38, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.231994760, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006105125 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 38, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 60 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 38, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 38, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 10, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 3, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 6, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-21T02:16:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 6, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 20, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-21T02:16:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-21T02:16:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 77, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 2, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 8, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 8, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 14, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:16:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1001, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 960247, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1001, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1001, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 960247, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 38, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 11332, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 38, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 11332, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 38, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 38, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 38, 2024-09-21T02:16:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 38, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x562ecd8bd380": { 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 4, 2024-09-21T02:16:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1120, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 4, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 3, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1120, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "trackedop": { 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-21T02:16:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:00.513 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-21T02:16:01.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:00 smithi136 ceph-mon[34754]: pgmap v691: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T02:16:01.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:00 smithi092 ceph-mon[31190]: pgmap v691: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T02:16:01.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:00 smithi099 ceph-mon[34404]: pgmap v691: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-21T02:16:01.309 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T02:16:01.309 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T02:16:01.309 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T02:16:01.309 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T02:16:01.556 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:16:02.021 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-21T02:16:02.021 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:16:02.021 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 6261, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 8101, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1902639, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 10996695, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 35, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.873711337, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.411137044, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.307580221, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.139288754, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 8102, 2024-09-21T02:16:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.801731157, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000098954 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6480, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001736694, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000268 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 884400, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 1511584 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1087, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1160, 2024-09-21T02:16:02.023 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 390102, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 890270, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 11, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.142224937, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.074417122, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.052810654, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.018162599, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1161, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.243495150, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000209728 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.024 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 938, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000249506, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000265 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 390016 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2407, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2834, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1533573, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 3409303, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 209, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 9, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.360343241, 2024-09-21T02:16:02.025 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.143733145, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.217255963, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.022483376, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2828, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.562297607, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000198832 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3421, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000814855, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000238 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 956816, 2024-09-21T02:16:02.026 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 162832 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6009, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.703108506, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000283426 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:16:02.027 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 24, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000867486, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000036145 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 646, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 646, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 646, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 3.242176477, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005018849 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-21T02:16:02.028 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 347, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 25, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 5, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 20, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 17, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 1, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 1, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-09-21T02:16:02.029 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 751, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 742, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-21T02:16:02.030 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 206, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-09-21T02:16:02.031 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 206, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 133, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 15, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 118, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 73, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 1, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 639, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 629, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 599, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:16:02.032 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 723, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 687, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 2, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 622 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 8, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:16:02.033 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 3, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 3, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 15, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-21T02:16:02.034 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_started": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_succeeded": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_write_failed": 0 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 2421, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 2367, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 2367, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 54, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 50, 2024-09-21T02:16:02.035 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 53, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 52, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 52, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 1, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 1, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 1, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 11889513, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 12053593, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2405, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 5.953597774, 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002475508 2024-09-21T02:16:02.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 23, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 41, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 18, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 18, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 20, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 2, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 24, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 29, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 5, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 1, 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 16, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 15, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 123624, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 338440192 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 1867, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 1, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 646, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 20, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 2, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 5, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.003753946, 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000750789 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 15, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.003026768, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000201784 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-21T02:16:02.040 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.041 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 8, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.013842859, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001730357 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000302094, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000302094 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.004278618, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002139309 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.002299647, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001149823 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.011262758, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005631379 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 600, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 3.176107814, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005293513 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 10, 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.025061581, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002506158 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.002240392, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002240392 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:16:02.049 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 89, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 89, 2024-09-21T02:16:02.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 698, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:16:02.051 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-21T02:16:02.052 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 303156, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 124, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-21T02:16:02.053 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7688, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 158, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 98888, 2024-09-21T02:16:02.054 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 803, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 4370, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 8215981, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 37, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 4370, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4370, 2024-09-21T02:16:02.055 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 28.755346264, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006580170 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4370, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 5109 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 4370, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 238, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 4132, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 327, 2024-09-21T02:16:02.056 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 25, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 17, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 2407, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 116, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 517, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 221, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 50, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:16:02.057 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 599, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-21T02:16:02.058 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 77, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 2, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-09-21T02:16:02.059 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 818, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 12 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 2, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-21T02:16:02.060 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 2, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 3, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 3, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 9751, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2828178, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 9751, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 9751, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2828178, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:02.062 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 4370, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 8216284, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4370, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 8216284, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 4370, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 4370, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 4370, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 4370, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.064 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 3, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 303, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 3, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 303, 2024-09-21T02:16:02.065 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55e625cfd2b0": { 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 2421, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 7859289, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 2421, 2024-09-21T02:16:02.066 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2405, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 7859289, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "trackedop": { 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:02.067 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-21T02:16:02.900 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T02:16:02.901 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T02:16:02.901 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T02:16:02.901 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T02:16:03.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:02 smithi136 ceph-mon[34754]: pgmap v692: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:16:03.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:02 smithi092 ceph-mon[31190]: pgmap v692: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:16:03.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:02 smithi099 ceph-mon[34404]: pgmap v692: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T02:16:03.144 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:16:03.647 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-21T02:16:03.647 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T02:16:03.647 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2327, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2327, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 661737, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 739169, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.272323996, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.123427066, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.056023859, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.075541785, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2326, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.119062363, 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000051187 2024-09-21T02:16:03.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 3, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 176, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 716, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 386522, 2024-09-21T02:16:03.649 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.013432275, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.010185828, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.001349473, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000076776, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 174, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.012412371, 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000071335 2024-09-21T02:16:03.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 385824 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 247, 2024-09-21T02:16:03.651 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 237, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 889244, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 142366, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 3, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.046294305, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.021268305, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.035667769, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000036956, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 236, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.016451944, 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000069711 2024-09-21T02:16:03.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 888976, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 142048 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-09-21T02:16:03.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3933, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.127805995, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000032495 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.654 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-09-21T02:16:03.655 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-09-21T02:16:03.656 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-09-21T02:16:03.657 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T02:16:03.658 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-09-21T02:16:03.659 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-09-21T02:16:03.660 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_started": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_succeeded": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "uninline_write_failed": 0 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-09-21T02:16:03.661 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 6, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-09-21T02:16:03.662 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4195956, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 4195815, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 6 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 3, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 3, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 0, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 1, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 1, 2024-09-21T02:16:03.663 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 117436, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 316428288 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-09-21T02:16:03.664 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.666 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-09-21T02:16:03.667 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.669 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.672 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.673 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-09-21T02:16:03.674 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.675 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 0, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1384, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-09-21T02:16:03.676 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T02:16:03.677 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T02:16:03.678 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 181320, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 47, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-09-21T02:16:03.679 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7688, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 158, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 8672, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 13, 2024-09-21T02:16:03.680 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 2326, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 2326, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2326, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.137973974, 2024-09-21T02:16:03.681 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000489240 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2326, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2327 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 2326, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 2326, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 1514, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 810, 2024-09-21T02:16:03.682 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-09-21T02:16:03.683 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-09-21T02:16:03.684 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 77, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 2, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 4, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 4, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-09-21T02:16:03.685 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-09-21T02:16:03.686 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 2573, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1304518, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 2573, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:03.687 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2573, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1304518, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:03.688 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2326, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 1657, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2326, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1657, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-09-21T02:16:03.689 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2326, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 2326, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2326, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2326, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-09-21T02:16:03.690 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.691 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x563cd6acf380": { 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-09-21T02:16:03.692 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: "trackedop": { 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-21T02:16:03.693 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-21T02:16:04.461 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T02:16:04.461 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T02:16:04.461 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T02:16:04.471 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T02:16:04.472 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T02:16:04.472 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:04.493 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-21T02:16:04.493 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:04.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T02:16:04.847 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:04.968 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:04 smithi092 ceph-mon[31190]: pgmap v693: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-21T02:16:05.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:04 smithi136 ceph-mon[34754]: pgmap v693: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-21T02:16:05.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:04 smithi099 ceph-mon[34404]: pgmap v693: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-21T02:16:05.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 -- 172.21.15.92:0/2407777053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 msgr2=0x7f5704103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 --2- 172.21.15.92:0/2407777053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f5704103fb0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009ca0 tx=0x7f56f402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 -- 172.21.15.92:0/2407777053 shutdown_connections 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 --2- 172.21.15.92:0/2407777053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 0x7f5704110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 --2- 172.21.15.92:0/2407777053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57041094e0 0x7f57041098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 --2- 172.21.15.92:0/2407777053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f5704103fb0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 -- 172.21.15.92:0/2407777053 >> 172.21.15.92:0/2407777053 conn(0x7f57040ff630 msgr2=0x7f5704101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 -- 172.21.15.92:0/2407777053 shutdown_connections 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.327+0000 7f570b859640 1 -- 172.21.15.92:0/2407777053 wait complete. 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.328+0000 7f570b859640 1 Processor -- start 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.328+0000 7f570b859640 1 -- start start 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f570419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57041094e0 0x7f570419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 0x7f570419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:05.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5704113af0 con 0x7f5704103ba0 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5704113970 con 0x7f5704109e30 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5704113c70 con 0x7f57041094e0 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f570419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f570419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40744/0 (socket says 172.21.15.92:40744) 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 -- 172.21.15.92:0/1227915612 learned_addr learned my addr 172.21.15.92:0/1227915612 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f5708df4640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57041094e0 0x7f570419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f5709df6640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 0x7f570419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 -- 172.21.15.92:0/1227915612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57041094e0 msgr2=0x7f570419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57041094e0 0x7f570419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 -- 172.21.15.92:0/1227915612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 msgr2=0x7f570419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 0x7f570419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 -- 172.21.15.92:0/1227915612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57041a0470 con 0x7f5704103ba0 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f5709df6640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 0x7f570419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:16:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57095f5640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f570419a500 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006010 tx=0x7f56f40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4003bf0 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56f4003d90 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.329+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57041a0700 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.330+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f402fc20 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.330+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57041a6ec0 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.330+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5704103fb0 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.332+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f56f40046a0 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.333+0000 7f57027fc640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56ec0b6590 0x7f56ec0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.333+0000 7f5708df4640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56ec0b6590 0x7f56ec0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.333+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f56f41087b0 con 0x7f5704103ba0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.334+0000 7f5708df4640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56ec0b6590 0x7f56ec0b8aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5704109da0 tx=0x7f56f800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.337+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56f40cf020 con 0x7f5704103ba0 2024-09-21T02:16:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.527+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f570419b630 con 0x7f5704103ba0 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.528+0000 7f57027fc640 1 -- 172.21.15.92:0/1227915612 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v77) ==== 81+0+591 (secure 0 0 0) 0x7f570419b630 con 0x7f5704103ba0 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/4144615167 2024-09-22T01:55:10.932400+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2108554208 2024-09-22T01:55:10.932400+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/207017941 2024-09-22T01:55:10.932400+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3151574869 2024-09-22T01:55:10.932400+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3151574869 2024-09-22T01:55:10.932400+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/3466917826 2024-09-22T01:54:45.168174+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/3466917826 2024-09-22T01:54:45.168174+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1290991696 2024-09-22T01:54:45.168174+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2823790923 2024-09-22T01:54:45.168174+0000 2024-09-21T02:16:05.529 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2207814153 2024-09-22T01:54:45.168174+0000 2024-09-21T02:16:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:listed 10 entries 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56ec0b6590 msgr2=0x7f56ec0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56ec0b6590 0x7f56ec0b8aa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5704109da0 tx=0x7f56f800a250 comp rx=0 tx=0).stop 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 msgr2=0x7f570419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f570419a500 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006010 tx=0x7f56f40319f0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 shutdown_connections 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f56ec0b6590 0x7f56ec0b8aa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5704109e30 0x7f570419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57041094e0 0x7f570419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 --2- 172.21.15.92:0/1227915612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5704103ba0 0x7f570419a500 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 >> 172.21.15.92:0/1227915612 conn(0x7f57040ff630 msgr2=0x7f5704101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 shutdown_connections 2024-09-21T02:16:05.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:05.530+0000 7f570b859640 1 -- 172.21.15.92:0/1227915612 wait complete. 2024-09-21T02:16:05.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:05 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1227915612' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T02:16:06.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:05 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1227915612' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T02:16:06.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:05 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1227915612' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T02:16:06.287 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T02:16:06.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:06.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:06.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:16:06.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:06.335 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:16:06.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:06.369 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-21T02:16:06.424 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:16:06.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:06.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:06.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:06.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-09-21T02:16:06.588 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T02:16:06.597 INFO:tasks.cephadm:Teardown begin 2024-09-21T02:16:06.597 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:16:06.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:16:06.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:06 smithi092 ceph-mon[31190]: pgmap v694: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T02:16:06.848 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:07.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:06 smithi136 ceph-mon[34754]: pgmap v694: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T02:16:07.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:06 smithi099 ceph-mon[34404]: pgmap v694: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T02:16:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 -- 172.21.15.92:0/1414917598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 msgr2=0x7f4618110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 --2- 172.21.15.92:0/1414917598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f4618110620 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f461002a410 comp rx=0 tx=0).stop 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 -- 172.21.15.92:0/1414917598 shutdown_connections 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 --2- 172.21.15.92:0/1414917598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f4618110620 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 --2- 172.21.15.92:0/1414917598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 0x7f46181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 --2- 172.21.15.92:0/1414917598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4618103b60 0x7f4618103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 -- 172.21.15.92:0/1414917598 >> 172.21.15.92:0/1414917598 conn(0x7f461807b850 msgr2=0x7f461807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 -- 172.21.15.92:0/1414917598 shutdown_connections 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.355+0000 7f461fdbe640 1 -- 172.21.15.92:0/1414917598 wait complete. 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 Processor -- start 2024-09-21T02:16:07.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 -- start start 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4618103b60 0x7f461819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 0x7f461819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f461819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4618113ad0 con 0x7f4618109df0 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4618113950 con 0x7f46181094a0 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461fdbe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4618113c50 con 0x7f4618103b60 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461e35b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f461819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461db5a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4618103b60 0x7f461819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461e35b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f461819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40768/0 (socket says 172.21.15.92:40768) 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.356+0000 7f461e35b640 1 -- 172.21.15.92:0/2158225063 learned_addr learned my addr 172.21.15.92:0/2158225063 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461d359640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 0x7f461819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461e35b640 1 -- 172.21.15.92:0/2158225063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4618103b60 msgr2=0x7f461819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461e35b640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4618103b60 0x7f461819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461e35b640 1 -- 172.21.15.92:0/2158225063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 msgr2=0x7f461819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461e35b640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 0x7f461819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461e35b640 1 -- 172.21.15.92:0/2158225063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46181a0410 con 0x7f4618109df0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461d359640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 0x7f461819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461e35b640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f461819fce0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f4610004770 tx=0x7f4610004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461003f030 con 0x7f4618109df0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181a06a0 con 0x7f4618109df0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4610003aa0 con 0x7f4618109df0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610035070 con 0x7f4618109df0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.357+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a6e60 con 0x7f4618109df0 2024-09-21T02:16:07.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.358+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4618103f70 con 0x7f4618109df0 2024-09-21T02:16:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.361+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f4610042030 con 0x7f4618109df0 2024-09-21T02:16:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.361+0000 7f4616ffd640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46000b6590 0x7f46000b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.362+0000 7f461db5a640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46000b6590 0x7f46000b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.362+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4610107cc0 con 0x7f4618109df0 2024-09-21T02:16:07.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.362+0000 7f461db5a640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46000b6590 0x7f46000b8aa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4608005fd0 tx=0x7f4608009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:07.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.365+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46100ce530 con 0x7f4618109df0 2024-09-21T02:16:07.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.578+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f461819b5d0 con 0x7f4618109df0 2024-09-21T02:16:07.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.579+0000 7f4616ffd640 1 -- 172.21.15.92:0/2158225063 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f46100d2910 con 0x7f4618109df0 2024-09-21T02:16:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:16:07.580 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:16:07.581 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.581+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46000b6590 msgr2=0x7f46000b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.581+0000 7f461fdbe640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46000b6590 0x7f46000b8aa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4608005fd0 tx=0x7f4608009680 comp rx=0 tx=0).stop 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.581+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 msgr2=0x7f461819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.581+0000 7f461fdbe640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f461819fce0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f4610004770 tx=0x7f4610004270 comp rx=0 tx=0).stop 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 shutdown_connections 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f46000b6590 0x7f46000b8aa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4618109df0 0x7f461819fce0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46181094a0 0x7f461819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 --2- 172.21.15.92:0/2158225063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4618103b60 0x7f461819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 >> 172.21.15.92:0/2158225063 conn(0x7f461807b850 msgr2=0x7f4618105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 shutdown_connections 2024-09-21T02:16:07.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:07.582+0000 7f461fdbe640 1 -- 172.21.15.92:0/2158225063 wait complete. 2024-09-21T02:16:08.004 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:07 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2158225063' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:16:08.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:07 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2158225063' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:16:08.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:07 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2158225063' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:16:08.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:16:08.635 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:09.021 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:08 smithi092 ceph-mon[31190]: pgmap v695: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-21T02:16:09.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:08 smithi136 ceph-mon[34754]: pgmap v695: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-21T02:16:09.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:08 smithi099 ceph-mon[34404]: pgmap v695: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-21T02:16:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 -- 172.21.15.92:0/1524597475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 msgr2=0x7f290c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 --2- 172.21.15.92:0/1524597475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c102e70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2904009900 tx=0x7f290402a410 comp rx=0 tx=0).stop 2024-09-21T02:16:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 -- 172.21.15.92:0/1524597475 shutdown_connections 2024-09-21T02:16:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 --2- 172.21.15.92:0/1524597475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f290c108cf0 0x7f290c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 --2- 172.21.15.92:0/1524597475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f290c1083a0 0x7f290c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 --2- 172.21.15.92:0/1524597475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c102e70 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 -- 172.21.15.92:0/1524597475 >> 172.21.15.92:0/1524597475 conn(0x7f290c079730 msgr2=0x7f290c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 -- 172.21.15.92:0/1524597475 shutdown_connections 2024-09-21T02:16:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.109+0000 7f29115c5640 1 -- 172.21.15.92:0/1524597475 wait complete. 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.110+0000 7f29115c5640 1 Processor -- start 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 -- start start 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f290c1083a0 0x7f290c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f290c108cf0 0x7f290c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f290c113ac0 con 0x7f290c102a60 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f290c113940 con 0x7f290c1083a0 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f29115c5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f290c113c40 con 0x7f290c108cf0 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40776/0 (socket says 172.21.15.92:40776) 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 -- 172.21.15.92:0/7410055 learned_addr learned my addr 172.21.15.92:0/7410055 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290a7fc640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f290c1083a0 0x7f290c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290b7fe640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f290c108cf0 0x7f290c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 -- 172.21.15.92:0/7410055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f290c108cf0 msgr2=0x7f290c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f290c108cf0 0x7f290c19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 -- 172.21.15.92:0/7410055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f290c1083a0 msgr2=0x7f290c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f290c1083a0 0x7f290c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 -- 172.21.15.92:0/7410055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c1a0440 con 0x7f290c102a60 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.111+0000 7f290affd640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c19a4d0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2904006010 tx=0x7f29040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.112+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290403f030 con 0x7f290c102a60 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.112+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2904003800 con 0x7f290c102a60 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.112+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290c1a06d0 con 0x7f290c102a60 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.112+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904035070 con 0x7f290c102a60 2024-09-21T02:16:09.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.112+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290c1a6e90 con 0x7f290c102a60 2024-09-21T02:16:09.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.113+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290c102e70 con 0x7f290c102a60 2024-09-21T02:16:09.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.114+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f2904003cb0 con 0x7f290c102a60 2024-09-21T02:16:09.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.115+0000 7f28f3fff640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28f40b6510 0x7f28f40b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:09.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.115+0000 7f290a7fc640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28f40b6510 0x7f28f40b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:09.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.115+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2904107fa0 con 0x7f290c102a60 2024-09-21T02:16:09.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.116+0000 7f290a7fc640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28f40b6510 0x7f28f40b8a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f290c100d20 tx=0x7f29000094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:09.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.118+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29040ce810 con 0x7f290c102a60 2024-09-21T02:16:09.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.333+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f290c19b600 con 0x7f290c102a60 2024-09-21T02:16:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.334+0000 7f28f3fff640 1 -- 172.21.15.92:0/7410055 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11963 (secure 0 0 0) 0x7f29040d2bf0 con 0x7f290c102a60 2024-09-21T02:16:09.335 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:16:09.335 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"btime":"2024-09-21T02:04:20:857710+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/859830568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":859830568},{"type":"v1","addr":"172.21.15.92:6835","nonce":859830568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/67515545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":67515545},{"type":"v1","addr":"172.21.15.92:6837","nonce":67515545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:02:31.545262+0000","modified":"2024-09-21T02:04:20.742614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24637,"mds_1":24623,"mds_2":24619,"mds_3":24610,"mds_4":14799},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14793":{"gid":14793,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/4068035941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":4068035941},{"type":"v1","addr":"172.21.15.92:6839","nonce":4068035941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14799":{"gid":14799,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6841/2220276047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":2220276047},{"type":"v1","addr":"172.21.15.92:6841","nonce":2220276047}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/3451484092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3451484092},{"type":"v1","addr":"172.21.15.99:6833","nonce":3451484092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/2004123702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2004123702},{"type":"v1","addr":"172.21.15.99:6835","nonce":2004123702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/2879720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2879720066},{"type":"v1","addr":"172.21.15.136:6833","nonce":2879720066}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24593":{"gid":24593,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3685475120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3685475120},{"type":"v1","addr":"172.21.15.99:6837","nonce":3685475120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24610":{"gid":24610,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.136:6835/2933531534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2933531534},{"type":"v1","addr":"172.21.15.136:6835","nonce":2933531534}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24619":{"gid":24619,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.136:6837/3734933147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3734933147},{"type":"v1","addr":"172.21.15.136:6837","nonce":3734933147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24623":{"gid":24623,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/2904178656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":2904178656},{"type":"v1","addr":"172.21.15.99:6839","nonce":2904178656}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.136:6839/419135468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":419135468},{"type":"v1","addr":"172.21.15.136:6839","nonce":419135468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24637,"qdb_cluster":[24637,24619,24623,24610,14799]},"id":1}]} 2024-09-21T02:16:09.335 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28f40b6510 msgr2=0x7f28f40b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28f40b6510 0x7f28f40b8a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f290c100d20 tx=0x7f29000094f0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 msgr2=0x7f290c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c19a4d0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2904006010 tx=0x7f29040042f0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 shutdown_connections 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f28f40b6510 0x7f28f40b8a20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f290c108cf0 0x7f290c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f290c1083a0 0x7f290c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 --2- 172.21.15.92:0/7410055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f290c102a60 0x7f290c19a4d0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.336+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 >> 172.21.15.92:0/7410055 conn(0x7f290c079730 msgr2=0x7f290c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.337+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 shutdown_connections 2024-09-21T02:16:09.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:09.337+0000 7f29115c5640 1 -- 172.21.15.92:0/7410055 wait complete. 2024-09-21T02:16:10.001 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:09 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/7410055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:16:10.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:16:10.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:09 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/7410055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:16:10.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:09 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/7410055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:16:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:10.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.749+0000 7f9154dbc640 1 -- 172.21.15.92:0/1433447779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150108cf0 msgr2=0x7f915010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:10.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.749+0000 7f9154dbc640 1 --2- 172.21.15.92:0/1433447779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150108cf0 0x7f915010f520 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9148009ca0 tx=0x7f914802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 -- 172.21.15.92:0/1433447779 shutdown_connections 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 --2- 172.21.15.92:0/1433447779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9150108cf0 0x7f915010f520 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 --2- 172.21.15.92:0/1433447779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91501083a0 0x7f91501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 --2- 172.21.15.92:0/1433447779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9150102a60 0x7f9150102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 -- 172.21.15.92:0/1433447779 >> 172.21.15.92:0/1433447779 conn(0x7f9150079730 msgr2=0x7f9150079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 -- 172.21.15.92:0/1433447779 shutdown_connections 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 -- 172.21.15.92:0/1433447779 wait complete. 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 Processor -- start 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.750+0000 7f9154dbc640 1 -- start start 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f9154dbc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9150102a60 0x7f915019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f9154dbc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 0x7f915019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f9154dbc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9150108cf0 0x7f915019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f9154dbc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91501129c0 con 0x7f91501083a0 2024-09-21T02:16:10.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f9154dbc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9150112840 con 0x7f9150108cf0 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f9154dbc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9150112b40 con 0x7f9150102a60 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 0x7f915019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 0x7f915019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40788/0 (socket says 172.21.15.92:40788) 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 -- 172.21.15.92:0/2363524563 learned_addr learned my addr 172.21.15.92:0/2363524563 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914e59c640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9150102a60 0x7f915019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914ed9d640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9150108cf0 0x7f915019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 -- 172.21.15.92:0/2363524563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9150102a60 msgr2=0x7f915019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9150102a60 0x7f915019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 -- 172.21.15.92:0/2363524563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9150108cf0 msgr2=0x7f915019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9150108cf0 0x7f915019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 -- 172.21.15.92:0/2363524563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91501a0410 con 0x7f91501083a0 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.751+0000 7f914dd9b640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 0x7f915019aa40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f914400b450 tx=0x7f914400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.752+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144015020 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.752+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91501a0700 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.752+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9144011d40 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.752+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914400f340 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.752+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9150105040 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.753+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9150102e70 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.754+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f9144018020 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.755+0000 7f913f7fe640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91380b6510 0x7f91380b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.755+0000 7f914e59c640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91380b6510 0x7f91380b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.756+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f91440e71f0 con 0x7f91501083a0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.756+0000 7f914e59c640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91380b6510 0x7f91380b8a20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f914000bbe0 tx=0x7f914000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:10.794 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.761+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f914401f020 con 0x7f91501083a0 2024-09-21T02:16:10.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.940+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f915019b5d0 con 0x7f91501083a0 2024-09-21T02:16:10.941 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:10 smithi092 ceph-mon[31190]: pgmap v696: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T02:16:10.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.942+0000 7f913f7fe640 1 -- 172.21.15.92:0/2363524563 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22797 (secure 0 0 0) 0x7f914400d040 con 0x7f91501083a0 2024-09-21T02:16:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-21T02:16:10.943 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"49513f9c-77bc-11ef-baf5-efdc52797490","created":"2024-09-21T01:54:21.538024+0000","modified":"2024-09-21T02:03:12.414418+0000","last_up_change":"2024-09-21T02:01:41.927918+0000","last_in_change":"2024-09-21T02:01:18.883272+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:57:59.243065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:02:27.085163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:02:29.579582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e256750d-9725-4215-90dc-41a4dc35dfb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6803","nonce":2450056429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6805","nonce":2450056429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6809","nonce":2450056429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2450056429},{"type":"v1","addr":"172.21.15.92:6807","nonce":2450056429}]},"public_addr":"172.21.15.92:6803/2450056429","cluster_addr":"172.21.15.92:6805/2450056429","heartbeat_back_addr":"172.21.15.92:6809/2450056429","heartbeat_front_addr":"172.21.15.92:6807/2450056429","state":["exists","up"]},{"osd":1,"uuid":"e1b3ce9c-9425-4d9c-b63f-4a3d3181f253","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6801","nonce":3284764622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6803","nonce":3284764622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6807","nonce":3284764622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3284764622},{"type":"v1","addr":"172.21.15.99:6805","nonce":3284764622}]},"public_addr":"172.21.15.99:6801/3284764622","cluster_addr":"172.21.15.99:6803/3284764622","heartbeat_back_addr":"172.21.15.99:6807/3284764622","heartbeat_front_addr":"172.21.15.99:6805/3284764622","state":["exists","up"]},{"osd":2,"uuid":"d560a67b-26ab-47d8-9a94-c9f293238171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6801","nonce":2651717363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6803","nonce":2651717363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6807","nonce":2651717363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2651717363},{"type":"v1","addr":"172.21.15.136:6805","nonce":2651717363}]},"public_addr":"172.21.15.136:6801/2651717363","cluster_addr":"172.21.15.136:6803/2651717363","heartbeat_back_addr":"172.21.15.136:6807/2651717363","heartbeat_front_addr":"172.21.15.136:6805/2651717363","state":["exists","up"]},{"osd":3,"uuid":"200ba4ba-2dd5-4dbc-a805-d0b25c4816b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6811","nonce":2531772265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6813","nonce":2531772265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6817","nonce":2531772265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2531772265},{"type":"v1","addr":"172.21.15.92:6815","nonce":2531772265}]},"public_addr":"172.21.15.92:6811/2531772265","cluster_addr":"172.21.15.92:6813/2531772265","heartbeat_back_addr":"172.21.15.92:6817/2531772265","heartbeat_front_addr":"172.21.15.92:6815/2531772265","state":["exists","up"]},{"osd":4,"uuid":"39775100-fc5d-4e58-b5d8-e81dbc327b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6809","nonce":1012266354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6811","nonce":1012266354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6815","nonce":1012266354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1012266354},{"type":"v1","addr":"172.21.15.99:6813","nonce":1012266354}]},"public_addr":"172.21.15.99:6809/1012266354","cluster_addr":"172.21.15.99:6811/1012266354","heartbeat_back_addr":"172.21.15.99:6815/1012266354","heartbeat_front_addr":"172.21.15.99:6813/1012266354","state":["exists","up"]},{"osd":5,"uuid":"fa5283a3-4b38-49cb-9edc-622f2c227ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6809","nonce":659398601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6811","nonce":659398601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6815","nonce":659398601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":659398601},{"type":"v1","addr":"172.21.15.136:6813","nonce":659398601}]},"public_addr":"172.21.15.136:6809/659398601","cluster_addr":"172.21.15.136:6811/659398601","heartbeat_back_addr":"172.21.15.136:6815/659398601","heartbeat_front_addr":"172.21.15.136:6813/659398601","state":["exists","up"]},{"osd":6,"uuid":"0d1d32dc-5414-40bf-824a-230ac53e834f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6819","nonce":1034212183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6821","nonce":1034212183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6825","nonce":1034212183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1034212183},{"type":"v1","addr":"172.21.15.92:6823","nonce":1034212183}]},"public_addr":"172.21.15.92:6819/1034212183","cluster_addr":"172.21.15.92:6821/1034212183","heartbeat_back_addr":"172.21.15.92:6825/1034212183","heartbeat_front_addr":"172.21.15.92:6823/1034212183","state":["exists","up"]},{"osd":7,"uuid":"4f02b02a-3345-49b2-aa55-bd16a66a1c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6817","nonce":1697733163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6819","nonce":1697733163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6823","nonce":1697733163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1697733163},{"type":"v1","addr":"172.21.15.99:6821","nonce":1697733163}]},"public_addr":"172.21.15.99:6817/1697733163","cluster_addr":"172.21.15.99:6819/1697733163","heartbeat_back_addr":"172.21.15.99:6823/1697733163","heartbeat_front_addr":"172.21.15.99:6821/1697733163","state":["exists","up"]},{"osd":8,"uuid":"813f34cb-ed23-4d4e-b578-0f9e906c8d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6817","nonce":3496989963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6819","nonce":3496989963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6823","nonce":3496989963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3496989963},{"type":"v1","addr":"172.21.15.136:6821","nonce":3496989963}]},"public_addr":"172.21.15.136:6817/3496989963","cluster_addr":"172.21.15.136:6819/3496989963","heartbeat_back_addr":"172.21.15.136:6823/3496989963","heartbeat_front_addr":"172.21.15.136:6821/3496989963","state":["exists","up"]},{"osd":9,"uuid":"574cf4c9-c764-453d-9661-a05a8f62b827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6827","nonce":1884194979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6829","nonce":1884194979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6833","nonce":1884194979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1884194979},{"type":"v1","addr":"172.21.15.92:6831","nonce":1884194979}]},"public_addr":"172.21.15.92:6827/1884194979","cluster_addr":"172.21.15.92:6829/1884194979","heartbeat_back_addr":"172.21.15.92:6833/1884194979","heartbeat_front_addr":"172.21.15.92:6831/1884194979","state":["exists","up"]},{"osd":10,"uuid":"769d723d-f839-4eca-8b28-0336cdb68063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6825","nonce":3821192974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6827","nonce":3821192974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6831","nonce":3821192974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3821192974},{"type":"v1","addr":"172.21.15.99:6829","nonce":3821192974}]},"public_addr":"172.21.15.99:6825/3821192974","cluster_addr":"172.21.15.99:6827/3821192974","heartbeat_back_addr":"172.21.15.99:6831/3821192974","heartbeat_front_addr":"172.21.15.99:6829/3821192974","state":["exists","up"]},{"osd":11,"uuid":"bccf2012-84f1-4ce9-a21d-9016f7d5cce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6825","nonce":2995433302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6827","nonce":2995433302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6831","nonce":2995433302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2995433302},{"type":"v1","addr":"172.21.15.136:6829","nonce":2995433302}]},"public_addr":"172.21.15.136:6825/2995433302","cluster_addr":"172.21.15.136:6827/2995433302","heartbeat_back_addr":"172.21.15.136:6831/2995433302","heartbeat_front_addr":"172.21.15.136:6829/2995433302","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:04.886740+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:30.790266+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:57:55.701088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:19.846230+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:58:45.843445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:10.589090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:59:36.161295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:00.700479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:24.988134+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:00:50.388930+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:16.232364+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:01:40.462389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/4144615167":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/2108554208":"2024-09-22T01:55:10.932400+0000","172.21.15.92:0/207017941":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6800/3151574869":"2024-09-22T01:55:10.932400+0000","172.21.15.92:6801/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:6800/3466917826":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/1290991696":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2823790923":"2024-09-22T01:54:45.168174+0000","172.21.15.92:0/2207814153":"2024-09-22T01:54:45.168174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91380b6510 msgr2=0x7f91380b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91380b6510 0x7f91380b8a20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f914000bbe0 tx=0x7f914000b390 comp rx=0 tx=0).stop 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 msgr2=0x7f915019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 0x7f915019aa40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f914400b450 tx=0x7f914400b910 comp rx=0 tx=0).stop 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 shutdown_connections 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f91380b6510 0x7f91380b8a20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9150108cf0 0x7f915019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91501083a0 0x7f915019aa40 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 --2- 172.21.15.92:0/2363524563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9150102a60 0x7f915019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 >> 172.21.15.92:0/2363524563 conn(0x7f9150079730 msgr2=0x7f915006b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 shutdown_connections 2024-09-21T02:16:10.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:10.944+0000 7f9154dbc640 1 -- 172.21.15.92:0/2363524563 wait complete. 2024-09-21T02:16:11.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:10 smithi136 ceph-mon[34754]: pgmap v696: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T02:16:11.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:10 smithi099 ceph-mon[34404]: pgmap v696: 97 pgs: 97 active+clean; 4.1 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-21T02:16:11.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T02:16:11.934 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:12.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:11 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/2363524563' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:16:12.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:11 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/2363524563' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:16:12.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:11 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/2363524563' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 -- 172.21.15.92:0/3557380804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109500 msgr2=0x7fc36c109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 --2- 172.21.15.92:0/3557380804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109500 0x7fc36c109910 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc360009900 tx=0x7fc36002a410 comp rx=0 tx=0).stop 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 -- 172.21.15.92:0/3557380804 shutdown_connections 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 --2- 172.21.15.92:0/3557380804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109e50 0x7fc36c110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 --2- 172.21.15.92:0/3557380804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109500 0x7fc36c109910 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 --2- 172.21.15.92:0/3557380804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 0x7fc36c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.434+0000 7fc371fea640 1 -- 172.21.15.92:0/3557380804 >> 172.21.15.92:0/3557380804 conn(0x7fc36c079f90 msgr2=0x7fc36c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.435+0000 7fc371fea640 1 -- 172.21.15.92:0/3557380804 shutdown_connections 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.435+0000 7fc371fea640 1 -- 172.21.15.92:0/3557380804 wait complete. 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.435+0000 7fc371fea640 1 Processor -- start 2024-09-21T02:16:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 -- start start 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 0x7fc36c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109500 0x7fc36c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 0x7fc36c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c113b10 con 0x7fc36c109e50 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c113990 con 0x7fc36c109500 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc371fea640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c113c90 con 0x7fc36c101fc0 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 0x7fc36c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 0x7fc36c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40802/0 (socket says 172.21.15.92:40802) 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 -- 172.21.15.92:0/665785178 learned_addr learned my addr 172.21.15.92:0/665785178 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36affd640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109500 0x7fc36c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36b7fe640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 0x7fc36c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 -- 172.21.15.92:0/665785178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 msgr2=0x7fc36c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 0x7fc36c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 -- 172.21.15.92:0/665785178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109500 msgr2=0x7fc36c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109500 0x7fc36c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 -- 172.21.15.92:0/665785178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c1a0470 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36b7fe640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 0x7fc36c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36affd640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109500 0x7fc36c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.436+0000 7fc36bfff640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 0x7fc36c19fd40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc36400c970 tx=0x7fc36400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.437+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364007bb0 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.437+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c1a0760 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.437+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a6ec0 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.437+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc364007d50 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.437+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36401d060 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.438+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc358005290 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.439+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7fc364004e80 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.440+0000 7fc368ff9640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3540b6510 0x7fc3540b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.440+0000 7fc36b7fe640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3540b6510 0x7fc3540b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.440+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc3640e6fa0 con 0x7fc36c109e50 2024-09-21T02:16:12.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.440+0000 7fc36b7fe640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3540b6510 0x7fc3540b8a20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c0093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.444+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc364014030 con 0x7fc36c109e50 2024-09-21T02:16:12.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.660+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc358005580 con 0x7fc36c109e50 2024-09-21T02:16:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.662+0000 7fc368ff9640 1 -- 172.21.15.92:0/665785178 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7fc3640ad810 con 0x7fc36c109e50 2024-09-21T02:16:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:16:12.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3540b6510 msgr2=0x7fc3540b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:12.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3540b6510 0x7fc3540b8a20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c0093e0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 msgr2=0x7fc36c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 0x7fc36c19fd40 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc36400c970 tx=0x7fc36400ce30 comp rx=0 tx=0).stop 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 shutdown_connections 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7fc3540b6510 0x7fc3540b8a20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c109e50 0x7fc36c19fd40 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c109500 0x7fc36c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 --2- 172.21.15.92:0/665785178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc36c101fc0 0x7fc36c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 >> 172.21.15.92:0/665785178 conn(0x7fc36c079f90 msgr2=0x7fc36c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.666+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 shutdown_connections 2024-09-21T02:16:12.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:12.667+0000 7fc371fea640 1 -- 172.21.15.92:0/665785178 wait complete. 2024-09-21T02:16:12.744 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b[62418]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:12.744 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b[62418]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:12.744 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e[61873]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e[61873]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mds[61878]: -- 172.21.15.99:0/1175656194 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cee549c20 con 0x560cf26f1600 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e[61873]: starting mds.e at 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k[61591]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k[61591]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mds[61595]: -- 172.21.15.99:0/458070281 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b92680d860 con 0x55b92a9b5180 2024-09-21T02:16:12.745 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k[61591]: starting mds.k at 2024-09-21T02:16:12.984 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a[72610]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:12.984 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a[72610]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:12.984 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mds[72614]: -- 172.21.15.92:0/3380798856 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d49097860 con 0x557d4d23fa80 2024-09-21T02:16:12.984 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a[72610]: starting mds.a at 2024-09-21T02:16:12.984 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a[31186]: 2024-09-21T02:16:12.661+0000 7ff69fe90640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: pgmap v697: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/665785178' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/665785178' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mon[31190]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g[72323]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g[72323]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-mds[72330]: -- 172.21.15.92:0/849112655 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8eef2b860 con 0x55a8f30d3180 2024-09-21T02:16:12.985 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:16:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g[72323]: starting mds.g at 2024-09-21T02:16:13.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: pgmap v697: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/665785178' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/665785178' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mon[34754]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f[63231]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f[63231]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:13.034 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mds[63235]: -- 172.21.15.136:0/1431756163 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55910e727860 con 0x5591128cf180 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f[63231]: starting mds.f at 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c[63501]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c[63501]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mds[63505]: -- 172.21.15.136:0/1021479752 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5874b7c20 con 0x55d58b65fa80 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c[63501]: starting mds.c at 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l[62961]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l[62961]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mds[62965]: -- 172.21.15.136:0/2166312338 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c847a27860 con 0x55c84bbcf180 2024-09-21T02:16:13.035 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l[62961]: starting mds.l at 2024-09-21T02:16:13.036 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:13.036 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:13.036 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-mds[63771]: -- 172.21.15.136:0/3837664781 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56025251bc20 con 0x5602566c3180 2024-09-21T02:16:13.036 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:16:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: starting mds.i at 2024-09-21T02:16:13.081 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mds[62422]: -- 172.21.15.99:0/3198629198 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622bf823860 con 0x5622c39cb180 2024-09-21T02:16:13.081 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b[62418]: starting mds.b at 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: pgmap v697: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/665785178' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/665785178' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mon[34404]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h[62143]: ignoring --setuser ceph since I am not root 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h[62143]: ignoring --setgroup ceph since I am not root 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-mds[62147]: -- 172.21.15.99:0/1610965641 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec9380bc20 con 0x55ec979b3180 2024-09-21T02:16:13.082 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:16:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h[62143]: starting mds.h at 2024-09-21T02:16:13.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:16:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 -- 172.21.15.92:0/2688285368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 msgr2=0x7f0c081087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 --2- 172.21.15.92:0/2688285368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c081087b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009900 tx=0x7f0bfc02a410 comp rx=0 tx=0).stop 2024-09-21T02:16:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 -- 172.21.15.92:0/2688285368 shutdown_connections 2024-09-21T02:16:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 --2- 172.21.15.92:0/2688285368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08108cf0 0x7f0c0810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 --2- 172.21.15.92:0/2688285368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c081087b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 --2- 172.21.15.92:0/2688285368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08102a60 0x7f0c08102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 -- 172.21.15.92:0/2688285368 >> 172.21.15.92:0/2688285368 conn(0x7f0c08079730 msgr2=0x7f0c08079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 -- 172.21.15.92:0/2688285368 shutdown_connections 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.069+0000 7f0c0d074640 1 -- 172.21.15.92:0/2688285368 wait complete. 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 Processor -- start 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 -- start start 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08102a60 0x7f0c0819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c0819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08108cf0 0x7f0c081a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08112960 con 0x7f0c081083a0 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081127e0 con 0x7f0c08108cf0 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0d074640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08112ae0 con 0x7f0c08102a60 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c0819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c0819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34122/0 (socket says 172.21.15.92:34122) 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.070+0000 7f0c0659c640 1 -- 172.21.15.92:0/704750285 learned_addr learned my addr 172.21.15.92:0/704750285 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0759e640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08108cf0 0x7f0c081a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c06d9d640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08102a60 0x7f0c0819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0659c640 1 -- 172.21.15.92:0/704750285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08102a60 msgr2=0x7f0c0819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0659c640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08102a60 0x7f0c0819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0659c640 1 -- 172.21.15.92:0/704750285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08108cf0 msgr2=0x7f0c081a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0659c640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08108cf0 0x7f0c081a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0659c640 1 -- 172.21.15.92:0/704750285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c081a2560 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0759e640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08108cf0 0x7f0c081a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c06d9d640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08102a60 0x7f0c0819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0659c640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c0819cc50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc031a90 tx=0x7f0bfc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc03f030 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c081a27f0 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc003cb0 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc035070 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.071+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c081a2d20 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.072+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c08102e70 con 0x7f0c081083a0 2024-09-21T02:16:14.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.074+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7f0bfc042030 con 0x7f0c081083a0 2024-09-21T02:16:14.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.074+0000 7f0beffff640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0bf00b6540 0x7f0bf00b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:14.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.074+0000 7f0c06d9d640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0bf00b6540 0x7f0bf00b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:14.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.075+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8699+0+0 (secure 0 0 0) 0x7f0bfc107f40 con 0x7f0c081083a0 2024-09-21T02:16:14.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.075+0000 7f0c06d9d640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0bf00b6540 0x7f0bf00b8a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80058c0 tx=0x7f0bf8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:14.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.078+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0c081a2d20 con 0x7f0c081083a0 2024-09-21T02:16:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.294+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0c081a91d0 con 0x7f0c081083a0 2024-09-21T02:16:14.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.763+0000 7f0beffff640 1 -- 172.21.15.92:0/704750285 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f0bfc0ce600 con 0x7f0c081083a0 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0bf00b6540 msgr2=0x7f0bf00b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0bf00b6540 0x7f0bf00b8a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80058c0 tx=0x7f0bf8005d30 comp rx=0 tx=0).stop 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 msgr2=0x7f0c0819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c0819cc50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc031a90 tx=0x7f0bfc004270 comp rx=0 tx=0).stop 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 shutdown_connections 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7f0bf00b6540 0x7f0bf00b8a50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c08108cf0 0x7f0c081a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c081083a0 0x7f0c0819cc50 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 --2- 172.21.15.92:0/704750285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08102a60 0x7f0c0819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 >> 172.21.15.92:0/704750285 conn(0x7f0c08079730 msgr2=0x7f0c0806afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 shutdown_connections 2024-09-21T02:16:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:14.766+0000 7f0c0d074640 1 -- 172.21.15.92:0/704750285 wait complete. 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: pgmap v699: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6832/3617435988,v1:172.21.15.99:6834/3617435988] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6832/2727014334,v1:172.21.15.136:6833/2727014334] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6833/3178125219,v1:172.21.15.99:6835/3178125219] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6834/3427241718,v1:172.21.15.136:6835/3427241718] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6838/1187174495,v1:172.21.15.92:6840/1187174495] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.92:6839/3661678389,v1:172.21.15.92:6841/3661678389] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6836/2797727639,v1:172.21.15.136:6837/2797727639] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6836/1438742888,v1:172.21.15.99:6838/1438742888] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.99:6837/4131105658,v1:172.21.15.99:6839/4131105658] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: mds.? [v2:172.21.15.136:6838/2174065392,v1:172.21.15.136:6839/2174065392] up:boot 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T02:16:15.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:16:15.035 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:14 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/704750285' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:16:15.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: pgmap v699: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6832/3617435988,v1:172.21.15.99:6834/3617435988] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6832/2727014334,v1:172.21.15.136:6833/2727014334] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6833/3178125219,v1:172.21.15.99:6835/3178125219] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6834/3427241718,v1:172.21.15.136:6835/3427241718] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6838/1187174495,v1:172.21.15.92:6840/1187174495] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.92:6839/3661678389,v1:172.21.15.92:6841/3661678389] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6836/2797727639,v1:172.21.15.136:6837/2797727639] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6836/1438742888,v1:172.21.15.99:6838/1438742888] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.99:6837/4131105658,v1:172.21.15.99:6839/4131105658] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: mds.? [v2:172.21.15.136:6838/2174065392,v1:172.21.15.136:6839/2174065392] up:boot 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:16:15.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:16:15.056 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:14 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/704750285' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:16:15.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: pgmap v699: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6832/3617435988,v1:172.21.15.99:6834/3617435988] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6832/2727014334,v1:172.21.15.136:6833/2727014334] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6833/3178125219,v1:172.21.15.99:6835/3178125219] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6834/3427241718,v1:172.21.15.136:6835/3427241718] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6838/1187174495,v1:172.21.15.92:6840/1187174495] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.92:6839/3661678389,v1:172.21.15.92:6841/3661678389] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6836/2797727639,v1:172.21.15.136:6837/2797727639] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6836/1438742888,v1:172.21.15.99:6838/1438742888] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.99:6837/4131105658,v1:172.21.15.99:6839/4131105658] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: mds.? [v2:172.21.15.136:6838/2174065392,v1:172.21.15.136:6839/2174065392] up:boot 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:16:15.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='mgr.14150 172.21.15.92:0/2890347326' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:16:15.083 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:14 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/704750285' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:16:15.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:16:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:15.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:15 smithi092 ceph-mon[31190]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T02:16:15.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:15 smithi092 ceph-mon[31190]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T02:16:15.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:15 smithi092 ceph-mon[31190]: Cluster is now healthy 2024-09-21T02:16:15.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:15 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/704750285' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:16:15.804 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:15 smithi092 ceph-mon[31190]: fsmap 12 up:standby 2024-09-21T02:16:16.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:15 smithi136 ceph-mon[34754]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T02:16:16.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:15 smithi136 ceph-mon[34754]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T02:16:16.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:15 smithi136 ceph-mon[34754]: Cluster is now healthy 2024-09-21T02:16:16.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:15 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/704750285' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:16:16.034 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:15 smithi136 ceph-mon[34754]: fsmap 12 up:standby 2024-09-21T02:16:16.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:15 smithi099 ceph-mon[34404]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T02:16:16.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:15 smithi099 ceph-mon[34404]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T02:16:16.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:15 smithi099 ceph-mon[34404]: Cluster is now healthy 2024-09-21T02:16:16.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:15 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/704750285' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:16:16.082 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:15 smithi099 ceph-mon[34404]: fsmap 12 up:standby 2024-09-21T02:16:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.245+0000 7faa35313640 1 -- 172.21.15.92:0/2947666113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30103b60 msgr2=0x7faa30103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.245+0000 7faa35313640 1 --2- 172.21.15.92:0/2947666113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30103b60 0x7faa30103f70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7faa20009900 tx=0x7faa2002a410 comp rx=0 tx=0).stop 2024-09-21T02:16:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.245+0000 7faa35313640 1 -- 172.21.15.92:0/2947666113 shutdown_connections 2024-09-21T02:16:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 --2- 172.21.15.92:0/2947666113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa30109df0 0x7faa30110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 --2- 172.21.15.92:0/2947666113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa301094a0 0x7faa301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 --2- 172.21.15.92:0/2947666113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30103b60 0x7faa30103f70 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 -- 172.21.15.92:0/2947666113 >> 172.21.15.92:0/2947666113 conn(0x7faa3007b850 msgr2=0x7faa3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 -- 172.21.15.92:0/2947666113 shutdown_connections 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 -- 172.21.15.92:0/2947666113 wait complete. 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 Processor -- start 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.246+0000 7faa35313640 1 -- start start 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa35313640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa30103b60 0x7faa3019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa35313640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa301094a0 0x7faa3019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa35313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 0x7faa3019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa35313640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa30113ab0 con 0x7faa30109df0 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa35313640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa30113930 con 0x7faa301094a0 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa35313640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa30113c30 con 0x7faa30103b60 2024-09-21T02:16:16.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 0x7faa3019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 0x7faa3019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34138/0 (socket says 172.21.15.92:34138) 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 -- 172.21.15.92:0/1557113935 learned_addr learned my addr 172.21.15.92:0/1557113935 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2effd640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa30103b60 0x7faa3019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2e7fc640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa301094a0 0x7faa3019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 -- 172.21.15.92:0/1557113935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa30103b60 msgr2=0x7faa3019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa30103b60 0x7faa3019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 -- 172.21.15.92:0/1557113935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa301094a0 msgr2=0x7faa3019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa301094a0 0x7faa3019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 -- 172.21.15.92:0/1557113935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa301a03b0 con 0x7faa30109df0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa2f7fe640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 0x7faa3019fd40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7faa2800cce0 tx=0x7faa2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.247+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2800f450 con 0x7faa30109df0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.248+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2800faa0 con 0x7faa30109df0 2024-09-21T02:16:16.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.248+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa301a06a0 con 0x7faa30109df0 2024-09-21T02:16:16.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.248+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28018b70 con 0x7faa30109df0 2024-09-21T02:16:16.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.248+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa301a0b90 con 0x7faa30109df0 2024-09-21T02:16:16.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.249+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa30103f70 con 0x7faa30109df0 2024-09-21T02:16:16.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.251+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7faa28018d10 con 0x7faa30109df0 2024-09-21T02:16:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.252+0000 7faa17fff640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa180b6540 0x7faa180b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.252+0000 7faa2effd640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa180b6540 0x7faa180b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.252+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8699+0+0 (secure 0 0 0) 0x7faa280e7880 con 0x7faa30109df0 2024-09-21T02:16:16.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.252+0000 7faa2effd640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa180b6540 0x7faa180b8a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7faa20031440 tx=0x7faa20009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.255+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa301a0b90 con 0x7faa30109df0 2024-09-21T02:16:16.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.432+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faa3010bc20 con 0x7faa30109df0 2024-09-21T02:16:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.771+0000 7faa17fff640 1 -- 172.21.15.92:0/1557113935 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) ==== 181+0+0 (secure 0 0 0) 0x7faa28014030 con 0x7faa30109df0 2024-09-21T02:16:16.773 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:16:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.774+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa180b6540 msgr2=0x7faa180b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.774+0000 7faa35313640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa180b6540 0x7faa180b8a50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7faa20031440 tx=0x7faa20009560 comp rx=0 tx=0).stop 2024-09-21T02:16:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.774+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 msgr2=0x7faa3019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.774+0000 7faa35313640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 0x7faa3019fd40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7faa2800cce0 tx=0x7faa2800a620 comp rx=0 tx=0).stop 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 shutdown_connections 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa180b6540 0x7faa180b8a50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa30109df0 0x7faa3019fd40 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa301094a0 0x7faa3019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 --2- 172.21.15.92:0/1557113935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa30103b60 0x7faa3019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 >> 172.21.15.92:0/1557113935 conn(0x7faa3007b850 msgr2=0x7faa301075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.775+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 shutdown_connections 2024-09-21T02:16:16.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:16.776+0000 7faa35313640 1 -- 172.21.15.92:0/1557113935 wait complete. 2024-09-21T02:16:17.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:16 smithi136 ceph-mon[34754]: pgmap v700: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 614 B/s wr, 10 op/s 2024-09-21T02:16:17.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:16 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1557113935' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:16:17.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:16 smithi092 ceph-mon[31190]: pgmap v700: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 614 B/s wr, 10 op/s 2024-09-21T02:16:17.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:16 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1557113935' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:16:17.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:16 smithi099 ceph-mon[34404]: pgmap v700: 97 pgs: 97 active+clean; 4.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 614 B/s wr, 10 op/s 2024-09-21T02:16:17.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:16 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1557113935' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:16:17.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:16:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:16:18.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:17 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1557113935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:16:18.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:17 smithi136 ceph-mon[34754]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:16:18.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:17 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1557113935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:16:18.040 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:17 smithi092 ceph-mon[31190]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:16:18.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:17 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1557113935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:16:18.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:17 smithi099 ceph-mon[34404]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:16:18.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 -- 172.21.15.92:0/3736760269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38106c20 msgr2=0x7faa38110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/3736760269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38106c20 0x7faa38110670 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7faa30009ca0 tx=0x7faa3002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 -- 172.21.15.92:0/3736760269 shutdown_connections 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/3736760269 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38106c20 0x7faa38110670 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/3736760269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa381062d0 0x7faa381066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/3736760269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa38101ff0 0x7faa38102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 -- 172.21.15.92:0/3736760269 >> 172.21.15.92:0/3736760269 conn(0x7faa38079e30 msgr2=0x7faa3807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 -- 172.21.15.92:0/3736760269 shutdown_connections 2024-09-21T02:16:18.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 -- 172.21.15.92:0/3736760269 wait complete. 2024-09-21T02:16:18.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.263+0000 7faa3dfbe640 1 Processor -- start 2024-09-21T02:16:18.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 -- start start 2024-09-21T02:16:18.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 0x7faa3819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa381062d0 0x7faa3819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa38106c20 0x7faa3819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa38113ac0 con 0x7faa38101ff0 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa38113940 con 0x7faa38106c20 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa3dfbe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa38113c40 con 0x7faa381062d0 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 0x7faa3819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 0x7faa3819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34166/0 (socket says 172.21.15.92:34166) 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 -- 172.21.15.92:0/1981578 learned_addr learned my addr 172.21.15.92:0/1981578 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa2ffff640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa381062d0 0x7faa3819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa37fff640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa38106c20 0x7faa3819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 -- 172.21.15.92:0/1981578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa381062d0 msgr2=0x7faa3819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa381062d0 0x7faa3819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 -- 172.21.15.92:0/1981578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa38106c20 msgr2=0x7faa3819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa38106c20 0x7faa3819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 -- 172.21.15.92:0/1981578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa381a03e0 con 0x7faa38101ff0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.264+0000 7faa377fe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 0x7faa3819a470 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7faa2800d950 tx=0x7faa2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.265+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28017070 con 0x7faa38101ff0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.265+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa28009c30 con 0x7faa38101ff0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.265+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa381a06d0 con 0x7faa38101ff0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.265+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2800f620 con 0x7faa38101ff0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.265+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa381a6e30 con 0x7faa38101ff0 2024-09-21T02:16:18.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.266+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa38102400 con 0x7faa38101ff0 2024-09-21T02:16:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.267+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 154512+0+0 (secure 0 0 0) 0x7faa28005030 con 0x7faa38101ff0 2024-09-21T02:16:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.268+0000 7faa357fa640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa240b6590 0x7faa240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.268+0000 7faa2ffff640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa240b6590 0x7faa240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.268+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8174+0+0 (secure 0 0 0) 0x7faa280e6de0 con 0x7faa38101ff0 2024-09-21T02:16:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.269+0000 7faa2ffff640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa240b6590 0x7faa240b8aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa3810a280 tx=0x7faa20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:18.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.271+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa280ad780 con 0x7faa38101ff0 2024-09-21T02:16:18.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.450+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faa3819b5a0 con 0x7faa38101ff0 2024-09-21T02:16:18.786 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.785+0000 7faa357fa640 1 -- 172.21.15.92:0/1981578 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) ==== 169+0+0 (secure 0 0 0) 0x7faa28009de0 con 0x7faa38101ff0 2024-09-21T02:16:18.786 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2024-09-21T02:16:18.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa240b6590 msgr2=0x7faa240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa240b6590 0x7faa240b8aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa3810a280 tx=0x7faa20005f90 comp rx=0 tx=0).stop 2024-09-21T02:16:18.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 msgr2=0x7faa3819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 0x7faa3819a470 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7faa2800d950 tx=0x7faa2800de10 comp rx=0 tx=0).stop 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 shutdown_connections 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:6800/1870152537,v1:172.21.15.92:6801/1870152537] conn(0x7faa240b6590 0x7faa240b8aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa38106c20 0x7faa3819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa381062d0 0x7faa3819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 --2- 172.21.15.92:0/1981578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa38101ff0 0x7faa3819a470 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.789+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 >> 172.21.15.92:0/1981578 conn(0x7faa38079e30 msgr2=0x7faa381074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.790+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 shutdown_connections 2024-09-21T02:16:18.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-21T02:16:18.790+0000 7faa3dfbe640 1 -- 172.21.15.92:0/1981578 wait complete. 2024-09-21T02:16:19.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:18 smithi136 ceph-mon[34754]: pgmap v702: 65 pgs: 65 active+clean; 577 KiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:16:19.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:18 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1981578' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:16:19.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:18 smithi092 ceph-mon[31190]: pgmap v702: 65 pgs: 65 active+clean; 577 KiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:16:19.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:18 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1981578' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:16:19.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:18 smithi099 ceph-mon[34404]: pgmap v702: 65 pgs: 65 active+clean; 577 KiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:16:19.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:18 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1981578' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T02:16:19.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:16:19.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:16:19.454 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:16:19.487 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T02:16:19.488 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:16:19.503 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:16:19.521 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:16:19.544 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T02:16:19.544 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T02:16:19.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a 2024-09-21T02:16:19.792 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:19 smithi092 systemd[1]: Stopping Ceph mon.a for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:20.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:19 smithi092 ceph-mon[31190]: from='client.? 172.21.15.92:0/1981578' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:16:20.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:19 smithi092 ceph-mon[31190]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T02:16:20.054 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:19 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a[31186]: 2024-09-21T02:16:19.853+0000 7ff6a569b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:19 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a[31186]: 2024-09-21T02:16:19.853+0000 7ff6a569b640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T02:16:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:19 smithi092 podman[108536]: 2024-09-21 02:16:19.895126981 +0000 UTC m=+0.159894097 container died 082f4472a4da4395e63f54c692064907d4a8c4d89713730cea5497c983850d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T02:16:20.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:19 smithi099 ceph-mon[34404]: from='client.? 172.21.15.92:0/1981578' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:16:20.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:19 smithi099 ceph-mon[34404]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T02:16:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:19 smithi136 ceph-mon[34754]: from='client.? 172.21.15.92:0/1981578' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T02:16:20.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:19 smithi136 ceph-mon[34754]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T02:16:20.531 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:20 smithi092 podman[108536]: 2024-09-21 02:16:20.253578514 +0000 UTC m=+0.518345613 container cleanup 082f4472a4da4395e63f54c692064907d4a8c4d89713730cea5497c983850d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T02:16:20.531 INFO:journalctl@ceph.mon.a.smithi092.stdout:Sep 21 02:16:20 smithi092 bash[108536]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-a 2024-09-21T02:16:20.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T02:16:20.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:20.731 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T02:16:20.731 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T02:16:20.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.b 2024-09-21T02:16:21.081 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:20 smithi099 systemd[1]: Stopping Ceph mon.b for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:21.519 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b[34400]: 2024-09-21T02:16:21.175+0000 7fc3216c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:21.519 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b[34400]: 2024-09-21T02:16:21.175+0000 7fc3216c3640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T02:16:21.519 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:21 smithi099 podman[66442]: 2024-09-21 02:16:21.247132199 +0000 UTC m=+0.317803642 container died 8ed0cbe597e5f7f0a7eb399cfb6d87e665b5784d400564380fd441043593d439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:16:21.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:21 smithi099 podman[66442]: 2024-09-21 02:16:21.617172261 +0000 UTC m=+0.687843706 container cleanup 8ed0cbe597e5f7f0a7eb399cfb6d87e665b5784d400564380fd441043593d439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-21T02:16:21.831 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 21 02:16:21 smithi099 bash[66442]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-b 2024-09-21T02:16:22.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T02:16:22.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:22.094 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T02:16:22.094 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T02:16:22.094 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.c 2024-09-21T02:16:22.376 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:22 smithi136 systemd[1]: Stopping Ceph mon.c for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:22.776 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-c[34750]: 2024-09-21T02:16:22.375+0000 7fa68ebe0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:22.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-c[34750]: 2024-09-21T02:16:22.375+0000 7fa68ebe0640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T02:16:22.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:22 smithi136 podman[67160]: 2024-09-21 02:16:22.518762313 +0000 UTC m=+0.231387917 container died e245b3464d87d7d51c148d131ef09e94124e2d3aead4bd364d2bcb93099fa188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T02:16:23.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:22 smithi136 podman[67160]: 2024-09-21 02:16:22.875468835 +0000 UTC m=+0.588094433 container cleanup e245b3464d87d7d51c148d131ef09e94124e2d3aead4bd364d2bcb93099fa188 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:16:23.033 INFO:journalctl@ceph.mon.c.smithi136.stdout:Sep 21 02:16:22 smithi136 bash[67160]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mon-c 2024-09-21T02:16:23.293 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T02:16:23.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:23.346 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T02:16:23.346 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T02:16:23.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x 2024-09-21T02:16:23.658 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:16:23 smithi092 systemd[1]: Stopping Ceph mgr.x for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:23.992 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:16:23 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:16:23.657+0000 7fc21e254640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:23.993 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:16:23 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x[31413]: 2024-09-21T02:16:23.657+0000 7fc21e254640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T02:16:23.993 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:16:23 smithi092 podman[108660]: 2024-09-21 02:16:23.688494091 +0000 UTC m=+0.148938865 container died 024a4f715ef9ece78f6764b43ee2b02d2c896b0ee3c85d23ea1dbaf319358a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T02:16:24.304 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:16:24 smithi092 podman[108660]: 2024-09-21 02:16:24.083033795 +0000 UTC m=+0.543478552 container cleanup 024a4f715ef9ece78f6764b43ee2b02d2c896b0ee3c85d23ea1dbaf319358a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T02:16:24.304 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Sep 21 02:16:24 smithi092 bash[108660]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-x 2024-09-21T02:16:24.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T02:16:24.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:24.542 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T02:16:24.543 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T02:16:24.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.y 2024-09-21T02:16:24.831 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 02:16:24 smithi099 systemd[1]: Stopping Ceph mgr.y for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:25.147 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 02:16:24 smithi099 podman[66565]: 2024-09-21 02:16:24.884377918 +0000 UTC m=+0.148106121 container died 9c2cd6367ad0a9d1909c7a45481ea513fe6270d7542084f78e22dcb9dfedf44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T02:16:25.448 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 02:16:25 smithi099 podman[66565]: 2024-09-21 02:16:25.230346795 +0000 UTC m=+0.494075005 container cleanup 9c2cd6367ad0a9d1909c7a45481ea513fe6270d7542084f78e22dcb9dfedf44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-y, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:16:25.448 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 21 02:16:25 smithi099 bash[66565]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-y 2024-09-21T02:16:25.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T02:16:25.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:25.695 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T02:16:25.695 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T02:16:25.696 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.z 2024-09-21T02:16:26.033 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 02:16:25 smithi136 systemd[1]: Stopping Ceph mgr.z for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:26.322 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 02:16:26 smithi136 podman[67286]: 2024-09-21 02:16:26.034139392 +0000 UTC m=+0.144542155 container died 4917858bb7a6c3e3a7c42142a26e9733d5ea2ea5937ccd65d199c3624679f791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:16:26.681 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 02:16:26 smithi136 podman[67286]: 2024-09-21 02:16:26.421422862 +0000 UTC m=+0.531825673 container cleanup 4917858bb7a6c3e3a7c42142a26e9733d5ea2ea5937ccd65d199c3624679f791 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:16:26.681 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Sep 21 02:16:26 smithi136 bash[67286]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mgr-z 2024-09-21T02:16:26.804 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T02:16:26.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:26.839 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T02:16:26.839 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T02:16:26.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.0 2024-09-21T02:16:27.155 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:26 smithi092 systemd[1]: Stopping Ceph osd.0 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:27.554 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0[46912]: 2024-09-21T02:16:27.154+0000 7fda21fbe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:27.554 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0[46912]: 2024-09-21T02:16:27.154+0000 7fda21fbe640 -1 osd.0 80 *** Got signal Terminated *** 2024-09-21T02:16:27.555 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0[46912]: 2024-09-21T02:16:27.154+0000 7fda21fbe640 -1 osd.0 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:16:32.963 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:32 smithi092 podman[108781]: 2024-09-21 02:16:32.627018435 +0000 UTC m=+5.595964367 container died e7f5cee346ee85ac060c92391335a80f11a01554cd66486fb1736b54db96589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T02:16:33.263 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108781]: 2024-09-21 02:16:33.028995982 +0000 UTC m=+5.997941914 container cleanup e7f5cee346ee85ac060c92391335a80f11a01554cd66486fb1736b54db96589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T02:16:33.264 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 bash[108781]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0 2024-09-21T02:16:33.554 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108798]: 2024-09-21 02:16:33.263671068 +0000 UTC m=+0.631869581 container remove e7f5cee346ee85ac060c92391335a80f11a01554cd66486fb1736b54db96589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-21T02:16:33.867 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108883]: 2024-09-21 02:16:33.498852659 +0000 UTC m=+0.015709414 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:16:33.867 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108883]: 2024-09-21 02:16:33.614567948 +0000 UTC m=+0.131424703 container create c3801742ff5fcc0c015ab3492f215aa4b45b591ed53aeeaaa49d815ee1bad1d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:16:33.867 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108883]: 2024-09-21 02:16:33.867198236 +0000 UTC m=+0.384054999 container init c3801742ff5fcc0c015ab3492f215aa4b45b591ed53aeeaaa49d815ee1bad1d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T02:16:34.305 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108883]: 2024-09-21 02:16:33.870830208 +0000 UTC m=+0.387686976 container start c3801742ff5fcc0c015ab3492f215aa4b45b591ed53aeeaaa49d815ee1bad1d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:16:34.306 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:33 smithi092 podman[108883]: 2024-09-21 02:16:33.958759064 +0000 UTC m=+0.475615854 container attach c3801742ff5fcc0c015ab3492f215aa4b45b591ed53aeeaaa49d815ee1bad1d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T02:16:34.306 INFO:journalctl@ceph.osd.0.smithi092.stdout:Sep 21 02:16:34 smithi092 podman[108883]: 2024-09-21 02:16:34.055975152 +0000 UTC m=+0.572831918 container died c3801742ff5fcc0c015ab3492f215aa4b45b591ed53aeeaaa49d815ee1bad1d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:16:34.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T02:16:34.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:34.661 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T02:16:34.661 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T02:16:34.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.1 2024-09-21T02:16:34.959 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:34 smithi099 systemd[1]: Stopping Ceph osd.1 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:35.331 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1[38361]: 2024-09-21T02:16:34.958+0000 7fc056247640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:35.331 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1[38361]: 2024-09-21T02:16:34.958+0000 7fc056247640 -1 osd.1 80 *** Got signal Terminated *** 2024-09-21T02:16:35.331 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1[38361]: 2024-09-21T02:16:34.958+0000 7fc056247640 -1 osd.1 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:16:40.651 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:40 smithi099 podman[66703]: 2024-09-21 02:16:40.372038255 +0000 UTC m=+5.522983160 container died 31a60bbe08b141bc4dcdd376ad514c9a6d9f36ac779bc0a4c7a2394b4ef807b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:16:40.993 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:40 smithi099 podman[66703]: 2024-09-21 02:16:40.758288885 +0000 UTC m=+5.909233782 container cleanup 31a60bbe08b141bc4dcdd376ad514c9a6d9f36ac779bc0a4c7a2394b4ef807b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T02:16:40.993 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:40 smithi099 bash[66703]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1 2024-09-21T02:16:41.318 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:40 smithi099 podman[66720]: 2024-09-21 02:16:40.992957704 +0000 UTC m=+0.617832790 container remove 31a60bbe08b141bc4dcdd376ad514c9a6d9f36ac779bc0a4c7a2394b4ef807b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:16:41.581 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:41 smithi099 podman[66807]: 2024-09-21 02:16:41.221401633 +0000 UTC m=+0.015664018 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:16:41.581 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:41 smithi099 podman[66807]: 2024-09-21 02:16:41.360634567 +0000 UTC m=+0.154896988 container create af2e0d04f6558dd7be35e530682e1b6e548e04c29e3b9f842fc3d44cd240acb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T02:16:42.082 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:41 smithi099 podman[66807]: 2024-09-21 02:16:41.61082226 +0000 UTC m=+0.405084642 container init af2e0d04f6558dd7be35e530682e1b6e548e04c29e3b9f842fc3d44cd240acb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T02:16:42.082 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:41 smithi099 podman[66807]: 2024-09-21 02:16:41.614195926 +0000 UTC m=+0.408458304 container start af2e0d04f6558dd7be35e530682e1b6e548e04c29e3b9f842fc3d44cd240acb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:16:42.083 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:41 smithi099 podman[66807]: 2024-09-21 02:16:41.70467384 +0000 UTC m=+0.498936226 container attach af2e0d04f6558dd7be35e530682e1b6e548e04c29e3b9f842fc3d44cd240acb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:16:42.083 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 21 02:16:41 smithi099 podman[66807]: 2024-09-21 02:16:41.79707761 +0000 UTC m=+0.591339999 container died af2e0d04f6558dd7be35e530682e1b6e548e04c29e3b9f842fc3d44cd240acb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:16:42.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T02:16:42.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:42.449 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T02:16:42.449 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T02:16:42.450 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.2 2024-09-21T02:16:42.741 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:42 smithi136 systemd[1]: Stopping Ceph osd.2 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:43.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2[39147]: 2024-09-21T02:16:42.740+0000 7f4c9d5f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:43.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2[39147]: 2024-09-21T02:16:42.740+0000 7f4c9d5f3640 -1 osd.2 80 *** Got signal Terminated *** 2024-09-21T02:16:43.033 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2[39147]: 2024-09-21T02:16:42.740+0000 7f4c9d5f3640 -1 osd.2 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:16:48.533 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:48 smithi136 podman[67427]: 2024-09-21 02:16:48.254576768 +0000 UTC m=+5.606432348 container died 1bcce4f4557356399f43a2a8c5644376f47473a72ae7e19c7c55b24f7d7b98a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:16:48.874 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:48 smithi136 podman[67427]: 2024-09-21 02:16:48.647919083 +0000 UTC m=+5.999774662 container cleanup 1bcce4f4557356399f43a2a8c5644376f47473a72ae7e19c7c55b24f7d7b98a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T02:16:48.874 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:48 smithi136 bash[67427]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2 2024-09-21T02:16:49.203 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:48 smithi136 podman[67443]: 2024-09-21 02:16:48.874032766 +0000 UTC m=+0.613545505 container remove 1bcce4f4557356399f43a2a8c5644376f47473a72ae7e19c7c55b24f7d7b98a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T02:16:49.511 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:49 smithi136 podman[67533]: 2024-09-21 02:16:49.10566609 +0000 UTC m=+0.015449892 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:16:49.511 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:49 smithi136 podman[67533]: 2024-09-21 02:16:49.23339023 +0000 UTC m=+0.143174071 container create 360bd672ff82bbac254b0fadc6297ae3dd5633f30d448a101d4ca2a8016057a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:16:49.511 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:49 smithi136 podman[67533]: 2024-09-21 02:16:49.435685156 +0000 UTC m=+0.345468956 container init 360bd672ff82bbac254b0fadc6297ae3dd5633f30d448a101d4ca2a8016057a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:16:49.511 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:49 smithi136 podman[67533]: 2024-09-21 02:16:49.439105147 +0000 UTC m=+0.348888947 container start 360bd672ff82bbac254b0fadc6297ae3dd5633f30d448a101d4ca2a8016057a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:16:49.511 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:49 smithi136 podman[67533]: 2024-09-21 02:16:49.510582906 +0000 UTC m=+0.420366706 container attach 360bd672ff82bbac254b0fadc6297ae3dd5633f30d448a101d4ca2a8016057a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:16:49.784 INFO:journalctl@ceph.osd.2.smithi136.stdout:Sep 21 02:16:49 smithi136 podman[67533]: 2024-09-21 02:16:49.625233043 +0000 UTC m=+0.535016847 container died 360bd672ff82bbac254b0fadc6297ae3dd5633f30d448a101d4ca2a8016057a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:16:50.200 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T02:16:50.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:50.240 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T02:16:50.240 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T02:16:50.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.3 2024-09-21T02:16:50.547 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:50 smithi092 systemd[1]: Stopping Ceph osd.3 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:50.804 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:50 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T02:16:50.545+0000 7f25a0c5b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:50.804 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:50 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T02:16:50.545+0000 7f25a0c5b640 -1 osd.3 80 *** Got signal Terminated *** 2024-09-21T02:16:50.804 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:50 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T02:16:50.545+0000 7f25a0c5b640 -1 osd.3 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:16:53.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:52.872+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:53.805 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:53 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:53.493+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:16:54.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:53.858+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:54.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:54 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:54.526+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:16:55.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:16:54 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:16:54.677+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:16:55.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:54.810+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:55.804 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:55 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3[53496]: 2024-09-21T02:16:55.340+0000 7f259923e640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.887615+0000 front 2024-09-21T02:16:29.887820+0000 (oldest deadline 2024-09-21T02:16:54.587718+0000) 2024-09-21T02:16:55.805 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:55 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:55.495+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:16:56.033 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:55.768+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:56.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:16:55 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:16:55.718+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:16:56.246 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:55 smithi092 podman[108986]: 2024-09-21 02:16:55.957122963 +0000 UTC m=+5.512487242 container died ac8a8b2b877bbdaa727c89d57c863f209a15863c542563e6c00f12146dd979a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T02:16:56.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:16:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:16:56.086+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:16:56.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:56 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:56.484+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:16:56.554 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:56 smithi092 podman[108986]: 2024-09-21 02:16:56.437499137 +0000 UTC m=+5.992863452 container cleanup ac8a8b2b877bbdaa727c89d57c863f209a15863c542563e6c00f12146dd979a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T02:16:56.554 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:56 smithi092 bash[108986]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3 2024-09-21T02:16:57.033 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:56.772+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:57.054 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:56 smithi092 podman[109001]: 2024-09-21 02:16:56.72254712 +0000 UTC m=+0.764210188 container remove ac8a8b2b877bbdaa727c89d57c863f209a15863c542563e6c00f12146dd979a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T02:16:57.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:16:56 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:16:56.876+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:16:57.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:16:56 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:16:56.716+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:16:57.392 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:57 smithi092 podman[109092]: 2024-09-21 02:16:56.968218906 +0000 UTC m=+0.015455790 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:16:57.392 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:57 smithi092 podman[109092]: 2024-09-21 02:16:57.081652253 +0000 UTC m=+0.128889119 container create 4c58f48a6c089402d445a5cfa62370b14b2f1ac0fc371b5d30d29ec30aa4b6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:16:57.392 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:57 smithi092 podman[109092]: 2024-09-21 02:16:57.308970899 +0000 UTC m=+0.356207787 container init 4c58f48a6c089402d445a5cfa62370b14b2f1ac0fc371b5d30d29ec30aa4b6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T02:16:57.392 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:57 smithi092 podman[109092]: 2024-09-21 02:16:57.312366175 +0000 UTC m=+0.359603043 container start 4c58f48a6c089402d445a5cfa62370b14b2f1ac0fc371b5d30d29ec30aa4b6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T02:16:57.392 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:57 smithi092 podman[109092]: 2024-09-21 02:16:57.392287669 +0000 UTC m=+0.439524550 container attach 4c58f48a6c089402d445a5cfa62370b14b2f1ac0fc371b5d30d29ec30aa4b6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T02:16:57.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:16:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:16:57.071+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:16:57.788 INFO:journalctl@ceph.osd.3.smithi092.stdout:Sep 21 02:16:57 smithi092 podman[109092]: 2024-09-21 02:16:57.499434844 +0000 UTC m=+0.546671719 container died 4c58f48a6c089402d445a5cfa62370b14b2f1ac0fc371b5d30d29ec30aa4b6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T02:16:57.788 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:57 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:57.500+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:16:58.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:16:57 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:16:57.917+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:16:58.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:16:57 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:16:57.748+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:16:58.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T02:16:58.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:16:58.129 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T02:16:58.129 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T02:16:58.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.4 2024-09-21T02:16:58.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:57.812+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:58.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:16:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:16:58.034+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:16:58.460 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:16:58 smithi099 systemd[1]: Stopping Ceph osd.4 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:16:58.761 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:16:58 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T02:16:58.459+0000 7f64644aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:16:58.761 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:16:58 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T02:16:58.459+0000 7f64644aa640 -1 osd.4 80 *** Got signal Terminated *** 2024-09-21T02:16:58.761 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:16:58 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T02:16:58.459+0000 7f64644aa640 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:16:58.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:58 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:58.543+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:16:59.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:16:58 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:16:58.760+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:16:59.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:58.811+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:16:59.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:16:59 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:16:59.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:16:59.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:16:58 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:16:58.960+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:16:59.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:16:59 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:16:59.526+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:00.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:16:59 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:16:59.761+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:00.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:16:59 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:16:59.838+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:00.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:00 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:00.067+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:00.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:16:59 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:16:59.921+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:00.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:00 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:00.498+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:01.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:00 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:00.806+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:01.193 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:00 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:00.870+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:01.194 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:01 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:01.062+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:01.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:00 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:00.884+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:01.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:01 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:01.192+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:01.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:01 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:01.545+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:01.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:01 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:01.545+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:02.189 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:01 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:01.869+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:02.190 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:02 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:02.022+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:02.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:01 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:01.856+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:02.331 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:02 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T02:17:02.067+0000 7f645ca8d640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.902006+0000 front 2024-09-21T02:16:37.902054+0000 (oldest deadline 2024-09-21T02:17:02.002039+0000) 2024-09-21T02:17:02.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:01 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:01.844+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:02.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:02 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:02.188+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:02.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:02 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:02.551+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:02.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:02 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:02.551+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:03.142 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:02 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:02.841+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:03.142 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:03 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:03.018+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:03.232 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:03 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4[44493]: 2024-09-21T02:17:03.037+0000 7f645ca8d640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.902006+0000 front 2024-09-21T02:16:37.902054+0000 (oldest deadline 2024-09-21T02:17:02.002039+0000) 2024-09-21T02:17:03.233 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:02 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:02.859+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:03.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:02 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:02.824+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:03.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:03 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:03.141+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:03.581 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:03 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:03.398+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:03.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:03 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:03.534+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:03.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:03 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:03.534+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:04.136 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:04 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:04.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:04.136 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:03 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:03.814+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:04.136 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:03 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:03.814+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:04.253 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:03 smithi099 podman[66912]: 2024-09-21 02:17:03.973694983 +0000 UTC m=+5.646610785 container died 82c3ec017e4a15d08ead35427ee33dc6332737b282221f1e84152d3a7bc5abde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T02:17:04.253 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:03 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:03.851+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:04.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:03 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:03.861+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:04.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:03 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:03.861+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:04.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:04 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:04.135+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:04.581 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:04 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:04.401+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:04.581 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:04 smithi099 podman[66912]: 2024-09-21 02:17:04.342870384 +0000 UTC m=+6.015786184 container cleanup 82c3ec017e4a15d08ead35427ee33dc6332737b282221f1e84152d3a7bc5abde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T02:17:04.582 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:04 smithi099 bash[66912]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4 2024-09-21T02:17:04.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:04 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:04.492+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:04.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:04 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:04.492+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:04.896 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:04 smithi099 podman[66927]: 2024-09-21 02:17:04.627633971 +0000 UTC m=+0.655313777 container remove 82c3ec017e4a15d08ead35427ee33dc6332737b282221f1e84152d3a7bc5abde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T02:17:05.096 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:04 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:04.841+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:05.096 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:04 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:04.841+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:05.206 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:04 smithi099 podman[67017]: 2024-09-21 02:17:04.860425131 +0000 UTC m=+0.015529143 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:05.206 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:04 smithi099 podman[67017]: 2024-09-21 02:17:04.986746921 +0000 UTC m=+0.141850934 container create 53de6a91b94f282e875c72eb409ac393f2c9c8cff080668654da919ade50b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T02:17:05.206 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:04 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:04.896+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:05.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:04 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:04.861+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:05.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:04 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:04.861+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:05.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:05 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:05.176+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:05.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:05 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:05.095+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:05.581 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:05 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:05.374+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:05.582 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:05 smithi099 podman[67017]: 2024-09-21 02:17:05.205724349 +0000 UTC m=+0.360828361 container init 53de6a91b94f282e875c72eb409ac393f2c9c8cff080668654da919ade50b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T02:17:05.582 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:05 smithi099 podman[67017]: 2024-09-21 02:17:05.209162946 +0000 UTC m=+0.364266965 container start 53de6a91b94f282e875c72eb409ac393f2c9c8cff080668654da919ade50b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T02:17:05.582 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:05 smithi099 podman[67017]: 2024-09-21 02:17:05.41417381 +0000 UTC m=+0.569277835 container attach 53de6a91b94f282e875c72eb409ac393f2c9c8cff080668654da919ade50b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:17:05.582 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 21 02:17:05 smithi099 podman[67017]: 2024-09-21 02:17:05.414500198 +0000 UTC m=+0.569604220 container died 53de6a91b94f282e875c72eb409ac393f2c9c8cff080668654da919ade50b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T02:17:05.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:05.491+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:05.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:05.491+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:05.959 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:05 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:05.864+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:05.959 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:05 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:05.864+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:05.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T02:17:06.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:06.024 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T02:17:06.024 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T02:17:06.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.5 2024-09-21T02:17:06.075 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:05 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:05.822+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:06.075 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:05 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:05.822+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:06.075 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:06.051+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:06.075 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:06.051+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:06.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:05.826+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:06.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:05 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:05.826+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:06.340 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:06 smithi136 systemd[1]: Stopping Ceph osd.5 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:06.340 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:06.212+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:06.783 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:06.339+0000 7eff98440640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:06.783 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:06.339+0000 7eff98440640 -1 osd.5 80 *** Got signal Terminated *** 2024-09-21T02:17:06.783 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:06.339+0000 7eff98440640 -1 osd.5 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:06.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:06 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:06.494+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:06.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:06 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:06.494+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:06.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:06 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:06.337+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:07.170 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:06.859+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:07.170 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:06 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:06.859+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:07.170 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:07 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:07.075+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:07.170 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:07 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:07.075+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:07.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:06 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:06.870+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:07.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:06 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:06.870+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:07.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:06 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:06.862+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:07.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:06 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:06.862+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:07.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:07 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:07.170+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:07.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:07 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:07.522+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:07.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:07 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:07.522+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:07.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:07 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:07.344+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:08.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:07 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:07.853+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:08.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:07 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:07.853+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:08.182 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:08 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:08.042+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:08.183 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:08 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:08.042+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:08.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:07 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:07.896+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:08.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:07 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:07.896+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:08.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:07 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:07.878+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:08.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:07 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:07.878+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:08.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:08 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:08.180+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:08.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:08 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:08.478+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:08.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:08 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:08.478+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:08.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:08 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:08.340+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:09.191 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:09 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:09.071+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:09.191 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:09 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:09.071+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:09.191 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:08 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:08.822+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:09.191 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:08 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:08.822+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:09.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:08 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:08.925+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:09.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:08 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:08.925+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:09.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:08 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:08.849+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:09.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:08 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:08.849+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:09.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:08 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:08.849+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:09.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:09 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:09.190+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:09.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:09.504+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:09.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:09.504+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:09.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:09 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:09.374+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:10.090 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:09 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:09.795+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:10.090 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:09 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:09.795+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:10.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:09.878+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:10.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:09.878+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:10.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:09.878+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:10.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:09 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:09.837+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:10.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:09 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:09.837+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:10.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:09 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:09.837+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:10.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:10.231+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:10.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:10.089+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:10.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:10.089+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:10.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:10.089+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:10.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:10 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:10.461+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:10.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:10 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:10.461+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:10.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:10 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:10.337+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:10.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:10 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:10.337+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:11.033 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:10.765+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:31.612968+0000 front 2024-09-21T02:16:31.613010+0000 (oldest deadline 2024-09-21T02:16:52.713058+0000) 2024-09-21T02:17:11.033 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:10.765+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:37.413923+0000 front 2024-09-21T02:16:37.413980+0000 (oldest deadline 2024-09-21T02:17:03.313900+0000) 2024-09-21T02:17:11.033 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:10 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5[45713]: 2024-09-21T02:17:10.765+0000 7eff90a23640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.815058+0000 front 2024-09-21T02:16:46.815090+0000 (oldest deadline 2024-09-21T02:17:10.315196+0000) 2024-09-21T02:17:11.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:10 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:10.926+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:11.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:10 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:10.926+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:11.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:10 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:10.926+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:11.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:10 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:10.873+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:11.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:10 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:10.873+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:11.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:10 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:10.873+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:11.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:11 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:11.277+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:11.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:11 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:11.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:11.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:11 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:11.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:11.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:11 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:11.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:11.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:11.416+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:11.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:11.416+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:11.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:11 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:11.382+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:11.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:11 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:11.382+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:12.033 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:11 smithi136 podman[67646]: 2024-09-21 02:17:11.764977795 +0000 UTC m=+5.543906014 container died 11097b7156176a0caf3a641221b5207be55f954dc3f61fd6ec07f7d746a774a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T02:17:12.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:11.886+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:12.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:11.886+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:12.306 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:11 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:11.886+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:12.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:11 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:11.846+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:12.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:11 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:11.846+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:12.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:11 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:11.846+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:12.382 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 podman[67646]: 2024-09-21 02:17:12.163514001 +0000 UTC m=+5.942442220 container cleanup 11097b7156176a0caf3a641221b5207be55f954dc3f61fd6ec07f7d746a774a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T02:17:12.382 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 bash[67646]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5 2024-09-21T02:17:12.382 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:12.261+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:12.382 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:12.146+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:12.382 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:12.146+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:12.383 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:12 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:12.146+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:12.690 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 podman[67664]: 2024-09-21 02:17:12.381639422 +0000 UTC m=+0.611180852 container remove 11097b7156176a0caf3a641221b5207be55f954dc3f61fd6ec07f7d746a774a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T02:17:12.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:12.418+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:12.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:12.418+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:12.805 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:12.418+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:12.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:12.386+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:12.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:12.386+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:13.001 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 podman[67752]: 2024-09-21 02:17:12.59289997 +0000 UTC m=+0.015920589 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:13.001 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 podman[67752]: 2024-09-21 02:17:12.699147726 +0000 UTC m=+0.122168364 container create 90f33bf362ff87b04462b31ebe6345a25e20d703decc009b761bde3c7c8e6dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T02:17:13.002 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 podman[67752]: 2024-09-21 02:17:12.917920574 +0000 UTC m=+0.340941178 container init 90f33bf362ff87b04462b31ebe6345a25e20d703decc009b761bde3c7c8e6dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:17:13.002 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:12 smithi136 podman[67752]: 2024-09-21 02:17:12.921198924 +0000 UTC m=+0.344219520 container start 90f33bf362ff87b04462b31ebe6345a25e20d703decc009b761bde3c7c8e6dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T02:17:13.002 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:13 smithi136 podman[67752]: 2024-09-21 02:17:13.001114694 +0000 UTC m=+0.424135300 container attach 90f33bf362ff87b04462b31ebe6345a25e20d703decc009b761bde3c7c8e6dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T02:17:13.283 INFO:journalctl@ceph.osd.5.smithi136.stdout:Sep 21 02:17:13 smithi136 podman[67752]: 2024-09-21 02:17:13.109607192 +0000 UTC m=+0.532627789 container died 90f33bf362ff87b04462b31ebe6345a25e20d703decc009b761bde3c7c8e6dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:17:13.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:13 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:13.108+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:13.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:13 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:13.108+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:13.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:13 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:13.108+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:13.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:12.896+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:13.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:12.896+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:13.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:12 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:12.896+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:13.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:12.838+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:13.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:12.838+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:13.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:12 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:12.838+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:13.654 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:13 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:13.299+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:13.654 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:13 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:13.299+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:13.675 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T02:17:13.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:13.714 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T02:17:13.714 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T02:17:13.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.6 2024-09-21T02:17:13.740 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:13.379+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:13.740 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:13.379+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:13.740 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:13.379+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:13.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:13 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:13.409+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:13.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:13 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:13.409+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:13.997 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 systemd[1]: Stopping Ceph osd.6 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:13.998 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:13.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:13.998 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:13.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:13.998 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:13.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:14.304 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:13.996+0000 7fd143fba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:14.305 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:13.996+0000 7fd143fba640 -1 osd.6 80 *** Got signal Terminated *** 2024-09-21T02:17:14.305 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:13.996+0000 7fd143fba640 -1 osd.6 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:14.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:13 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:13.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:14.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:13 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:13.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:14.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:13 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:13.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:14.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:14 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:14.314+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:14.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:14 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:14.314+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:14.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:14 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:14.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:14.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:14 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:14.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:14.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:14 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:14.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:14.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:14 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:14.358+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:14.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:14 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:14.358+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:14.805 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:14 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:14.358+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:14.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:14 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:14.367+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:14.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:14 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:14.367+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:15.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:15 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:15.031+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:15.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:15 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:15.031+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:15.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:15 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:15.031+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:15.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:14 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:14.912+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:15.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:14 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:14.912+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:15.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:14 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:14.912+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:15.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:14 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:14.894+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:15.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:14 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:14.894+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:15.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:14 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:14.894+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:15.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:15 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:15.338+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:15.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:15 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:15.338+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:15.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:15 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:15.317+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:15.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:15 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:15.317+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:15.804 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:15 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:15.317+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:15.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:15 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:15.391+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:15.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:15 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:15.391+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:16.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:16.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:16.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:16.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:16.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:16.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:16.290 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:15 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:15.894+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:16.290 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:15 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:15.894+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:16.290 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:15 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:15.894+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:16.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:15 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:15.855+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:16.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:15 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:15.855+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:16.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:15 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:15.855+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:16.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:16.290+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:16.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:16.290+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:16.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:16.290+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:16.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:16.322+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:16.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:16.322+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:16.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:16 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:16.396+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:16.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:16 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:16.396+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:17.276 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:16.930+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:17.277 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:16.930+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:17.277 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:16.930+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:17.277 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:16 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:16.930+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:17.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:16.962+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:17.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:16.962+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:17.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:16 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:16.962+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:17.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:16 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:16.887+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:17.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:16 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:16.887+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:17.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:16 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:16.887+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:17.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:17.275+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:17.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:17.275+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:17.555 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:17.275+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:17.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:17 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:17.348+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:17.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:17 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:17.348+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:17.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:17 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:17.348+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:17.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:17 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:17.436+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:17.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:17 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:17.436+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:18.251 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:17.927+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:18.251 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:17.927+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:18.251 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:17.927+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:18.251 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:17 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:17.927+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:18.251 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:18.250+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:27.059527+0000 front 2024-09-21T02:16:27.059769+0000 (oldest deadline 2024-09-21T02:16:52.959721+0000) 2024-09-21T02:17:18.252 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:18.250+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:35.261270+0000 front 2024-09-21T02:16:35.261105+0000 (oldest deadline 2024-09-21T02:17:01.161140+0000) 2024-09-21T02:17:18.252 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:18.250+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:47.463734+0000 front 2024-09-21T02:16:47.463634+0000 (oldest deadline 2024-09-21T02:17:11.563443+0000) 2024-09-21T02:17:18.252 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6[59943]: 2024-09-21T02:17:18.250+0000 7fd13c59d640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.563822+0000 front 2024-09-21T02:16:51.563909+0000 (oldest deadline 2024-09-21T02:17:17.964370+0000) 2024-09-21T02:17:18.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:18 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:18.000+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:18.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:18 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:18.000+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:18.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:18 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:18.000+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:18.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:17 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:17.841+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:18.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:17 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:17.841+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:18.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:17 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:17.841+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:18.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:18 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:18.396+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:18.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:18 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:18.396+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:18.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:18 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:18.396+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:18.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:18.388+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:18.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:18.388+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:18.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:18.388+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:19.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:19 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:19.024+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:19.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:19 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:19.024+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:19.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:19 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:19.024+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:19.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:18.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:19.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:18.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:19.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:18.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:19.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:18 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:18.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:19.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:18.868+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:19.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:18.868+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:19.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:18.868+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:19.332 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:18.868+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:19.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:19 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:19.408+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:19.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:19 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:19.408+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:19.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:19 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:19.408+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:19.789 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:19 smithi092 podman[109194]: 2024-09-21 02:17:19.513794546 +0000 UTC m=+5.613085569 container died 6b240f89d58c76975a6804fdb4b897a8a579a773a04e7d7600d2bc77f6a1928c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T02:17:19.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:19.396+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:19.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:19.396+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:19.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:19.396+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:20.055 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109194]: 2024-09-21 02:17:20.004491457 +0000 UTC m=+6.103782510 container cleanup 6b240f89d58c76975a6804fdb4b897a8a579a773a04e7d7600d2bc77f6a1928c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:17:20.055 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 bash[109194]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6 2024-09-21T02:17:20.055 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:19 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:19.918+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:20.055 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:19 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:19.918+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:20.055 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:19 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:19.918+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:20.055 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:19 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:19.918+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:20.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:19.871+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:20.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:19.871+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:20.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:19.871+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:20.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:19 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:19.871+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:20.460 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:20.068+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:20.460 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:20.068+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:20.460 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:20.068+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:20.460 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:20.068+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:20.554 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109210]: 2024-09-21 02:17:20.239014048 +0000 UTC m=+0.728311900 container remove 6b240f89d58c76975a6804fdb4b897a8a579a773a04e7d7600d2bc77f6a1928c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T02:17:20.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:20.459+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:20.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:20.459+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:20.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:20 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:20.459+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:20.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:20.405+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:20.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:20.405+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:20.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:20.405+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:20.834 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109299]: 2024-09-21 02:17:20.477392075 +0000 UTC m=+0.015629112 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:20.834 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109299]: 2024-09-21 02:17:20.577881042 +0000 UTC m=+0.116118079 container create 028653c36191b58447e9d733fb1931527c45955e29622f5c2c0a2a624ad5381a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:17:20.834 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109299]: 2024-09-21 02:17:20.834404842 +0000 UTC m=+0.372641883 container init 028653c36191b58447e9d733fb1931527c45955e29622f5c2c0a2a624ad5381a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:17:21.297 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109299]: 2024-09-21 02:17:20.837757595 +0000 UTC m=+0.375994635 container start 028653c36191b58447e9d733fb1931527c45955e29622f5c2c0a2a624ad5381a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T02:17:21.298 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:20 smithi092 podman[109299]: 2024-09-21 02:17:20.917685614 +0000 UTC m=+0.455922668 container attach 028653c36191b58447e9d733fb1931527c45955e29622f5c2c0a2a624ad5381a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:17:21.298 INFO:journalctl@ceph.osd.6.smithi092.stdout:Sep 21 02:17:21 smithi092 podman[109299]: 2024-09-21 02:17:21.023901358 +0000 UTC m=+0.562138396 container died 028653c36191b58447e9d733fb1931527c45955e29622f5c2c0a2a624ad5381a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T02:17:21.298 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:20 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:20.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:21.298 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:20 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:20.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:21.298 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:20 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:20.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:21.298 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:20 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:20.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:21.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:20.832+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:21.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:20.832+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:21.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:20.832+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:21.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:20 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:20.832+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:21.450 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:21.109+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:21.450 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:21.109+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:21.450 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:21.109+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:21.450 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:21.109+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:21.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T02:17:21.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:21.719 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T02:17:21.719 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T02:17:21.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.7 2024-09-21T02:17:21.746 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:21.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:21.746 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:21.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:21.746 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:21.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:21.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:21.449+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:21.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:21.449+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:21.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:21 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:21.449+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:22.019 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:21.860+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:22.019 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:21.860+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:22.019 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:21.860+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:22.019 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:21.860+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:22.019 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:21 smithi099 systemd[1]: Stopping Ceph osd.7 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:22.020 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:22.018+0000 7f4ca9d57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:22.020 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:22.018+0000 7f4ca9d57640 -1 osd.7 80 *** Got signal Terminated *** 2024-09-21T02:17:22.020 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:22.018+0000 7f4ca9d57640 -1 osd.7 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:22.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:21.828+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:22.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:21.828+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:22.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:21.828+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:22.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:21 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:21.828+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:22.444 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:22.136+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:22.445 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:22.136+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:22.445 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:22.136+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:22.445 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:22.136+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:22.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:22.444+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:22.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:22.444+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:22.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:22 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:22.444+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:22.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:22.367+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:22.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:22.367+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:22.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:22.367+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:23.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:22 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:22.845+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:23.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:22 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:22.845+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:23.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:22 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:22.845+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:23.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:22 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:22.845+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:23.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:22.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:23.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:22.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:23.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:22.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:23.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:22 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:22.853+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:23.480 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:23.107+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:23.480 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:23.107+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:23.480 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:23.107+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:23.480 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:23.107+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:23.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:23.479+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:23.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:23.479+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:23.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:23 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:23.479+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:23.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:23.403+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:23.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:23.403+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:23.831 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:23.403+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:24.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:23 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:23.844+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:24.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:23 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:23.844+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:24.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:23 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:23.844+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:24.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:23 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:23.844+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:24.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:23.830+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:24.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:23.830+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:24.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:23.830+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:24.331 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:23 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:23.830+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:24.504 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:24.135+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:24.504 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:24.135+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:24.504 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:24.135+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:24.504 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:24.135+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:24.782 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:24.399+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:24.782 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:24.399+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:24.782 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:24.399+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:24.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:24.503+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:24.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:24.503+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:24.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:24 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:24.503+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:25.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:24.782+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:25.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:24.782+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:25.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:24.782+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:25.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:24 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:24.782+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:25.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:24.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:25.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:24.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:25.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:24.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:25.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:24.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:25.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:24 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:24.869+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:25.498 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:25.111+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:25.499 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:25.111+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:25.499 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:25.111+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:25.499 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:25.111+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:25.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:25.497+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:25.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:25.497+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:25.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:25 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:25.497+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:25.794 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:25.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:25.795 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:25.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:25.795 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:25.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:25.795 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:25.417+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.296736+0000 front 2024-09-21T02:17:02.296794+0000 (oldest deadline 2024-09-21T02:17:24.596853+0000) 2024-09-21T02:17:26.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:25.791+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:26.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:25.791+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:26.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:25.791+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:26.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:25 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:25.791+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:26.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:25 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:25.898+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:26.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:25 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:25.898+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:26.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:25 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:25.898+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:26.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:25 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:25.898+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:26.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:25 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:25.898+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:26.495 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:26.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:26.495 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:26.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:26.495 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:26.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:26.495 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:26.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:26.747 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:26.427+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.093123+0000 front 2024-09-21T02:16:39.093182+0000 (oldest deadline 2024-09-21T02:17:02.593270+0000) 2024-09-21T02:17:26.747 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:26.427+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.294509+0000 front 2024-09-21T02:16:44.294475+0000 (oldest deadline 2024-09-21T02:17:10.194265+0000) 2024-09-21T02:17:26.748 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:26.427+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.395586+0000 front 2024-09-21T02:16:55.395567+0000 (oldest deadline 2024-09-21T02:17:17.695622+0000) 2024-09-21T02:17:26.748 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7[50844]: 2024-09-21T02:17:26.427+0000 7f4ca233a640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.296736+0000 front 2024-09-21T02:17:02.296794+0000 (oldest deadline 2024-09-21T02:17:24.596853+0000) 2024-09-21T02:17:26.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:26.494+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:26.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:26.494+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:26.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:26 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:26.494+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:27.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:26.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:27.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:26.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:27.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:26.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:27.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:26.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:27.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:26.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:27.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:26 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:26.908+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:27.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:26 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:26.908+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:27.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:26 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:26.908+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:27.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:26 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:26.908+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:27.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:26 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:26.908+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:27.522 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:27.110+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:27.522 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:27.110+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:27.522 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:27.110+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:27.522 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:27.110+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:27.522 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:27.110+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:27.669 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:27 smithi099 podman[67119]: 2024-09-21 02:17:27.404062749 +0000 UTC m=+5.493290804 container died cad197159864cd22b64c408a9063b64cbc1824c8195ee16530502165bfc7e53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:17:27.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:27.521+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:27.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:27.521+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:27.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:27.521+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:27.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:27 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:27.521+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:28.035 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:27 smithi099 podman[67119]: 2024-09-21 02:17:27.75929159 +0000 UTC m=+5.848519643 container cleanup cad197159864cd22b64c408a9063b64cbc1824c8195ee16530502165bfc7e53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T02:17:28.035 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:27 smithi099 bash[67119]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7 2024-09-21T02:17:28.038 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:27 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:27.744+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:28.038 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:27 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:27.744+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:28.038 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:27 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:27.744+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:28.038 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:27 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:27.744+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:28.038 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:27 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:27.744+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:28.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:27.944+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:28.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:27.944+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:28.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:27.944+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:28.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:27.944+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:28.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:27 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:27.944+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:28.331 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67134]: 2024-09-21 02:17:28.035495724 +0000 UTC m=+0.628857336 container remove cad197159864cd22b64c408a9063b64cbc1824c8195ee16530502165bfc7e53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T02:17:28.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:28.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:28.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:28.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:28.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:28.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:28.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:28.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:28.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:28.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:28.680 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67225]: 2024-09-21 02:17:28.271954316 +0000 UTC m=+0.015527180 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:28.680 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67225]: 2024-09-21 02:17:28.386348489 +0000 UTC m=+0.129921332 container create dea5b45735f25183475f2e215e1ece65789c7e8e2d4a1b14d4e4cb89de810db1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:17:28.681 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67225]: 2024-09-21 02:17:28.605528469 +0000 UTC m=+0.349101312 container init dea5b45735f25183475f2e215e1ece65789c7e8e2d4a1b14d4e4cb89de810db1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T02:17:28.681 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67225]: 2024-09-21 02:17:28.608704395 +0000 UTC m=+0.352277238 container start dea5b45735f25183475f2e215e1ece65789c7e8e2d4a1b14d4e4cb89de810db1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T02:17:28.681 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67225]: 2024-09-21 02:17:28.680564317 +0000 UTC m=+0.424137174 container attach dea5b45735f25183475f2e215e1ece65789c7e8e2d4a1b14d4e4cb89de810db1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T02:17:29.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:28.570+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:29.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:28.570+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:29.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:28.570+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:29.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:28 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:28.570+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:29.081 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 21 02:17:28 smithi099 podman[67225]: 2024-09-21 02:17:28.802179652 +0000 UTC m=+0.545752505 container died dea5b45735f25183475f2e215e1ece65789c7e8e2d4a1b14d4e4cb89de810db1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T02:17:29.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:28 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:28.740+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:29.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:28 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:28.740+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:29.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:28 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:28.740+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:29.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:28 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:28.740+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:29.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:28 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:28.740+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:29.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:28 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:28.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:29.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:28 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:28.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:29.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:28 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:28.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:29.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:28 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:28.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:29.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:28 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:28.899+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:29.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T02:17:29.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:29.460 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T02:17:29.460 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T02:17:29.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.8 2024-09-21T02:17:29.488 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:29.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:29.488 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:29.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:29.488 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:29.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:29.488 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:29.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:29.489 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:29.122+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:29.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 systemd[1]: Stopping Ceph osd.8 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:29.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.540+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:29.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.540+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:29.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.540+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:29.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.540+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:30.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.763+0000 7ff5d9705640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:30.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.763+0000 7ff5d9705640 -1 osd.8 80 *** Got signal Terminated *** 2024-09-21T02:17:30.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:29 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:29.763+0000 7ff5d9705640 -1 osd.8 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:30.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:29 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:29.706+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:30.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:29 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:29.706+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:30.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:29 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:29.706+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:30.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:29 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:29.706+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:30.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:29 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:29.706+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:30.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:29 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:29.934+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:30.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:29 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:29.934+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:30.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:29 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:29.934+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:30.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:29 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:29.934+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:30.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:29 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:29.934+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:30.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:30.090+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:30.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:30.090+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:30.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:30.090+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:30.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:30.090+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:30.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:30.090+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:31.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:30.569+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:31.034 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:30.569+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:31.034 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:30.569+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:31.034 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:30 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:30.569+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:31.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:30.681+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:31.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:30.681+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:31.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:30.681+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:31.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:30.681+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:31.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:30.681+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:31.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:30 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:30.978+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:31.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:30 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:30.978+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:31.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:30 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:30.978+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:31.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:30 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:30.978+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:31.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:30 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:30.978+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:31.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:31.054+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:31.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:31.054+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:31.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:31.054+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:31.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:31.054+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:31.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:31.054+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:32.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:31.545+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:32.033 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:31.545+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:32.034 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:31.545+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:32.034 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:31 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:31.545+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:32.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:31 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:31.969+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:32.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:31 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:31.969+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:32.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:31 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:31.969+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:32.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:31 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:31.969+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:32.055 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:31 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:31.969+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:32.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:31 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:31.657+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:32.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:31 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:31.657+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:32.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:31 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:31.657+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:32.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:31 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:31.657+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:32.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:31 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:31.657+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:32.510 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:32.091+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:32.511 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:32.091+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:32.511 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:32.091+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:32.511 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:32.091+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:32.511 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:32.091+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:32.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:32.509+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:32.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:32.509+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:32.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:32.509+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:32.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:32 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:32.509+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:33.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:32.667+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:33.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:32.667+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:33.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:32.667+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:33.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:32.667+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:33.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:32 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:32.667+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:33.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:32 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:32.936+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:33.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:32 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:32.936+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:33.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:32 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:32.936+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:33.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:32 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:32.936+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:33.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:32 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:32.936+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:33.492 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:33.081+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:33.492 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:33.081+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:33.492 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:33.081+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:33.493 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:33.081+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:33.493 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:33.081+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:33.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:33.491+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:33.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:33.491+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:33.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:33.491+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:33.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:33.491+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:33.983 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:33 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:33.688+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:33.983 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:33 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:33.688+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:33.983 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:33 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:33.688+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:33.983 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:33 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:33.688+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:33.983 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:33 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:33.688+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:34.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:33 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:33.947+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:34.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:33 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:33.947+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:34.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:33 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:33.947+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:34.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:33 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:33.947+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:34.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:33 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:33.947+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:34.484 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:34.058+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:34.485 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:34.058+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:34.485 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:34.058+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:34.485 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:34.058+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:34.485 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:34.058+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:34.485 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:34.058+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:34.783 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:34.484+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.262626+0000 front 2024-09-21T02:16:36.262345+0000 (oldest deadline 2024-09-21T02:17:00.962374+0000) 2024-09-21T02:17:34.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:34.484+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:46.664200+0000 front 2024-09-21T02:16:46.664326+0000 (oldest deadline 2024-09-21T02:17:12.564422+0000) 2024-09-21T02:17:34.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:34.484+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:52.565233+0000 front 2024-09-21T02:16:52.565189+0000 (oldest deadline 2024-09-21T02:17:17.264962+0000) 2024-09-21T02:17:34.784 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:34 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8[52302]: 2024-09-21T02:17:34.484+0000 7ff5d1ce8640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.266816+0000 front 2024-09-21T02:17:01.266727+0000 (oldest deadline 2024-09-21T02:17:26.566642+0000) 2024-09-21T02:17:35.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:34.723+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:35.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:34.723+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:35.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:34.723+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:35.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:34.723+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:35.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:34 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:34.723+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:35.283 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:35 smithi136 podman[67856]: 2024-09-21 02:17:35.13225589 +0000 UTC m=+5.479389273 container died b48df93016dcd5f9e35045eff92f7e88fd4563d14d6cec2621ebc3c1d8808efe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T02:17:35.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.011+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:35.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.011+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:35.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.011+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:35.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.011+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:35.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.011+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:35.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.011+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:35.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:34 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:34.971+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:35.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:34 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:34.971+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:35.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:34 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:34.971+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:35.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:34 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:34.971+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:35.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:34 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:34.971+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:35.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:34 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:34.971+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:35.583 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:35 smithi136 podman[67856]: 2024-09-21 02:17:35.478917071 +0000 UTC m=+5.826050443 container cleanup b48df93016dcd5f9e35045eff92f7e88fd4563d14d6cec2621ebc3c1d8808efe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T02:17:35.583 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:35 smithi136 bash[67856]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8 2024-09-21T02:17:35.998 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:35 smithi136 podman[67872]: 2024-09-21 02:17:35.688558511 +0000 UTC m=+0.548954343 container remove b48df93016dcd5f9e35045eff92f7e88fd4563d14d6cec2621ebc3c1d8808efe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T02:17:36.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:35 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:35.674+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:36.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:35 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:35.674+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:36.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:35 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:35.674+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:36.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:35 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:35.674+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:36.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:35 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:35.674+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:36.272 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:36 smithi136 podman[67966]: 2024-09-21 02:17:36.155956575 +0000 UTC m=+0.097957068 container create b239dfd29c67adf84ea975a41ccbc309e7ac0e97fb84f9b3a64d3346b5e2fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:17:36.272 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:36 smithi136 podman[67966]: 2024-09-21 02:17:36.074019801 +0000 UTC m=+0.016020307 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:36.272 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.996+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:36.273 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.997+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:36.273 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.997+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:36.273 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.997+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:36.273 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.997+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:36.273 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:35 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:35.997+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:36.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:35.992+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:36.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:35.992+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:36.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:35.992+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:36.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:35.992+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:36.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:35.992+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:36.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:35 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:35.992+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:36.533 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:36 smithi136 podman[67966]: 2024-09-21 02:17:36.374698807 +0000 UTC m=+0.316699306 container init b239dfd29c67adf84ea975a41ccbc309e7ac0e97fb84f9b3a64d3346b5e2fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:17:36.534 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:36 smithi136 podman[67966]: 2024-09-21 02:17:36.37836038 +0000 UTC m=+0.320360878 container start b239dfd29c67adf84ea975a41ccbc309e7ac0e97fb84f9b3a64d3346b5e2fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T02:17:36.534 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:36 smithi136 podman[67966]: 2024-09-21 02:17:36.4413963 +0000 UTC m=+0.383396804 container attach b239dfd29c67adf84ea975a41ccbc309e7ac0e97fb84f9b3a64d3346b5e2fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T02:17:36.869 INFO:journalctl@ceph.osd.8.smithi136.stdout:Sep 21 02:17:36 smithi136 podman[67966]: 2024-09-21 02:17:36.5770203 +0000 UTC m=+0.519020789 container died b239dfd29c67adf84ea975a41ccbc309e7ac0e97fb84f9b3a64d3346b5e2fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:17:37.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:36 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:36.627+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:37.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:36 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:36.627+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:37.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:36 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:36.627+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:37.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:36 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:36.627+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:37.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:36 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:36.627+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:37.178 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.001+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:37.178 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.001+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:37.178 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.001+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:37.178 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.001+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:37.178 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.001+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:37.178 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.001+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:37.203 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T02:17:37.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:37.284 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T02:17:37.284 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T02:17:37.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.9 2024-09-21T02:17:37.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.005+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:37.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.005+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:37.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.005+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:37.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.005+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:37.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.005+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:37.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.005+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:37.589 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 systemd[1]: Stopping Ceph osd.9 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:38.019 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.588+0000 7f1bb185a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:38.020 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.588+0000 7f1bb185a640 -1 osd.9 80 *** Got signal Terminated *** 2024-09-21T02:17:38.020 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:37 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:37.588+0000 7f1bb185a640 -1 osd.9 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:38.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:37 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:37.628+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:38.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:37 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:37.628+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:38.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:37 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:37.628+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:38.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:37 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:37.628+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:38.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:37 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:37.628+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:38.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:37 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:37.628+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:38.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.967+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:38.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.967+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:38.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.967+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:38.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.967+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:38.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.967+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:38.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:37.967+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:38.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.019+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:38.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.019+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:38.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.019+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:38.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.019+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:38.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.019+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:38.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.019+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:39.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:38 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:38.632+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:39.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:38 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:38.632+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:39.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:38 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:38.632+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:39.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:38 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:38.632+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:39.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:38 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:38.632+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:39.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:38 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:38.632+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:39.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:38 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:38.993+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:39.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:38 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:38.993+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:39.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:38 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:38.993+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:39.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:38 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:38.993+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:39.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:38 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:38.993+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:39.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:38 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:38.993+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:39.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.983+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:39.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.983+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:39.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.983+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:39.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.983+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:39.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.983+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:39.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:38 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:38.983+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:40.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:39 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:39.652+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:40.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:39 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:39.652+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:40.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:39 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:39.652+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:40.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:39 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:39.652+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:40.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:39 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:39.652+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:40.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:39 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:39.652+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:40.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.015+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:40.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.015+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:40.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.015+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:40.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.015+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:40.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.015+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:40.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.015+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:40.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:40.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:40.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:40.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:40.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:40.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:40.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:39 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:39.993+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:14.467510+0000 front 2024-09-21T02:17:14.467550+0000 (oldest deadline 2024-09-21T02:17:39.767565+0000) 2024-09-21T02:17:41.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:40 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:40.696+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:41.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:40 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:40.696+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:41.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:40 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:40.696+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:41.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:40 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:40.696+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:41.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:40 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:40.696+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:41.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:40 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:40.696+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:41.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.977+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:41.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.977+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:41.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.977+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:41.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.977+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:41.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.977+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:41.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:40 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:40.977+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:41.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:41.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:41.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:41.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:41.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:41.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:41.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:40 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:40.990+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:14.467510+0000 front 2024-09-21T02:17:14.467550+0000 (oldest deadline 2024-09-21T02:17:39.767565+0000) 2024-09-21T02:17:42.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:42.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:42.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:42.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:42.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:42.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:42.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:41 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:41.701+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:42.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.020+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:42.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.020+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:42.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.020+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:42.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.020+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:42.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.020+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:42.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.020+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:42.304 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:28.059622+0000 front 2024-09-21T02:16:28.059874+0000 (oldest deadline 2024-09-21T02:16:56.860287+0000) 2024-09-21T02:17:42.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:36.861067+0000 front 2024-09-21T02:16:36.860789+0000 (oldest deadline 2024-09-21T02:17:03.861341+0000) 2024-09-21T02:17:42.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.362471+0000 front 2024-09-21T02:16:44.362530+0000 (oldest deadline 2024-09-21T02:17:09.062431+0000) 2024-09-21T02:17:42.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:55.463708+0000 front 2024-09-21T02:16:55.463800+0000 (oldest deadline 2024-09-21T02:17:15.963774+0000) 2024-09-21T02:17:42.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:00.664767+0000 front 2024-09-21T02:17:00.664818+0000 (oldest deadline 2024-09-21T02:17:24.164689+0000) 2024-09-21T02:17:42.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:09.266779+0000 front 2024-09-21T02:17:09.266725+0000 (oldest deadline 2024-09-21T02:17:34.467125+0000) 2024-09-21T02:17:42.305 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:41 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9[66170]: 2024-09-21T02:17:41.976+0000 7f1ba9e3d640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:14.467510+0000 front 2024-09-21T02:17:14.467550+0000 (oldest deadline 2024-09-21T02:17:39.767565+0000) 2024-09-21T02:17:43.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:43.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:43.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:43.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:43.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:43.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:43.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:42 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:42.746+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:43.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.984+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:43.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.984+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:43.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.984+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:43.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.984+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:43.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.984+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:43.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:42 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:42.984+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:43.364 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:43 smithi092 podman[109415]: 2024-09-21 02:17:43.085846082 +0000 UTC m=+5.609407484 container died c72060b71898bed3a8d846dbb65d961ebaa2d8af56215b8c185ac72defe85cc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T02:17:43.798 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:43 smithi092 podman[109415]: 2024-09-21 02:17:43.538173627 +0000 UTC m=+6.061735020 container cleanup c72060b71898bed3a8d846dbb65d961ebaa2d8af56215b8c185ac72defe85cc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:17:43.798 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:43 smithi092 bash[109415]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9 2024-09-21T02:17:44.054 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:43 smithi092 podman[109431]: 2024-09-21 02:17:43.7976513 +0000 UTC m=+0.710852518 container remove c72060b71898bed3a8d846dbb65d961ebaa2d8af56215b8c185ac72defe85cc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T02:17:44.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:44.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:44.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:44.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:44.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:44.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:44.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:43 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:43.711+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:44.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:43 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:43.987+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:44.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:43 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:43.987+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:44.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:43 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:43.987+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:44.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:43 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:43.987+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:44.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:43 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:43.987+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:44.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:43 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:43.987+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:44.442 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:44 smithi092 podman[109519]: 2024-09-21 02:17:44.049314059 +0000 UTC m=+0.016493261 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:44.443 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:44 smithi092 podman[109519]: 2024-09-21 02:17:44.148495433 +0000 UTC m=+0.115674622 container create 2266d608d3a1278d6de71e50a7904e8e52bab435602b4f72ad55a4653f792422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T02:17:44.443 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:44 smithi092 podman[109519]: 2024-09-21 02:17:44.359078098 +0000 UTC m=+0.326257298 container init 2266d608d3a1278d6de71e50a7904e8e52bab435602b4f72ad55a4653f792422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T02:17:44.443 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:44 smithi092 podman[109519]: 2024-09-21 02:17:44.362330151 +0000 UTC m=+0.329509340 container start 2266d608d3a1278d6de71e50a7904e8e52bab435602b4f72ad55a4653f792422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:17:44.443 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:44 smithi092 podman[109519]: 2024-09-21 02:17:44.442403321 +0000 UTC m=+0.409582520 container attach 2266d608d3a1278d6de71e50a7904e8e52bab435602b4f72ad55a4653f792422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:17:44.804 INFO:journalctl@ceph.osd.9.smithi092.stdout:Sep 21 02:17:44 smithi092 podman[109519]: 2024-09-21 02:17:44.546339481 +0000 UTC m=+0.513518684 container died 2266d608d3a1278d6de71e50a7904e8e52bab435602b4f72ad55a4653f792422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T02:17:45.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:45.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:45.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:45.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:45.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:45.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:45.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:44 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:44.732+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:45.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T02:17:45.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:45.243 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T02:17:45.243 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T02:17:45.243 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.10 2024-09-21T02:17:45.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:45.009+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:45.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:45.009+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:45.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:45.009+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:45.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:45.009+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:45.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:45.009+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:45.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:45.009+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:45.536 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 systemd[1]: Stopping Ceph osd.10 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.535+0000 7f1063411640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.535+0000 7f1063411640 -1 osd.10 80 *** Got signal Terminated *** 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.535+0000 7f1063411640 -1 osd.10 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:45.832 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:45 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:45.769+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:46.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:46 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:46.040+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:46.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:46 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:46.040+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:46.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:46 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:46.040+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:46.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:46 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:46.040+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:46.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:46 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:46.040+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:46.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:46 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:46.040+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:47.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:47.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:47.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:47.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:47.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:47.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:47.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:46 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:46.736+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:47.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:47 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:47.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:47.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:47 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:47.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:47.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:47 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:47.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:47.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:47 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:47.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:47.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:47 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:47.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:47.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:47 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:47.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:48.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:48.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:48.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:48.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:48.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:48.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:48.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:47 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:47.724+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:48.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:48 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:48.025+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:48.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:48 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:48.025+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:48.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:48 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:48.025+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:48.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:48 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:48.025+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:48.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:48 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:48.025+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:48.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:48 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:48.025+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:49.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.756+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:49.081 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.756+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:49.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.756+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:49.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.757+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:49.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.757+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:49.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.757+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:49.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:48 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:48.757+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:49.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:49.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:49.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:49.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:49.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:49.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:49.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:49 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:49.038+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.144239+0000 front 2024-09-21T02:16:29.144057+0000 (oldest deadline 2024-09-21T02:16:53.844039+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.645513+0000 front 2024-09-21T02:16:39.645356+0000 (oldest deadline 2024-09-21T02:17:04.945505+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:44.946022+0000 front 2024-09-21T02:16:44.946123+0000 (oldest deadline 2024-09-21T02:17:08.446059+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.247514+0000 front 2024-09-21T02:16:51.247498+0000 (oldest deadline 2024-09-21T02:17:18.247806+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:02.948582+0000 front 2024-09-21T02:17:02.948526+0000 (oldest deadline 2024-09-21T02:17:25.848615+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.149792+0000 front 2024-09-21T02:17:08.149829+0000 (oldest deadline 2024-09-21T02:17:36.950031+0000) 2024-09-21T02:17:50.082 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:49 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10[57538]: 2024-09-21T02:17:49.776+0000 7f105b9f4640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-09-21T02:17:16.950840+0000 front 2024-09-21T02:17:16.950959+0000 (oldest deadline 2024-09-21T02:17:41.650709+0000) 2024-09-21T02:17:50.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:50.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:50.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:50.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:50.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:50.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:50.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:50 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:50.007+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:51.212 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:50 smithi099 podman[67340]: 2024-09-21 02:17:50.952412935 +0000 UTC m=+5.516306594 container died 2df28c490ca655786246411bb1c20e4a4c4fbeac0d202e1c04b0298f1af721b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:17:51.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:51.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:51.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:51.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:51.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:51.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:51.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:51 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:51.026+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:51.478 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:51 smithi099 podman[67340]: 2024-09-21 02:17:51.293306493 +0000 UTC m=+5.857200162 container cleanup 2df28c490ca655786246411bb1c20e4a4c4fbeac0d202e1c04b0298f1af721b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T02:17:51.478 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:51 smithi099 bash[67340]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10 2024-09-21T02:17:51.831 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:51 smithi099 podman[67354]: 2024-09-21 02:17:51.477633722 +0000 UTC m=+0.518903387 container remove 2df28c490ca655786246411bb1c20e4a4c4fbeac0d202e1c04b0298f1af721b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T02:17:52.214 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:51 smithi099 podman[67445]: 2024-09-21 02:17:51.805530995 +0000 UTC m=+0.015349280 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:52.214 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:51 smithi099 podman[67445]: 2024-09-21 02:17:51.919995313 +0000 UTC m=+0.129813593 container create 933f8f1659c57fa0f481ed721686a552e01c82a57686384906994b35f6242951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T02:17:52.214 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:52 smithi099 podman[67445]: 2024-09-21 02:17:52.138937107 +0000 UTC m=+0.348755386 container init 933f8f1659c57fa0f481ed721686a552e01c82a57686384906994b35f6242951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:17:52.214 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:52 smithi099 podman[67445]: 2024-09-21 02:17:52.142145916 +0000 UTC m=+0.351964194 container start 933f8f1659c57fa0f481ed721686a552e01c82a57686384906994b35f6242951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T02:17:52.214 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:52 smithi099 podman[67445]: 2024-09-21 02:17:52.213927791 +0000 UTC m=+0.423746083 container attach 933f8f1659c57fa0f481ed721686a552e01c82a57686384906994b35f6242951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T02:17:52.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:52.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:52.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:52.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:52.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:52.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:52.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:52.070+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:52.581 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:52 smithi099 podman[67445]: 2024-09-21 02:17:52.333654239 +0000 UTC m=+0.543472534 container died 933f8f1659c57fa0f481ed721686a552e01c82a57686384906994b35f6242951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T02:17:52.884 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 21 02:17:52 smithi099 podman[67469]: 2024-09-21 02:17:52.867491182 +0000 UTC m=+0.523697727 container remove 933f8f1659c57fa0f481ed721686a552e01c82a57686384906994b35f6242951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T02:17:52.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T02:17:52.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:17:52.929 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T02:17:52.929 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T02:17:52.929 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.11 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:52 smithi136 systemd[1]: Stopping Ceph osd.11 for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:53.238 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.059+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:53.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.236+0000 7f9afbc4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:17:53.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.236+0000 7f9afbc4b640 -1 osd.11 80 *** Got signal Terminated *** 2024-09-21T02:17:53.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:53 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:53.236+0000 7f9afbc4b640 -1 osd.11 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T02:17:54.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:54.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:54.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:54.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:54.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:54.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:54.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:54 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:54.017+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:55.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:55.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:55.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:55.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:55.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:55.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:55.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:55 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:55.052+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:56.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:56.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:56.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:56.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:56.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:56.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:56.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:56.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:56 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:56.101+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-09-21T02:17:33.463336+0000 front 2024-09-21T02:17:33.463359+0000 (oldest deadline 2024-09-21T02:17:55.163393+0000) 2024-09-21T02:17:57.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.137+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.137+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.137+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.138+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.138+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.138+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.138+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:57.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:57 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:57.138+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-09-21T02:17:33.463336+0000 front 2024-09-21T02:17:33.463359+0000 (oldest deadline 2024-09-21T02:17:55.163393+0000) 2024-09-21T02:17:58.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-09-21T02:16:29.353687+0000 front 2024-09-21T02:16:29.353461+0000 (oldest deadline 2024-09-21T02:16:55.154129+0000) 2024-09-21T02:17:58.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-21T02:16:39.855157+0000 front 2024-09-21T02:16:39.855128+0000 (oldest deadline 2024-09-21T02:17:05.755109+0000) 2024-09-21T02:17:58.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-09-21T02:16:45.755512+0000 front 2024-09-21T02:16:45.755656+0000 (oldest deadline 2024-09-21T02:17:09.855681+0000) 2024-09-21T02:17:58.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-09-21T02:16:51.556700+0000 front 2024-09-21T02:16:51.556738+0000 (oldest deadline 2024-09-21T02:17:19.157338+0000) 2024-09-21T02:17:58.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-21T02:17:01.959010+0000 front 2024-09-21T02:17:01.958974+0000 (oldest deadline 2024-09-21T02:17:26.658903+0000) 2024-09-21T02:17:58.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-09-21T02:17:08.359909+0000 front 2024-09-21T02:17:08.360037+0000 (oldest deadline 2024-09-21T02:17:33.659969+0000) 2024-09-21T02:17:58.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-21T02:17:26.561890+0000 front 2024-09-21T02:17:26.561952+0000 (oldest deadline 2024-09-21T02:17:48.762355+0000) 2024-09-21T02:17:58.534 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11[58702]: 2024-09-21T02:17:58.180+0000 7f9af8236640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-09-21T02:17:33.463336+0000 front 2024-09-21T02:17:33.463359+0000 (oldest deadline 2024-09-21T02:17:55.163393+0000) 2024-09-21T02:17:58.945 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:58 smithi136 podman[68073]: 2024-09-21 02:17:58.686293716 +0000 UTC m=+5.562724779 container died 2015d2b33f89cca5c8b8bdd94ec4ad44c10ba2d06c7be522b20fae2fa7257dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:17:59.279 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68073]: 2024-09-21 02:17:59.035956685 +0000 UTC m=+5.912387751 container cleanup 2015d2b33f89cca5c8b8bdd94ec4ad44c10ba2d06c7be522b20fae2fa7257dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T02:17:59.279 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 bash[68073]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11 2024-09-21T02:17:59.533 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68089]: 2024-09-21 02:17:59.278705545 +0000 UTC m=+0.585118492 container remove 2015d2b33f89cca5c8b8bdd94ec4ad44c10ba2d06c7be522b20fae2fa7257dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T02:17:59.874 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68175]: 2024-09-21 02:17:59.497440346 +0000 UTC m=+0.015709872 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:17:59.874 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68175]: 2024-09-21 02:17:59.60469719 +0000 UTC m=+0.122966712 container create 52cda5df1281b34a9697db4cc11d3e82e2ee704096f3899a64b77aaf842f47d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-21T02:17:59.874 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68175]: 2024-09-21 02:17:59.873514602 +0000 UTC m=+0.391784133 container init 52cda5df1281b34a9697db4cc11d3e82e2ee704096f3899a64b77aaf842f47d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:18:00.283 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68175]: 2024-09-21 02:17:59.876903547 +0000 UTC m=+0.395173070 container start 52cda5df1281b34a9697db4cc11d3e82e2ee704096f3899a64b77aaf842f47d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:18:00.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:17:59 smithi136 podman[68175]: 2024-09-21 02:17:59.956657462 +0000 UTC m=+0.474926993 container attach 52cda5df1281b34a9697db4cc11d3e82e2ee704096f3899a64b77aaf842f47d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T02:18:00.284 INFO:journalctl@ceph.osd.11.smithi136.stdout:Sep 21 02:18:00 smithi136 podman[68175]: 2024-09-21 02:18:00.076663196 +0000 UTC m=+0.594932720 container died 52cda5df1281b34a9697db4cc11d3e82e2ee704096f3899a64b77aaf842f47d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T02:18:00.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T02:18:00.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:00.735 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T02:18:00.735 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T02:18:00.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.a 2024-09-21T02:18:01.054 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:00 smithi092 systemd[1]: Stopping Ceph mds.a for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:01.554 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:01 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a[72610]: 2024-09-21T02:18:01.271+0000 7f738dca8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:01.554 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:01 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a[72610]: 2024-09-21T02:18:01.271+0000 7f738dca8640 -1 mds.a *** got signal Terminated *** 2024-09-21T02:18:05.055 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:04 smithi092 podman[109622]: 2024-09-21 02:18:04.761977907 +0000 UTC m=+3.833100843 container died acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:18:05.514 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:05 smithi092 podman[109622]: 2024-09-21 02:18:05.287736106 +0000 UTC m=+4.358859038 container cleanup acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T02:18:05.514 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:05 smithi092 bash[109622]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a 2024-09-21T02:18:05.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T02:18:05.785 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:05 smithi092 podman[109646]: 2024-09-21 02:18:05.514004038 +0000 UTC m=+0.745856748 container remove acd43fff26f051cda6cd91247f976c7d8cd6cad7572620ba6e8c5d9c675320e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:18:05.785 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:05 smithi092 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.a.service: Deactivated successfully. 2024-09-21T02:18:05.785 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:05 smithi092 systemd[1]: Stopped Ceph mds.a for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:18:05.785 INFO:journalctl@ceph.mds.a.smithi092.stdout:Sep 21 02:18:05 smithi092 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.a.service: Consumed 2.008s CPU time. 2024-09-21T02:18:05.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:05.799 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T02:18:05.799 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T02:18:05.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.d 2024-09-21T02:18:06.054 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:18:05 smithi092 systemd[1]: Stopping Ceph mds.d for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:06.554 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:18:06 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-d[71586]: 2024-09-21T02:18:06.141+0000 7f3246d6c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:06.554 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:18:06 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-d[71586]: 2024-09-21T02:18:06.141+0000 7f3246d6c640 -1 mds.d *** got signal Terminated *** 2024-09-21T02:18:09.075 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:18:08 smithi092 podman[109758]: 2024-09-21 02:18:08.805408791 +0000 UTC m=+2.787663524 container died 20a3f8a231edcb66224de7509a2a57cc18a24e8ac2bdf2af2ebe58bdccb7d01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T02:18:09.468 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:18:09 smithi092 podman[109758]: 2024-09-21 02:18:09.174784667 +0000 UTC m=+3.157039399 container cleanup 20a3f8a231edcb66224de7509a2a57cc18a24e8ac2bdf2af2ebe58bdccb7d01a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T02:18:09.468 INFO:journalctl@ceph.mds.d.smithi092.stdout:Sep 21 02:18:09 smithi092 bash[109758]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-d 2024-09-21T02:18:09.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T02:18:09.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:09.646 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T02:18:09.646 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T02:18:09.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.g 2024-09-21T02:18:10.054 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:18:09 smithi092 systemd[1]: Stopping Ceph mds.g for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:10.054 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:18:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g[72323]: 2024-09-21T02:18:09.970+0000 7fa7558fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:10.055 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:18:09 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g[72323]: 2024-09-21T02:18:09.970+0000 7fa7558fc640 -1 mds.g *** got signal Terminated *** 2024-09-21T02:18:13.020 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:18:12 smithi092 podman[109877]: 2024-09-21 02:18:12.764926174 +0000 UTC m=+2.897253179 container died 982f38fa7da1bbf414408ad56c2c629ef1c83188064177ff04117a7ade49b29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T02:18:13.304 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:18:13 smithi092 podman[109877]: 2024-09-21 02:18:13.119966209 +0000 UTC m=+3.252293221 container cleanup 982f38fa7da1bbf414408ad56c2c629ef1c83188064177ff04117a7ade49b29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T02:18:13.304 INFO:journalctl@ceph.mds.g.smithi092.stdout:Sep 21 02:18:13 smithi092 bash[109877]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-g 2024-09-21T02:18:13.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T02:18:13.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:13.587 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T02:18:13.587 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T02:18:13.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.j 2024-09-21T02:18:14.055 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:18:13 smithi092 systemd[1]: Stopping Ceph mds.j for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:14.055 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:18:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j[71208]: 2024-09-21T02:18:13.915+0000 7f1ce2f9b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:14.055 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:18:13 smithi092 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j[71208]: 2024-09-21T02:18:13.915+0000 7f1ce2f9b640 -1 mds.j *** got signal Terminated *** 2024-09-21T02:18:17.649 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:18:17 smithi092 podman[109998]: 2024-09-21 02:18:17.377842105 +0000 UTC m=+3.573941040 container died c5d7e7469ad55876169bfa297c431ff19df388a1ea186b55235b5f2e9213c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-21T02:18:17.991 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:18:17 smithi092 podman[109998]: 2024-09-21 02:18:17.748453058 +0000 UTC m=+3.944551982 container cleanup c5d7e7469ad55876169bfa297c431ff19df388a1ea186b55235b5f2e9213c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:18:17.991 INFO:journalctl@ceph.mds.j.smithi092.stdout:Sep 21 02:18:17 smithi092 bash[109998]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-j 2024-09-21T02:18:18.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T02:18:18.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:18.144 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T02:18:18.144 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T02:18:18.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.b 2024-09-21T02:18:18.442 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:18:18 smithi099 systemd[1]: Stopping Ceph mds.b for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:18.695 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:18:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b[62418]: 2024-09-21T02:18:18.440+0000 7f7511952640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:18.695 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:18:18 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b[62418]: 2024-09-21T02:18:18.440+0000 7f7511952640 -1 mds.b *** got signal Terminated *** 2024-09-21T02:18:21.025 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:18:20 smithi099 podman[67547]: 2024-09-21 02:18:20.766482081 +0000 UTC m=+2.426976183 container died 4186d3a6afba9505a296c13dc3b465db0f7f6e633e5178bc6ee4e4ec6bc959e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:18:21.308 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:18:21 smithi099 podman[67547]: 2024-09-21 02:18:21.099000521 +0000 UTC m=+2.759494622 container cleanup 4186d3a6afba9505a296c13dc3b465db0f7f6e633e5178bc6ee4e4ec6bc959e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T02:18:21.309 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 21 02:18:21 smithi099 bash[67547]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-b 2024-09-21T02:18:21.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T02:18:21.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:21.559 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T02:18:21.559 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T02:18:21.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.e 2024-09-21T02:18:21.879 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:18:21 smithi099 systemd[1]: Stopping Ceph mds.e for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:22.331 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:18:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e[61873]: 2024-09-21T02:18:21.877+0000 7f9d46d56640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:22.331 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:18:21 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e[61873]: 2024-09-21T02:18:21.877+0000 7f9d46d56640 -1 mds.e *** got signal Terminated *** 2024-09-21T02:18:25.081 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:18:24 smithi099 podman[67681]: 2024-09-21 02:18:24.766079025 +0000 UTC m=+2.991976107 container died 1a54c297d594919802f95dad6e470cf9fdd27ced5931c5cf9f849309564e6bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T02:18:25.581 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:18:25 smithi099 podman[67681]: 2024-09-21 02:18:25.319361648 +0000 UTC m=+3.545258726 container cleanup 1a54c297d594919802f95dad6e470cf9fdd27ced5931c5cf9f849309564e6bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T02:18:25.581 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 21 02:18:25 smithi099 bash[67681]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-e 2024-09-21T02:18:25.760 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T02:18:25.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:25.794 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T02:18:25.794 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T02:18:25.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.h 2024-09-21T02:18:26.140 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:25 smithi099 systemd[1]: Stopping Ceph mds.h for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:26.581 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h[62143]: 2024-09-21T02:18:26.139+0000 7fdb6a033640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:26.581 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:26 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h[62143]: 2024-09-21T02:18:26.139+0000 7fdb6a033640 -1 mds.h *** got signal Terminated *** 2024-09-21T02:18:29.081 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:28 smithi099 podman[67800]: 2024-09-21 02:18:28.771857556 +0000 UTC m=+2.765501428 container died 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T02:18:29.424 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:29 smithi099 podman[67800]: 2024-09-21 02:18:29.197562524 +0000 UTC m=+3.191206389 container cleanup 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T02:18:29.424 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:29 smithi099 bash[67800]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h 2024-09-21T02:18:29.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T02:18:29.693 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:29 smithi099 podman[67813]: 2024-09-21 02:18:29.423782188 +0000 UTC m=+0.648069748 container remove 63964c222653c6a0a7e2ecf92fa1eb9b628e49ff40152950995850c68f5d39d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-h, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T02:18:29.693 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:29 smithi099 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.h.service: Deactivated successfully. 2024-09-21T02:18:29.693 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:29 smithi099 systemd[1]: Stopped Ceph mds.h for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:18:29.693 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 21 02:18:29 smithi099 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.h.service: Consumed 3.064s CPU time. 2024-09-21T02:18:29.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:29.705 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T02:18:29.705 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T02:18:29.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.k 2024-09-21T02:18:30.018 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:18:29 smithi099 systemd[1]: Stopping Ceph mds.k for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:30.331 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:18:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k[61591]: 2024-09-21T02:18:30.017+0000 7facef121640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:30.331 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:18:30 smithi099 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k[61591]: 2024-09-21T02:18:30.017+0000 7facef121640 -1 mds.k *** got signal Terminated *** 2024-09-21T02:18:33.018 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:18:32 smithi099 podman[67923]: 2024-09-21 02:18:32.764175511 +0000 UTC m=+2.846146488 container died 424c9ed6011cab1f5e26f3fae76d7ad05031152753846ed0c68a9ffc2afd435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-21T02:18:33.331 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:18:33 smithi099 podman[67923]: 2024-09-21 02:18:33.117946694 +0000 UTC m=+3.199917706 container cleanup 424c9ed6011cab1f5e26f3fae76d7ad05031152753846ed0c68a9ffc2afd435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T02:18:33.331 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 21 02:18:33 smithi099 bash[67923]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-k 2024-09-21T02:18:33.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T02:18:33.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:33.571 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T02:18:33.571 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T02:18:33.571 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.c 2024-09-21T02:18:33.957 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:18:33 smithi136 systemd[1]: Stopping Ceph mds.c for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:34.219 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:18:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c[63501]: 2024-09-21T02:18:33.954+0000 7f63fd575640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:34.219 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:18:33 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c[63501]: 2024-09-21T02:18:33.954+0000 7f63fd575640 -1 mds.c *** got signal Terminated *** 2024-09-21T02:18:37.033 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:18:36 smithi136 podman[68276]: 2024-09-21 02:18:36.774846213 +0000 UTC m=+3.007759221 container died 66597b4ff0b7d9a1c5897e0fb601879fb866b45d460b4c999ce5c8c65be0df22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T02:18:37.398 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:18:37 smithi136 podman[68276]: 2024-09-21 02:18:37.18078876 +0000 UTC m=+3.413701770 container cleanup 66597b4ff0b7d9a1c5897e0fb601879fb866b45d460b4c999ce5c8c65be0df22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:18:37.398 INFO:journalctl@ceph.mds.c.smithi136.stdout:Sep 21 02:18:37 smithi136 bash[68276]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-c 2024-09-21T02:18:37.539 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T02:18:37.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:37.575 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T02:18:37.575 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T02:18:37.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.f 2024-09-21T02:18:38.033 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:18:37 smithi136 systemd[1]: Stopping Ceph mds.f for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:38.033 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:18:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f[63231]: 2024-09-21T02:18:37.908+0000 7fb3a8d18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:38.033 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:18:37 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f[63231]: 2024-09-21T02:18:37.908+0000 7fb3a8d18640 -1 mds.f *** got signal Terminated *** 2024-09-21T02:18:41.033 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:18:40 smithi136 podman[68410]: 2024-09-21 02:18:40.75771832 +0000 UTC m=+2.957661483 container died b140a19dfe02b54a180391ba0c791831036712163bd16d633741095874f8d837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T02:18:41.359 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:18:41 smithi136 podman[68410]: 2024-09-21 02:18:41.150153111 +0000 UTC m=+3.350096273 container cleanup b140a19dfe02b54a180391ba0c791831036712163bd16d633741095874f8d837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:18:41.360 INFO:journalctl@ceph.mds.f.smithi136.stdout:Sep 21 02:18:41 smithi136 bash[68410]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-f 2024-09-21T02:18:41.516 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T02:18:41.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:41.549 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T02:18:41.549 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T02:18:41.549 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.i 2024-09-21T02:18:41.871 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:41 smithi136 systemd[1]: Stopping Ceph mds.i for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:42.283 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:41 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: 2024-09-21T02:18:41.870+0000 7fd415037640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:42.283 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:41 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i[63767]: 2024-09-21T02:18:41.870+0000 7fd415037640 -1 mds.i *** got signal Terminated *** 2024-09-21T02:18:45.033 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:44 smithi136 podman[68532]: 2024-09-21 02:18:44.769155002 +0000 UTC m=+2.997540573 container died 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T02:18:45.388 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:45 smithi136 podman[68532]: 2024-09-21 02:18:45.213264039 +0000 UTC m=+3.441649600 container cleanup 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T02:18:45.388 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:45 smithi136 bash[68532]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i 2024-09-21T02:18:45.628 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T02:18:45.649 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:45 smithi136 podman[68544]: 2024-09-21 02:18:45.388136353 +0000 UTC m=+0.608572463 container remove 8e7935b628a0074bb1400cef25bd3648a5277f6aba81566f932e1f9e908586ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T02:18:45.650 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:45 smithi136 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.i.service: Deactivated successfully. 2024-09-21T02:18:45.650 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:45 smithi136 systemd[1]: Stopped Ceph mds.i for 49513f9c-77bc-11ef-baf5-efdc52797490. 2024-09-21T02:18:45.650 INFO:journalctl@ceph.mds.i.smithi136.stdout:Sep 21 02:18:45 smithi136 systemd[1]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.i.service: Consumed 9.153s CPU time. 2024-09-21T02:18:45.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:45.660 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T02:18:45.660 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T02:18:45.661 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.l 2024-09-21T02:18:45.966 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:18:45 smithi136 systemd[1]: Stopping Ceph mds.l for 49513f9c-77bc-11ef-baf5-efdc52797490... 2024-09-21T02:18:46.283 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:18:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l[62961]: 2024-09-21T02:18:45.965+0000 7ffbeb126640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:18:46.283 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:18:45 smithi136 ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l[62961]: 2024-09-21T02:18:45.965+0000 7ffbeb126640 -1 mds.l *** got signal Terminated *** 2024-09-21T02:18:49.033 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:18:48 smithi136 podman[68651]: 2024-09-21 02:18:48.762991923 +0000 UTC m=+2.891450021 container died cdcff13c35e561a507584f1a750a24e1db8c8df82f0dd4a6efcd1b7808e03da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T02:18:49.457 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:18:49 smithi136 podman[68651]: 2024-09-21 02:18:49.139729605 +0000 UTC m=+3.268187682 container cleanup cdcff13c35e561a507584f1a750a24e1db8c8df82f0dd4a6efcd1b7808e03da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T02:18:49.457 INFO:journalctl@ceph.mds.l.smithi136.stdout:Sep 21 02:18:49 smithi136 bash[68651]: ceph-49513f9c-77bc-11ef-baf5-efdc52797490-mds-l 2024-09-21T02:18:49.604 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-49513f9c-77bc-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T02:18:49.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T02:18:49.635 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T02:18:49.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T02:18:49.836 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:18:52.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T02:18:52.576 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:18:55.014 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T02:18:55.215 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:18:57.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:18:57.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:18:57.643 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:18:57.670 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T02:18:57.672 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi092/crash 2024-09-21T02:18:57.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T02:18:57.708 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T02:18:57.709 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-09-21T02:18:57.711 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi099/crash 2024-09-21T02:18:57.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T02:18:57.765 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T02:18:57.766 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-09-21T02:18:57.767 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi136/crash 2024-09-21T02:18:57.768 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T02:18:57.806 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T02:18:57.807 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-09-21T02:18:57.808 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T02:18:57.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T02:18:57.847 INFO:tasks.cephadm:Compressing logs... 2024-09-21T02:18:57.847 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:18:57.890 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:18:57.892 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:18:57.915 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T02:18:57.916 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-09-21T02:18:57.916 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T02:18:57.916 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T02:18:57.916 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T02:18:57.921 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T02:18:57.921 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T02:18:57.921 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T02:18:57.922 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T02:18:57.922 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.a.log: 82.7% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T02:18:57.922 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T02:18:57.922 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.g.log: 90.2% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T02:18:57.922 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T02:18:57.922 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr: 82.6% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T02:18:57.923 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T02:18:57.924 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.5.log: 90.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T02:18:57.925 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log: 87.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T02:18:57.928 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T02:18:57.928 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2024-09-21T02:18:57.928 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T02:18:57.928 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T02:18:57.928 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T02:18:57.929 INFO:teuthology.orchestra.run.smithi136.stderr: 90.9% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T02:18:57.931 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T02:18:57.932 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T02:18:57.932 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T02:18:57.932 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T02:18:57.935 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T02:18:57.935 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T02:18:57.935 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T02:18:57.935 INFO:teuthology.orchestra.run.smithi099.stderr: 92.4%gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T02:18:57.935 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T02:18:57.935 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T02:18:57.937 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.8.log: 87.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T02:18:57.937 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T02:18:57.938 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log: /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.h.log: 87.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T02:18:57.938 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T02:18:57.940 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.x.log: 95.0% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T02:18:57.943 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.z.log: 92.6% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T02:18:57.943 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T02:18:57.947 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T02:18:57.948 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T02:18:57.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.j.log: 95.1% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T02:18:57.955 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T02:18:57.960 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T02:18:57.970 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T02:18:57.979 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T02:18:57.988 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T02:18:58.003 INFO:teuthology.orchestra.run.smithi099.stderr: 93.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T02:18:58.007 INFO:teuthology.orchestra.run.smithi092.stderr: 92.8% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T02:18:58.009 INFO:teuthology.orchestra.run.smithi092.stderr: 94.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T02:18:58.009 INFO:teuthology.orchestra.run.smithi099.stderr: 94.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T02:18:58.014 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T02:18:58.020 INFO:teuthology.orchestra.run.smithi136.stderr: 94.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T02:18:58.057 INFO:teuthology.orchestra.run.smithi136.stderr: 93.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T02:18:58.060 INFO:teuthology.orchestra.run.smithi099.stderr: 94.3% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T02:18:58.151 INFO:teuthology.orchestra.run.smithi092.stderr: 90.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T02:18:58.245 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T02:18:58.275 INFO:teuthology.orchestra.run.smithi099.stderr: 91.8% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T02:18:58.345 INFO:teuthology.orchestra.run.smithi136.stderr: 91.8% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T02:18:58.467 INFO:teuthology.orchestra.run.smithi092.stderr: 92.2% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T02:18:58.482 INFO:teuthology.orchestra.run.smithi092.stderr: 92.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T02:18:58.485 INFO:teuthology.orchestra.run.smithi136.stderr: 92.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T02:18:58.488 INFO:teuthology.orchestra.run.smithi099.stderr: 92.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T02:18:58.497 INFO:teuthology.orchestra.run.smithi136.stderr: 92.7% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T02:18:58.509 INFO:teuthology.orchestra.run.smithi092.stderr: 92.7% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T02:18:58.618 INFO:teuthology.orchestra.run.smithi099.stderr: 92.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T02:18:58.623 INFO:teuthology.orchestra.run.smithi099.stderr: 92.7% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T02:18:58.625 INFO:teuthology.orchestra.run.smithi136.stderr: 92.4% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T02:18:58.634 INFO:teuthology.orchestra.run.smithi092.stderr: 92.6% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T02:18:58.849 INFO:teuthology.orchestra.run.smithi092.stderr: 90.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T02:18:58.850 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-21T02:18:58.850 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m0.947s 2024-09-21T02:18:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m3.619s 2024-09-21T02:18:58.851 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.151s 2024-09-21T02:18:58.906 INFO:teuthology.orchestra.run.smithi099.stderr: 93.5% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T02:18:58.908 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-21T02:18:58.908 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m1.001s 2024-09-21T02:18:58.908 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m3.726s 2024-09-21T02:18:58.908 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.166s 2024-09-21T02:18:59.168 INFO:teuthology.orchestra.run.smithi136.stderr: 93.2% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T02:18:59.748 INFO:teuthology.orchestra.run.smithi136.stderr: 92.7% -- replaced with /var/log/ceph/49513f9c-77bc-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T02:18:59.750 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-21T02:18:59.750 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m1.843s 2024-09-21T02:18:59.750 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m5.391s 2024-09-21T02:18:59.750 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.199s 2024-09-21T02:18:59.751 INFO:tasks.cephadm:Archiving logs... 2024-09-21T02:18:59.751 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi092/log 2024-09-21T02:18:59.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:19:00.269 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi099/log 2024-09-21T02:19:00.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:19:00.779 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi136/log 2024-09-21T02:19:00.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:19:01.536 INFO:tasks.cephadm:Removing cluster... 2024-09-21T02:19:01.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --force 2024-09-21T02:19:01.731 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:19:01.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --force 2024-09-21T02:19:02.192 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:19:02.449 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 49513f9c-77bc-11ef-baf5-efdc52797490 --force 2024-09-21T02:19:02.649 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 49513f9c-77bc-11ef-baf5-efdc52797490 2024-09-21T02:19:02.908 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T02:19:02.908 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T02:19:02.926 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T02:19:02.942 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T02:19:02.957 INFO:tasks.cephadm:Teardown complete 2024-09-21T02:19:02.958 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T02:19:02.974 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T02:19:02.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:19:02.976 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:19:02.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:19:03.090 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T02:19:03.100 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T02:19:03.100 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:19:03.103 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:19:03.105 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:19:03.116 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:19:03.118 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:19:03.119 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:19:03.121 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:19:03.121 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-21T02:19:03.121 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +23us[ +28us] +/- 48ms 2024-09-21T02:19:03.122 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -670us[ -665us] +/- 68ms 2024-09-21T02:19:03.122 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +759us[ +759us] +/- 59ms 2024-09-21T02:19:03.122 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:19:03.122 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:19:03.123 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-21T02:19:03.123 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +50us[ +53us] +/- 48ms 2024-09-21T02:19:03.123 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -635us[ -635us] +/- 68ms 2024-09-21T02:19:03.123 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 107 +769us[ +771us] +/- 78ms 2024-09-21T02:19:03.123 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:19:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:19:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-09-21T02:19:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -48us[ -24us] +/- 48ms 2024-09-21T02:19:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -709us[ -685us] +/- 68ms 2024-09-21T02:19:03.124 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 44 +750us[ +774us] +/- 59ms 2024-09-21T02:19:03.125 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:19:03.125 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T02:19:03.134 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T02:19:03.135 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T02:19:03.144 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:19:03.242 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:19:03.308 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:19:03.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:19:03.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:19:03.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:19:03.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:19:03.428 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:19:03.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:19:03.456 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T02:19:03.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T02:19:03.476 INFO:teuthology.task.internal:Duration was 1906.102386 seconds 2024-09-21T02:19:03.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T02:19:03.486 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T02:19:03.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:19:03.489 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:19:03.491 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:19:03.525 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:19:03.527 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:19:03.535 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:19:03.873 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T02:19:03.873 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-21T02:19:03.873 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:19:03.904 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-21T02:19:03.904 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:19:03.938 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-09-21T02:19:03.939 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:19:03.960 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T02:19:03.960 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:19:03.962 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:19:03.981 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:19:04.027 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T02:19:04.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:19:04.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:19:04.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:19:05.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T02:19:05.241 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T02:19:05.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:19:05.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:19:05.295 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:19:05.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T02:19:05.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:19:05.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:19:05.337 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:19:05.355 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-21T02:19:05.361 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-21T02:19:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-09-21T02:19:05.436 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:19:05.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:19:05.451 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:19:05.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:19:05.466 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:19:05.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:19:05.491 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T02:19:05.501 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T02:19:05.502 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi092 2024-09-21T02:19:05.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:19:05.561 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi099 2024-09-21T02:19:05.561 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:19:05.597 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913401/remote/smithi136 2024-09-21T02:19:05.597 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:19:05.632 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T02:19:05.632 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:19:05.634 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:19:05.637 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:19:05.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T02:19:05.695 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T02:19:05.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T02:19:05.707 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T02:19:05.708 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:19:05.710 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:19:05.712 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:19:05.722 INFO:teuthology.orchestra.run.smithi092.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:19 /home/ubuntu/cephtest 2024-09-21T02:19:05.725 INFO:teuthology.orchestra.run.smithi099.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:19 /home/ubuntu/cephtest 2024-09-21T02:19:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:19 /home/ubuntu/cephtest 2024-09-21T02:19:05.739 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T02:19:05.750 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T02:19:05.775 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} duration: 1906.1023864746094 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T02:19:05.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:19:05.884 INFO:teuthology.run:pass